Ask Rusty no longer returns blank answers
Shipped 2026-09-30
In its first five weeks the new Ask Rusty returned a blank answer to about 4% of questions, mostly detailed skill questions from trainers and examiners. None of them were recorded as failures.
Why answers came back blank
The assistant may call tools for up to four rounds per question. The skill list in its prompt named every skill but not its id, so to fetch a skill’s detail it guessed an id, missed, then searched for the real one. On some questions that used all four rounds without writing anything, and the service returned the text so far: nothing.
A smaller set finished normally but with no text at all (a refusal, for example). Those were logged with a blank answer and no error, so reporting counted them as unanswered (a content gap) rather than failed.
What changed
-
Skill ids are in the prompt. The assistant calls
get_skilldirectly. Locally, skill questions went from three or four tool calls to one. The id is shared across languages, so French and Spanish members still get the translated skill. -
Running out of rounds forces an answer. One last call with tools disabled makes the assistant answer from what it has already found.
-
An empty answer is a logged failure. The member sees a short “please rephrase” message and the row records
empty answer (stop_reason: …), which the admin report counts as failed. -
No run-on text. Text written before a tool call is kept as its own paragraph instead of running into the answer (“Let me check.I don’t…”), and the assistant is told not to announce lookups at all.
-
Jira: TUN-880
-
PR: #950
-
Author: @alexhobday