agents
Safe-handoff: the line between automation and triage
An agent that 'tries to be helpful' on a question it has no business answering is worse than no agent at all. Safe-handoff is a first-class action, not a fallback.
The single design decision that determines whether an agent gets to keep running in a regulated environment is how it handles questions outside its scope. The default failure mode, “be helpful, give it your best shot”, is what gets agents pulled.
Safe-handoff is the alternative. The agent knows what it’s allowed to do. For everything else, it escalates to a human. Cleanly, immediately, with the full context attached.
What “first-class” means here
In most agent designs, escalation is a fallback: what happens when the model’s confidence drops, or when a tool call fails. We build it the opposite way. Escalation is one of the agent’s primary actions, on the same level as “answer” or “look up”. The agent’s prompt enumerates the allowed moves and treats everything else as an automatic escalate.
For a healthcare intake agent, the allowed moves might be:
- Capture symptoms (record what the patient says verbatim)
- Schedule an appointment
- Capture insurance information
- Confirm consent
- Escalate to clinician
Notice what’s NOT in the list: triage, dosing, severity assessment, emergency advice. None of those are graceful failure modes. They are allowed-list omissions: the agent literally has no path to producing them. Any input that pushes toward those moves triggers escalation.
The classifier in front
Safe-handoff is too important to leave to the model alone. We put a deterministic classifier in front of every user turn. It runs in milliseconds and looks for red flags: chest pain, suicidal ideation, severe bleeding, pregnancy red-flags, paediatric red-flags, mentions of medication overdose. If any fire, the model is bypassed entirely. The classifier pages the on-call clinician and the user gets a clear “we’re connecting you to a clinician now” message.
This combination of allowed-list prompt plus deterministic classifier gives two layers of protection. The model can fail at categorising a question; the classifier catches the worst categories before the model sees them. The classifier can miss; the allowed-list keeps the model from saying something it shouldn’t.
What this gets the team
Compliance teams sign off on safe-handoff agents. They don’t sign off on “be helpful, with a footnote that this isn’t medical advice”. The difference is whether the agent’s architecture makes the wrong answer impossible or merely unlikely.
The cost is that safe-handoff agents are less impressive in demos. They say “I can’t help with that” more often than the demo crowd would like. That’s the trade. Agents that get to keep running are agents that say no to the right questions.