A concept for the autonomous mid-cycle
Autonomous coding launched today.Here’s how you’d know when to trust it.
Threshold is the eval harness and exception router I’d want behind an autonomous inpatient coder. An agent reads each chart, cites every code to the exact sentence, and scores its own confidence. You choose where autonomy stops.
- synthetic stays, real agent runs
- 24
- synthetic stays, real agent runs
- coded autonomously at 0.80
- 46%
- coded autonomously at 0.80
- code precision when autonomous
- 95.3%
- code precision when autonomous
- over-coded when autonomous
- 0%
- over-coded when autonomous
You set the threshold. The evidence moves with it.
Drag the line. Stays spring between lanes and every metric recomputes from the recorded runs, graded against an answer key written before each chart. The number to watch is escapes: stays that needed a human but got through.
Each stay gets one confidence score from the agent. Stays at or above the line are finalized without a human. Everything below goes to a coder, with the reason it was held.
- Coded autonomously
- 46%
- 11 of 24 stays
- Code precisionOf the codes the system finalized, the share the answer key agrees with.
- 95.3%
- 95% CI 91%–99%
- DRG (MS-DRG)The payment group a hospital stay falls into, set by the main diagnosis plus any serious complications.
- 100%
- 95% CI 100%–100%
- Over-coding rateShare of finalized stays whose DRG is higher than the answer key's. The compliance number.
- 0%
- 95% CI 0%–0%
Coded autonomously
11COPD flare, respiratory failure considered and excluded
COPD · DRG 192
0.96Differs from answer key: missed Z79.51
Pulmonary embolism with leg clot
Pulmonary embolism · DRG 176
0.93E. coli UTI, many negatives
UTI · DRG 690
0.93Routine community-acquired pneumonia
Pneumonia · DRG 195Walkthrough
0.92Differs from answer key: +Z88.0
Pneumonia with a history of blood clots
Pneumonia · DRG 195
0.92Severe sepsis from pneumonia with kidney injury
Sepsis · DRG 871
0.90Differs from answer key: +E86.9 · missed Z79.82
Pulmonary embolism with respiratory failure
Pulmonary embolism · DRG 175
0.90Differs from answer key: missed Z79.84
Pneumonia, consultant names respiratory failure
Pneumonia · DRG 193
0.88Differs from answer key: +J18.1 · missed J18.9
COPD flare with low sodium
COPD · DRG 191
0.88Influenza A with pneumonia
Pneumonia · DRG 194
0.88Differs from answer key: +J10.08 · missed J10.00
E. coli UTI with metabolic encephalopathy
UTI · DRG 689
0.80
Sent to a coder
13Pneumonia with a late CT addendum
Pneumonia · DRG 193Walkthrough
0.60Kidney injury from dehydration after a stomach bug
Kidney injury · DRG –
0.60Heart failure, unaddressed low sodium
Heart failure · DRG 293
0.48Heart failure, pressure ulcer during stay
Heart failure · DRG 293
0.47Severe E. coli sepsis, shock explicitly absent
Sepsis · DRG 871
0.45Heart failure, cardiology specificity, unnamed creatinine rise
Heart failure · DRG 291Walkthrough
0.44AKI with malnutrition only in dietitian note
Kidney injury · DRG 684
0.44Pneumonia in a patient with COPD
Pneumonia · DRG 194
0.40Resolved AKI, copied-forward progress notes
Kidney injury · DRG 684
0.33UTI with sepsis ruled out at discharge
UTI · DRG 689Walkthrough
0.25AKI vs. baseline CKD, consultant disagrees
Kidney injury · DRG –
0.23Probable sepsis from pneumonia
Sepsis · DRG 871
0.17Sepsis on paper, thin clinical support
Sepsis · DRG 871
0.07
n = 24 synthetic stays. Intervals are 95% case-level bootstrap. With this few cases they are wide on purpose: the point is the method, not the number.
Four charts worth watching
Each replays the real agent trace: every tool call, every rejected quote, every code the critic removed.
Agent held it for a coder · 0.60
Pneumonia with a late CT addendum
A woman admitted for pneumonia was first told her CT showed no blood clot; a radiologist's next-day addendum found one, but the discharge summary still repeats the old 'no clot' line.
Replay the trace →
Agent held it for a coder · 0.25
UTI with sepsis ruled out at discharge
A nursing-home resident with a urinary infection was called septic and encephalopathic by one doctor, but the discharging doctor wrote that sepsis was ruled out and her thinking was at baseline, so a coder has to ask before coding.
Replay the trace →
Agent held it for a coder · 0.44
Heart failure, cardiology specificity, unnamed creatinine rise
A man with a weak heart and kidney disease was admitted for fluid overload; only the cardiologist spelled out the type of heart failure, and his kidney numbers rose mid-stay without any doctor naming a diagnosis.
Replay the trace →
Agent coded it autonomously · 0.92
Routine community-acquired pneumonia
A straightforward pneumonia admission in a 58-year-old man with stable, well-documented chronic conditions; nothing tricky.
Replay the trace →
Models where judgment is needed. Code everywhere else.
- 01code
Triage
Orders the documents, flags addenda and the discharge summary.
- 02Haiku 4.5
Evidence
Pulls every relevant statement as a verbatim quote. Quotes that aren't in the chart are rejected.
- 03Sonnet 5.5 + tools
Coder
Looks codes up in the real FY2027 code set. Non-billable or invented codes are sent back.
- 04Sonnet 5.5
Critic
Tries to refute every code like a payer auditor. Can remove codes and lower confidence, never raise it.
- 05code
Grouper + router
DRG is arithmetic, not a model output. One confidence per stay decides who finalizes it.
- 06Sonnet 5.5 + linter
Query
Drafts a neutral provider query for held stays. A linter blocks leading or money language.
Does the agent earn its cost?
Same charts, one prompt, no tools.
A single-prompt baseline on the same model reads each chart once and returns codes. Same answer key, same threshold. The agent costs more per stay. The evals page shows exactly what that buys.
See the evals| At 0.80 | Agent | Baseline |
|---|---|---|
| Code precision | 95.3% | 92.3% |
| DRG match | 100% | 100% |
| Escapes | 0 | 1 |
| Invalid codes | 0.0% | 0.0% |
| Cost per stay | $0.096 | $0.011 |