The model can read the invoice and the bill can still be wrong
A practical failure taxonomy for invoice automation: separate document extraction quality from arithmetic, duplicate, identity, and accounting controls.
Read the field noteField notes
Lessons from work that had to hold up.
Practical notes on agent evaluation, human review, and the decisions that sit between a promising model and useful software.
LightGBM missing values are a deployment contract
Why generated LightGBM inference code must preserve each node’s missing-value direction—and how to test parity across Python, C++, and JavaScript.
Read the field note
Tool-call correctness has four surfaces
A practical framework for evaluating agent tool choice, arguments, execution outcomes, and final responses without collapsing them into one score.
Read the field note
A response-only eval missed 176 of 263 broken agent runs
A reproducible comparison of response-only, trace-only, and trace-plus-outcome evaluators against the same 263 controlled agent failures.
Read the field note
Human review stops working when everything needs review.
Every review consumes a person's attention. A practical way to decide which AI actions should stop, what the reviewer needs to see, and where the software's authority ends.
Read the field noteYour agent eval passed. Would it catch a broken tool call?
Plant one known failure, rerun the same evals, and see whether the score changes. A practical introduction to mutation testing for agent tool use.
Read the field note