A little judgment inside a CLI
Sometimes the hard part of a script is deciding which question to ask next.
If a service is unhealthy, a script can collect facts exactly. Interpreting those facts can be messier: does the evidence point to storage, networking, or neither? I use a local model for that choice, then give ordinary code responsibility for checking the answer and selecting an allowed operation.
- EvidenceObserved facts and a short list of options
- JudgmentA local model proposes an option
- ValidationCode checks the response
- DispatchA named, authorized handler or an unresolved result
Try the boundary
Change the observations to explore the decision. Transparent browser rules simulate a response; this does not call a model or run a diagnostic.
Which read-only diagnostic should run first?
Initial example: A volume mount fails while DNS resolution succeeds. With scripts enabled, change the observations below.
Allowed options: storage or network. The model may also abstain.
The initial evidence suggests checking storage first.
Example response: {"label":"storage"}Code checks that the label is an allowed option, then selects the corresponding named handler. Invalid or absent labels remain unresolved.
The exact demo rules
If exactly one observed subsystem fails, select its label. If both fail, or neither has a recorded failure, abstain. An unchecked subsystem stays an evidence gap, even when the other subsystem suggests a first check. These deliberately simple rules illustrate the boundary; the real tool asks a model to interpret evidence.
A valid answer can still be wrong
Keep the initial observations and choose “Force network response.” It passes the option check, even though the evidence points toward storage. Validation keeps a choice within the program’s boundaries. It cannot prove that the model understood the situation. I need labeled examples and observed results to evaluate that part.
What the real CLI contract looks like
I pass a question, evidence, and a fixed map of options to agent-classify decide. The response contains an allowed label or null, plus abstention information. The caller checks the label and dispatches through a static map of handlers. A successful CLI exit alone does not mean it made a choice.
Generic decisions run locally. A model response cannot add permissions or provide executable code. When inference is unavailable, the tool keeps useful collected facts and follows a documented fallback or leaves the decision unresolved.
Why bother?
I can keep exact work in code while using a model for the small judgment that used to require a pile of brittle text rules. There is still a fallible decision in the middle, so the tool needs a useful way to say “I don’t know.”