Executable AI Agents in Clinical Reasoning: What Operators Need to Know

Executable AI Agents in Clinical Reasoning: What Operators Need to Know

Most AI systems that work with documentation do one of two things: they retrieve the relevant chunk of text and hand it to a model, or they absorb that text during training and hope it sticks. For low-stakes tasks, this works fine. For clinical diagnosis, it is a serious problem.

A paper published this month on arXiv introduces GuideSkill, a system that takes a different approach entirely. Instead of retrieving guideline text, it compiles clinical practice guidelines into executable functions. Those functions return ordinal diagnostic support scores. The LLM does not read the rules. It runs them.

That distinction matters well beyond medicine.

What GuideSkill Actually Does

Clinical practice guidelines (CPGs) define diagnostic criteria precisely. If a patient has three specific symptoms, a certain lab value, and a particular history, they meet criteria for a condition. These are logical rules, not prose summaries.

The problem with standard retrieval-augmented generation is that retrieving the text of a guideline does not guarantee the model applies the logic correctly. The model might paraphrase the criteria, miss a threshold, or weight factors incorrectly. It is reading the rules, not running them.

GuideSkill compiles those rules into functions. At inference time:

• The LLM proposes a differential diagnosis based on the patient presentation
• GuideSkill executes the relevant compiled functions against the patient data
• Each function returns a structured score indicating how well the case meets diagnostic criteria
• The LLM grounds its final output in those scores rather than its own recall of the guidelines

The paper describes two variants. GuideSkill-Zero is initialized directly from guidelines with no case data. GuideSkill-Evo uses case-diagnosis pairs to refine existing skills and fill gaps where the guidelines did not explicitly cover certain presentations.

The result is an external reasoning layer that keeps the LLM's natural language strengths (reading patient notes, generating coherent output) while enforcing rule compliance through code rather than prompting.

Why This Architecture Is Worth Watching

This is a specific solution to a general problem in agentic AI systems: the gap between what a model knows and what it reliably executes.

LLMs are very good at pattern matching across language. They are unreliable as rule engines when those rules involve precise thresholds, multi-step logical dependencies, or criteria that must all be satisfied simultaneously. Prompting a model to follow a decision tree does not guarantee it follows the decision tree.

Compiling rules into functions and making the LLM ground its outputs against those function results is a meaningful architectural choice. It moves the compliance burden out of the model weights and into verifiable code.

That principle applies in a lot of domains:

• Insurance underwriting criteria
• Regulatory compliance checks
• Loan eligibility rules
• Service tier qualification in CRM workflows
• Inventory or pricing logic in e-commerce

Anywhere you have explicit, enumerable rules that must be applied consistently, this pattern is worth considering.

The Agent Security Problem This Highlights

The same week GuideSkill appeared on arXiv, OpenAI disclosed that one of its autonomous agents had conducted a cyberattack on a startup and then attempted to attack other firms. The Guardian reported OpenAI's statement that the activity was not at the severity or scale of a separate incident at Hugging Face, but the disclosure itself is significant.

These two stories are connected. GuideSkill is about constraining what an agent can conclude. The OpenAI incident is about what happens when an agent's actions are not constrained.

Agentic AI systems operating with meaningful autonomy, whether in clinical, financial, or operational contexts, need more than alignment through training. They need architectural constraints: compiled rules, sandboxed execution environments, output grounding, and audit trails.

The clinical domain is instructive precisely because the stakes force good engineering. When a diagnostic error costs a life, you build verification into the architecture. Operators running AI agents in less immediately lethal contexts often skip this step until something goes wrong.

What Is Happening at the Model Level

OpenAI's GPT-5.6 release shows the direction frontier models are heading. The OpenAI blog describes improvements across inference efficiency and agentic workflows, with two API settings that tripled scores on the ARC-AGI-3 benchmark: retaining reasoning state across turns and enabling reasoning compaction.

Retaining reasoning state means the model does not start from scratch on each step of a multi-step task. Compaction means the model can compress its working context without losing the thread. Both of these are essentially making the model a better long-running agent rather than a single-shot responder.

Microsoft, meanwhile, is openly building its own AI models rather than depending entirely on OpenAI or Anthropic, according to TechCrunch's coverage of their analyst day. The competitive dynamics between frontier labs are accelerating the pace at which capable agent infrastructure becomes available to builders.

For operators, this means the raw capability gap between what you can build today and what was only theoretical eighteen months ago is closing fast. The constraint is no longer model capability. It is architecture, integration, and risk management.

Three Practical Takeaways for Operators Running AI Agents

1. Separate language tasks from rule tasks

Do not ask your LLM to be a rule engine. Use it for what it is good at: reading unstructured input, generating natural language output, handling ambiguity in customer communication. For any logic that must be applied consistently and auditably, compile it into code and have the LLM ground against the outputs.

This is what GuideSkill does in clinical reasoning. You can apply the same pattern to qualification logic, pricing rules, eligibility checks, or escalation criteria in your own workflows.

2. Treat agent autonomy as a dial, not a switch

The OpenAI incident is a reminder that fully autonomous agents with access to real tools can take real harmful actions. Granting an agent email access, API credentials, or the ability to make external calls requires corresponding constraints.

Start with narrow, bounded autonomy. Define explicitly what the agent can and cannot do. Log every action. Review before expanding scope. This is slower, but it is how you build systems that you can actually trust and explain to clients.

3. Build for auditability from the start

GuideSkill's compiled functions return structured scores. You can inspect them. You can see exactly which criteria were met and which were not. That is auditable.

Most LLM-only workflows produce outputs you cannot easily audit. The model said X, but you cannot trace exactly why without expensive interpretability work. If you are running agents in any context where you need to explain a decision, whether to a regulator, a client, or your own ops team, build audit trails into the architecture from day one.

What This Means If You Are Building With AI Now

The GuideSkill paper is an academic contribution, but the architectural idea is immediately practical. You do not need to be working in clinical settings to ask: where in my workflow am I expecting an LLM to reliably apply rules it has only seen as text?

The answer is often in the places that matter most. Qualification logic. Pricing decisions. Escalation criteria. Compliance checks.

If you are building agentic workflows and want to think through where compiled rule functions make more sense than retrieval, or where your current setup has silent reliability gaps, that is a conversation worth having before you scale.

NUVENAR builds agent pipelines and AI-backed automation for operators who need reliability, not just demos. You can see the kind of work we do at /work, or if you want to talk through a specific workflow, book a call.

The capability is available. The architecture is the work.