# Check LLM output against a written policy without another LLM as the judge

How to check extracted values against a written policy with no model in the loop — extraction on your model, the check on a solver, and what that buys you.

The common pattern for checking an LLM's work is a second LLM: a judge prompt, a rubric, a score. It inherits every property of the first model — variance, prompt sensitivity, an explanation written after the fact. DSAIL keeps the model where it is good, at reading the document, and puts no model in the loop for the check itself.

## The seam

```text
document ──(your model, prompt pack)──► claim dictionary ──(solver)──► per-assertion results
```

- **Left of the seam** is yours: your model, your credentials, your document.
  The service generates the prompt pack — one question per claim, with the
  answer format, vocabulary and unit — and never sees what you run it on.
- **Right of the seam** is the service: the compiled ruleset, solved over the
  claim values by an SMT solver in exact arithmetic. Deterministic. No language
  model is called, at compile or at check.

What crosses the wire is the claim dictionary — a handful of typed values
bounded by a JSON Schema — never the document.

## What this buys

- **The same answer every time.** Same ruleset, same values, same bytes. There
  is no temperature to set and no judge to re-prompt.
- **An explanation that is the reason, not a story.** A `FALSE` carries the
  rule that decided, with a counterexample: the values under which the
  assertion fails, found by the solver.
- **Honest gaps.** A fact your model could not find is submitted as `"unknown"`
  and the rule that needed it answers `UNKNOWN`. Unknown is an answer, not a
  guess, and a judge model tends to fill it in.
- **A smaller evaluation surface.** The only fuzzy step left is extraction —
  did the model read the amount correctly? — and that is a narrow, testable
  question with a ground truth, unlike "was the judgement right?"
- **Nothing to leak.** The service holds rules, not documents.

## What it does not do

It does not assess tone, relevance, toxicity or grounding; those are content
properties, and [guardrail frameworks](../compare/guardrails.md) are built for
them. It checks whether the **facts** extracted from a document satisfy a
written policy. Many applications need both, on different layers.

## Fitting it into a pipeline

1. Compile the policy once; keep the `ruleset_hash`.
2. For each document: fetch the prompt pack (or cache it per hash), run the
   prompts on your model, assemble the claim dictionary, `check`.
3. Read `rules[].assertions[].check` and act on it in your own code. The
   service publishes no verdict, because what a `FALSE` costs is your decision.
4. If validation rejects the dictionary, `error.failures` names every failing
   field at once — fix them in one pass and resubmit. The `dsail` Python
   client's `check_with_repair` does this loop for you.

## Try it

- [Quickstart: REST](../quickstart/rest.md) — the three calls.
- [Quickstart: Claude Code](../quickstart/claude-code.md) — a coding agent wiring the check into a service.
- [Turn a policy document into rules a program can check](policy-to-rules.md) — the language.