The hallucination was not the model. It was us.

A subject line that lied

Three weeks into building this, one of our test digests went out with this subject:

Airtable Raises Prices, Notion's Turn Next?

The user was tracking Notion. They were not tracking Airtable. Airtable hadn't raised prices. The model had fabricated the entire frame.

The temptation was to attribute this to the LLM. Bigger model. Better model. RLHF. Constitutional AI. It's tempting because it absolves the system designer.

It's also wrong.

What we found

We took the same model (Workers AI's Llama-3.1-8B) and added two things:

1. A hard rule in the system prompt: "You may ONLY reference these companies by name: {tracked list}. Never name any other company."
2. A post-generation validator: if the proposed subject line doesn't contain at least one tracked brand or domain (case-insensitive substring match), reject it. Fall back to a deterministic template.

The hallucinations stopped. Not "decreased" — stopped. We have not seen an untracked-brand mention in a subject line since.

What this tells us

The model wasn't lying. The model was doing what every well-trained completion model does: pattern-matching to the most likely-looking subject line for a competitor-intelligence newsletter. Subject lines like "Airtable Raises Prices, Notion's Turn Next?" rank high in the latent space of plausible newsletter subjects, because the training corpus is full of them.

Without a constraint, the model picks the most plausible completion. With a constraint — and with a validator that rejects violations rather than trying to softly steer them — the model picks the most plausible completion within the allowed space. Which is exactly what we wanted.

The general lesson

We've now run this experiment in three places: subject lines (above), takeaways (must reference reader's own positioning), and themes (evidence IDs must exist in the candidate signal list). In every case the model is fine when the prompt explicitly forbids the failure mode AND a downstream validator enforces it.

Without the validator, the model occasionally cheats — because LLMs are completion engines and "plausible" lives in tension with "true."

If you're shipping agent-composed content for real readers, the boring engineering reality is: every claim the model can make about your data has a verifiable shape. If you can write a validator for that shape, you can make hallucinations not just rare but structurally impossible.

The work is not picking a better model. The work is enumerating the shapes that matter and writing the validators.