Service 04
Testing and reliability
We build the tests and gates that make an AI system trustworthy in production. Evals are tests for a large language model's answers. Every change is graded. One that makes the system worse goes back instead of shipping.
What it includes
- A test suite per agent: known answers, plus repeat runs to catch drift
- A hard gate: precision, leak, and wrong-value thresholds a change has to clear
- A checker that blocks an answer it cannot trace, and says so
What you get
- A repeatable eval suite wired into your change process
- A gate that blocks a regression from reaching users
- A measured honesty baseline you can hold the system to
Proof
Where this ran
Delivered systems that used this service. Every figure carries the limit of what it measures.
A $3.4B US real-estate investment firm
Thirteen years of records. One box that answersRead the case1.000Correct every time it held backOne production system, at every release. It does not measure how often it should have refused.Ilayer research
AI that reviews code the way an adversary wouldRead the case7-gateEvery finding filtered before a human sees itOur own system, not a client's. The checks filter findings out; they do not prove one is exploitable.
Questions
What buyers ask about this
Why do evals matter for an AI product?
Without them you cannot tell whether a change made the system better or quietly worse. Evals turn that from a guess into a number you can gate on.
What does a hard gate check?
On one production system the gate holds four thresholds. Abstain precision at 1.000. Zero leaks of one client's data into another's answers. Zero wrong answers on the known-answer set. And the same answer on every run.
Can you add evals to a system we already have?
Yes. We wrap an existing agent or RAG pipeline with an eval battery and a gate without rebuilding it. RAG is short for retrieval-augmented generation.
