Explanation
The mental model
How Kno thinks about data as an experimental variable, the one idea that makes every stage and number fall into place.
Traditional evals answer one question: did my agent get better? They vary the model, the prompt, or the tools, and watch a score.
Kno asks a different question: which data made it better?
Traditional eval
Prompt ─────┐
Model ──────┼──► Agent ───► Score
Tools ──────┘
Kno
┌─ Document A ──► +12%
├─ Example B ───► +8%
Agent + Evals ──────┼─ Policy C ────► 0%
└─ Asset D ─────► -3%
│
▼
impact × cost × destination
Data is the variable
A document, an example, a policy, a conversation, each is a candidate asset. Kno treats each one as a variable in a controlled experiment: measure the agent without it, inject it, measure again, compare against fresh controls, and report the delta with its uncertainty.
The agent, the evals, and the goal stay fixed. Only the data changes. That is what makes the result attributable.
The stages
| Stage | Question | Status |
|---|---|---|
| Init | How is this run set up? | Shipped |
| Mine | Where do the evals come from? | Shipped |
| Baseline | How good is the agent now? | Shipped |
| Value | Which assets improve it? | Shipped |
| Select | Which combination should I keep? | Shipped |
| Validate | Does the combination still work on untouched evals? | Planned |
| Export | Where should each asset go? | Shipped |
| Report | What does the whole run say? | Shipped |
The holdout is the discipline
A slice of your cases is sealed at baseline and nothing reads it until validate. Without that separation, selection could silently overfit your dev set, and every number after it would be inflated.
The full argument, including what is deliberately out of scope: DESIGN.md and the mental model page.