Menu

KNO / OPEN SOURCELICENSE / APACHE-2.0

Know which data actually makes your AI better.

Measure which documents, examples, policies, and conversations improve your agent. Rank them by impact and cost, then put each where it belongs.

Run KnoView on GitHub ↗
curl -sSfL https://raw.githubusercontent.com/uknoAI/kno/main/install.sh | sh
  • Single Go binary
  • No infrastructure
  • Works with your existing evals
  • OpenAI-compatible endpoints
  • Anthropic
  • exec: your own agent
  • Local fake agent — free
  • Budget guarded
  • Apache-2.0
KNO / VALUERUN 0142
ASSET                  IMPACT    DECISION
new_refund_policy.md    +18%    keep → knowledge base
example_42.json         +7%    keep → context
example_91.json         +1%    reject
old_refund_policy.md    -9%    reject → harmful

Illustrative example. Real runs report the same shape — ASSET, DELTA, decision — with each delta carrying its 95% confidence interval, a control reading, and a rejection reason.

[01 / Problem]

You're measuring your models. Why aren't you measuring your data?

Agent teams route data constantly — put it in RAG, add it to context, fine-tune on it — with intuition as the decision method. Product docs, support conversations, policies, examples: lots of data, very little evidence about which of it actually helps. Kno treats data as an experimental variable and turns each of those decisions into a measured experiment.

WITHOUT KNO
  • Documents────────►RAG
  • Examples────────►Context
  • Conversations─────►Fine-tuning
  • Policies──────────►???

DECISION METHOD / INTUITION

WITH KNO
  • Data─────►KNO─────►impact
  • ├──►confidence
  • ├──►cost
  • ├──►destination
  • └──►keep / reject
[02 / Outcomes]

Know what helps. Know what hurts. Know what it costs.

  1. 01

    Know what helps

    Measure the marginal impact of each candidate asset against your own evals.

  2. 02

    Know what hurts

    Find contradictory and harmful data — and regressions other tools miss — before they reach production.

  3. 03

    Know what it costs

    Compare improvement against inference cost. Ranking is per dollar, not per point.

  4. 04

    Know where it belongs

    Context, knowledge base, or tuning set — every selected asset gets a destination, not just a score.

  5. 05

    Know what's missing

    Export reports the gaps: cases your current data cannot move, measured as GAP / UNKNOWN.

Metric samples are illustrative; real runs report each delta with its 95% confidence interval.

[03 / Evidence]

Every piece of agent data should earn its place.

A valuation answers one question per asset: keep it, drop it, or move it — with the measured delta that justifies the decision.

kno value — the free fake agentreal output
ASSET              DELTA (95% CI, positive = goal dir)  CONTROL     NOTE
brand-guide        +0.0000  [-0.1260, +0.1260]  low -0.1938 (underpowered)
refund-example-17  +0.0000  [-0.1260, +0.1260]  low -0.1938 (underpowered)
refund-policy-v3   +0.0000  [-0.1260, +0.1260]  low -0.1938 (underpowered)

The deltas read 0.0000 because the free local agent answers every case with what the case expects — the point is proving the loop: routing, injection, fresh controls, intervals. Point it at a real provider and the numbers start meaning something.

[04 / MECHANISM]

Five stages. One question each.

Kno treats data as an experimental variable. Each stage answers a single question about your agent and the assets you feed it — and each stage is recorded so the report can show its work.

  1. 01BaselineshippedHow good is the agent now?

    Run the agent over your dev cases, score against your goal, persist every result. A holdout is sealed here; nothing reads it until validate.

  2. 02ValueshippedWhich assets improve it?

    Route each asset to the slices it could affect, inject it, re-measure against fresh controls, record the delta with a confidence interval.

  3. 03SelectshippedWhich combination should I keep?

    Build a portfolio under budget with a rejection log; every keep/reject decision at a Bonferroni-corrected interval.

  4. 04ValidateplannedDoes the combination still work on untouched evals?

    Measure the portfolio as a set against the sealed holdout.

  5. 05ExportshippedWhere should each asset go?

    Render selected assets into the destination grammar: context pack, knowledge-base manifest, or tuning-set JSONL.

[05 / Use cases]

Answers to questions you already have

  1. 01Which examples deserve space in my context window?
  2. 02Which conversations should I fine-tune on?
  3. 03Which data is actively hurting my agent?
  4. 04Is this policy actually improving my agent?
  5. 05Which documents should go into my RAG system?
  6. 06Which data is redundant?
  7. 07Which knowledge sources actually help my support agent?
[06 / Differentiation]

Your eval framework measures the agent. Kno measures the data feeding it.

Kno does not replace your evals — it uses them. Existing evals tell you whether your agent got better or worse. Kno uses those evals to determine why a particular data asset changes the outcome, and whether that asset is worth keeping.

CapabilityEval frameworkKno
Model performance✓uses
Prompt performance✓uses
Asset-level impact—✓
Marginal data value—✓
Data cost—✓
Destination selection—✓
Portfolio validation—✓
[07 / Quickstart]

From zero to your first valuation

Four steps, no API keys, no spend. The default agent is a local fake that costs nothing, so you can see the whole loop work — routing, injection, controls, intervals — before pointing Kno at anything that bills you.

  1. Write some cases

    One scoreable interaction per line, each with a stable id.

    printf '{"id":"refund-01","input":"How do I get a refund?","expected":"Refunds are processed within 5 business days."}\n' > cases.jsonl

  2. Measure the agent as it is today

    The reference every later number is compared against. Kno seals a holdout here — nothing reads it until validate.

    kno baseline --evals cases.jsonl
    kno — measure the agent as it is todayoutput
    Baseline 20260821T091515-ffc3097d49da
  cases      44 scored, 0 errored (of 44 dev; 6 held back)
  score      1.000
  spent      $0.00 over 44 call(s)
  status     completed

  warning: the holdout has only 6 cases, too few for a meaningful confidence interval at validate

  3. Write a pool of candidate assets

    The documents, examples, and policies you are considering adding.

    printf '{"id":"refund-policy-v3","content":"Refunds are processed within 5 business days.","kind":"knowledge"}\n' > pool.jsonl

  4. Value them

    Each asset is injected into the slices it could affect, re-measured against controls, and reported with an interval. The deltas read zero because the fake agent answers every case with what the case expects — the point is proving the loop, not the asset.

    kno value --evals cases.jsonl --pool pool.jsonl --baseline-run-id <run id>
    kno — value themoutput
    Planning 150 measurements over 3 assets against baseline 20260821T091515-ffc3097d49da.
Value run 20260828T233124-05cda2dcdac6 (RUN_STATUS_COMPLETED)

ASSET              DELTA (95% CI, positive = goal dir)  CONTROL     NOTE
brand-guide        +0.0000 [-0.1260, +0.1260]  low -0.1938 (underpowered)
refund-example-17  +0.0000 [-0.1260, +0.1260]  low -0.1938 (underpowered)
refund-policy-v3   +0.0000 [-0.1260, +0.1260]  low -0.1938 (underpowered)

The quickstart costs nothing. Point Kno at a real provider — any OpenAI-compatible endpoint or Anthropic — with --agent and a budget cap, and the numbers start meaning something.

Terminal recording of the Kno quickstart: baseline, value, select, export, and report against the free fake agent
The whole loop against the free fake agent — no API keys, no spend.

Read the full walkthrough →

[08 / Objections]

Questions engineers actually ask

Kno is deliberately opinionated about experimentation. The short answers, from the source.

Does Kno replace my eval framework?
No — it uses it. Your evals tell you whether your agent is getting better or worse. Kno re-scores the agent with each candidate asset to measure why a particular asset changes the outcome, and whether that asset is worth keeping.
Does Kno send my data somewhere?
Runs are stored locally in SQLite — including agent output, which may be conversation content. Kno itself sends nothing anywhere: there is no telemetry of content, ever. Your cases go only to the provider you point Kno at, and that provider's retention is theirs.
What happens to traces?
They stay on your disk until you delete them, and nothing expires on its own. `kno purge` removes trace content when you no longer need it, keeping the scores and costs so the run stays resumable.
Can it accidentally spend unlimited API money?
No. Every path that can call a provider goes through a budget guard: estimate, confirm, checkpoint — and caps are enforced before the call, not discovered at settlement. A cap whose cost cannot be estimated is refused up front. `--max-cost-usd` with `--yes` makes a run unattended-safe, and an interrupted run resumes without paying twice.
Are the results statistically meaningful?
Kno reports confidence intervals, never naked point estimates — a delta without its interval is not reported at all. Controls measure regression separately, selection decisions are made at Bonferroni-corrected intervals, and small samples are flagged underpowered instead of dressed up as answers. The honest number is the holdout number.
What providers work?
`openai:` for any OpenAI-compatible endpoint (vLLM, Ollama, llama.cpp need no key), `anthropic:` for the Anthropic API, `bedrock:` for Claude on AWS, `vertex:` for Claude on Google Cloud, `exec:` for your own agent behind a shell command, and `fake:` for the local agent that costs nothing. `tuned:` arrives with the stages that need it. Keys come from the environment, never from a flag — `kno doctor` prints what this build supports and contacts nothing.
Do I need infrastructure?
No. Kno is a single Go binary. State is one local SQLite file, work checkpoints as each case completes, and CI can gate on real exit codes.
[09 / Open source]

Kno is designed to be extended

Adapters bring new data sources, judges add evaluation logic, goals add optimization targets, providers connect agent runtimes — and the core measurement loop itself is open. Pick the surface that matches how deep you want to go.

01

Adapters

Bring another data source. Langfuse, LangSmith, and Braintrust eval adapters, CSV and Markdown pools, and a full Zendesk recipe already ship — the pattern transfers to any stack that can export data.

02

Judges

Add evaluation logic so Kno can score a new kind of case.

03

Goals

Add an optimization target for the select stage.

04

Providers

Connect a new agent runtime.

05

Core

Improve the valuation engine itself — plan first, review adversarially, then code.

Find good first issuesRead CONTRIBUTING.mdAsk a questionOpen an issue
[10 / Status · v0.1.0]

Kno is early. Help define what comes next.

The measurement loop ships — baseline, value, select, export, and report are real, as is kno mine for building evals from production transcripts. Validate is next. Everything here is transparent on purpose, including the debt.

StageStatus
baseline Shipped
value Shipped
select Shipped
validate Planned
export Shipped
report Shipped

See the full status table →

Stop guessing which data makes your agent better.

Install the binary, run the quickstart, and get your first valuation in minutes — without spending API money.

curl -sSfL https://raw.githubusercontent.com/uknoAI/kno/main/install.sh | sh
Run the quickstartView on GitHub