Use case

Is this policy actually improving my agent?

Policies shape every answer an agent gives. Measure a proposed policy change against real cases before, and after, it goes live.

The problem

Policies are cheap to write and expensive when wrong: one bad policy degrades every answer in its domain. Teams ship policy changes on judgment because there was no fast way to measure them.

The workflow

1. Baseline

   Score the agent under the current policy.

2. Value

   Inject the proposed policy into the cases it affects and measure the delta with a confidence interval.

3. Select

   Keep it if it pays at a Bonferroni-corrected interval; reject with a reason if it regresses or does nothing.

4. Export

   Ship the kept policy as part of a context pack.

What it looks like

ASSET              DELTA (95% CI)             CONTROL    NOTE
refund-policy-v3   +0.4000 [-0.1000, 0.9000]  low 0.0000

The recipe

Put the proposed policy and the current one in the same pool. Kno measures each against the same baseline; the comparison falls out of the value table.

Stages involved

• baseline
• value
• select
• export