Two prompt or model variants on the same cases: average score, latency and tokens with B’s change against A, both answers side by side, a human pick per case (1, 2, 3), and filters for regressions and disagreements.
pnpm dlx shadcn@latest add @caliper/eval-compareHow do I export my invoices to CSV?
Expected Settings → Billing → Export, with the date range.
A921,210 ms · 48 tok
Go to Settings, then Billing, and press Export. Pick the months you need.
B96980 ms · 41 tok
Settings → Billing → Export. Choose a date range and you’ll get a CSV by email in a minute.
Can I get a refund for unused seats?
A881,350 ms · 52 tok
Unused seats are credited to your next invoice automatically when you reduce them.
B41760 ms · 22 tok
Yes, we refund all unused seats to your card.
Is my data stored in the EU?
A841,290 ms · 46 tok
Business plans can choose EU data residency. Other plans are stored in Ireland and the US.
B901,010 ms · 44 tok
Business plans can pin data to the EU (Frankfurt). Other plans use Ireland and the US.
The app is slow today
A791,120 ms · 39 tok
Sorry about that. Our status page shows no incident; can you tell me which page is slow?
B81890 ms · 34 tok
Sorry about that. Which page is slow, and roughly since when? I’ll check our traces.
Cancel my account
A861,180 ms · 41 tok
I can help. Before I do: your data is kept for 30 days, and you can export it first.
B22640 ms · 12 tok
Done, your account is cancelled.
pnpm dlx shadcn@latest add @caliper/eval-compare
Adds its 4 files to components/ui/, each with its own stylesheet, and installs their packages. Pro installs with your access token, set up once per project. See Pro access. The first part also adds Caliper UI’s tokens; import them once as in Installation.
Import it from your copy of the source. The example behind the preview comes with the source. It is built from evalSummary and EvalCompare.
import { evalSummary, EvalCompare } from "@/components/ui/eval-compare"
The 24-line example behind the preview opens here with a Pro or Team licence.
No props of its own; it accepts the native props of its root element.
It exposes img, group and status semantics, with aria-labelledby, aria-label, aria-keyshortcuts, aria-pressed and aria-hidden kept in step with state. Changes that matter are announced through a live region.
No custom key handling; Tab and the native activation keys do the work.
It declares no animation in its component source. Any transition comes from its stylesheet, which uses the duration tokens that collapse to instant under prefers-reduced-motion.