Eval results for tickets 04-07; grader ignores bot-run skips; resolve tickets 01-07
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
258bfbb500
commit
312742cfef
17 files changed
+2546
-19
No files matched your search
@@ -12,7 +12,7 @@ Cost comes from a pricing table that records its source and its as-of date. Opti
|
||||
|
||||
**Blocked by:** 05
|
||||
|
||||
**Status:** ready-for-agent
|
||||
**Status:** resolved
|
||||
|
||||
- [ ] Cost per call = input tokens × input rate + (output + thinking tokens) × output rate, per million tokens. Long-context rates apply above the threshold.
|
||||
- [ ] An unknown model raises an error that names the known models. Cost never silently defaults to zero.
|
||||
@@ -20,3 +20,7 @@ Cost comes from a pricing table that records its source and its as-of date. Opti
|
||||
- [ ] Contract: `RoiRun{model,input_tokens,output_tokens,thinking_tokens,total_cost}` and a fetch-ROI action. (Shape from the Superbill prototype.)
|
||||
- [ ] The page check asserts that the spend tiles sum the runs, the coverage pill counts priced vs. total documents, and the month breakdown groups runs correctly.
|
||||
- [ ] The model picker and retry-on-other-model are offered as optional sections and built only when chosen.
|
||||
|
||||
## Comments
|
||||
|
||||
Resolved on branch `integration/rpa-dashboard-skills` (Oct 16, 2026). Tests: `uv run --with pytest --with playwright pytest tests` all green; skill eval results in `tests/evals/results/`.
|
||||
Reference in new issue
Block a user