Files
Brooke BallhausandLorenze Jay 5d58569ae2 feat: add offline jev routing and guardrail evaluation runner (#8230)
* feat(tools): add offline jev routing and guardrail evaluation runner

* fix(tools): record malformed jev responses as failed and split jsonl on newlines only

* docs(tools): note jev evaluation differs from runtime guardrail outcomes

* fix(tools): fail jev answers whose type does not match the question

* fix(tools): fail jev choice answers with non-mapping probabilities

* fix(tools): keep falsy non-mapping jev probabilities so they fail validation

* refactor: move jev evaluation runner to scripts/

---------

Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com>
2026-10-21 18:31:08 +00:00
..