aquin.eval_check Namespace Reference

Functions
dict[str, Any]	_normalize_eval_result (dict[str, Any] result)
bool	has_eval_payload (dict[str, Any] result)
tuple[Path, Path]	write_eval_check (dict[str, Any] result, *, str|None tool_name, str|Path cwd)
None	_plot_eval_check (dict[str, Any] result, Path png_path)
None	_plot_prompt_scores (Any ax, list[dict[str, Any]] rows, float threshold)
None	_plot_summary_panel (Any ax, float mean_score, float pass_rate, float threshold, list[dict[str, Any]] rows)
str	_score_color (float val)
str	_trunc (str text, int limit)

Variables
str	_JSON_NAME = "eval-check.json"
str	_PNG_NAME = "eval-check.png"

Function Documentation

_normalize_eval_result()

dict[str, Any] _normalize_eval_result ( dict[str, Any] result )

protected

Definition at line 19 of file eval_check.py.

Referenced by has_eval_payload(), and write_eval_check().

_plot_eval_check()

None _plot_eval_check ( dict[str, Any] result, Path png_path )

protected

Definition at line 64 of file eval_check.py.

References _plot_prompt_scores(), and _plot_summary_panel().

Referenced by write_eval_check().

_plot_prompt_scores()

None _plot_prompt_scores ( Any ax, list[dict[str, Any]] rows, float threshold )

protected

Definition at line 107 of file eval_check.py.

References _trunc().

Referenced by _plot_eval_check().

_plot_summary_panel()

None _plot_summary_panel ( Any ax, float mean_score, float pass_rate, float threshold, list[dict[str, Any]] rows )

protected

Definition at line 133 of file eval_check.py.

References _score_color().

Referenced by _plot_eval_check().

_score_color()

str _score_color ( float val )

protected

Definition at line 169 of file eval_check.py.

Referenced by _plot_summary_panel().

_trunc()

str _trunc ( str text, int limit )

protected

Definition at line 177 of file eval_check.py.

Referenced by _plot_prompt_scores().

has_eval_payload()

bool has_eval_payload

(

dict[str, Any]

result

)

Definition at line 36 of file eval_check.py.

References _normalize_eval_result().

write_eval_check()

tuple[Path, Path] write_eval_check	(	dict[str, Any]	result,
		*	,
		str | None	tool_name,
		str | Path	cwd )

Definition at line 43 of file eval_check.py.

References _normalize_eval_result(), and _plot_eval_check().

Variable Documentation

_JSON_NAME

str aquin.eval_check._JSON_NAME = "eval-check.json"

protected

Definition at line 15 of file eval_check.py.

_PNG_NAME

str aquin.eval_check._PNG_NAME = "eval-check.png"

protected

Definition at line 16 of file eval_check.py.