aquin.compute.sae_catalog_metrics Namespace Reference

Functions
list[dict[str, Any]]	compute_umap_points_from_sae (str|Path sae_path)
list[tuple[int, float, str]]	_collect_top_features (*, str model_id, int layer, list[str] prompts, int n_per_prompt, int max_features)
dict[str, Any]	compute_layer_interp_score (*, str model_id, int layer, str|Path sae_path, list[str]|None prompts=None, int n_features=8, int n_per_prompt=5, int n_samples=8, Any|None client=None)
tuple[Path, Path]	write_catalog_sidecars (*, str|Path sae_path, list[dict[str, Any]] umap_points, dict[str, Any] metrics)
dict[str, Any]	compute_and_write_catalog_sidecars (*, str model_id, int layer, str|Path sae_path, bool skip_interp=False, **Any interp_kwargs)

Variables
list	CATALOG_PROBE_PROMPTS

Function Documentation

_collect_top_features()

list[tuple[int, float, str]] _collect_top_features ( * , str model_id, int layer, list[str] prompts, int n_per_prompt, int max_features )

protected

Definition at line 50 of file sae_catalog_metrics.py.

Referenced by compute_layer_interp_score().

compute_and_write_catalog_sidecars()

dict[str, Any] compute_and_write_catalog_sidecars	(	*	,
		str	model_id,
		int	layer,
		str | Path	sae_path,
		bool	skip_interp = False,
		**Any	interp_kwargs )

Definition at line 204 of file sae_catalog_metrics.py.

References compute_layer_interp_score(), compute_umap_points_from_sae(), and write_catalog_sidecars().

compute_layer_interp_score()

dict[str, Any] compute_layer_interp_score	(	*	,
		str	model_id,
		int	layer,
		str | Path	sae_path,
		list[str] | None	prompts = None,
		int	n_features = 8,
		int	n_per_prompt = 5,
		int	n_samples = 8,
		Any | None	client = None )

Mean InterpScore over top response features from catalog probe prompts.

Definition at line 95 of file sae_catalog_metrics.py.

References _collect_top_features().

Referenced by compute_and_write_catalog_sidecars().

compute_umap_points_from_sae()

list[dict[str, Any]] compute_umap_points_from_sae

(

str | Path

sae_path

)

3D UMAP of decoder rows (same recipe as bridge._compute_umap_points).

Definition at line 27 of file sae_catalog_metrics.py.

Referenced by compute_and_write_catalog_sidecars().

write_catalog_sidecars()

tuple[Path, Path] write_catalog_sidecars	(	*	,
		str | Path	sae_path,
		list[dict[str, Any]]	umap_points,
		dict[str, Any]	metrics )

Definition at line 190 of file sae_catalog_metrics.py.

Referenced by compute_and_write_catalog_sidecars().

Variable Documentation

CATALOG_PROBE_PROMPTS

list aquin.compute.sae_catalog_metrics.CATALOG_PROBE_PROMPTS

Initial value:

=  [
    "The capital of France is",
    "In quantum mechanics, particles can",
    "The recipe calls for two cups of",
    "She opened the letter and read",
    "function merge_sort(arr):",
]

Definition at line 18 of file sae_catalog_metrics.py.