Classes
class	ModelPhase
class	RuntimeSnapshot

Functions
RuntimeSnapshot	snapshot ()
None	_set_phase (str phase, *, str\|None model_id=None, str\|None target_id=None, Literal["llm"]\|None kind=None, str\|None error=None)
None	reset_state ()
tuple[str, Literal["llm"]]	resolve_kind (str model_id)
tuple[str\|None, Literal["llm"]\|None]	resident_from_cache ()
RuntimeSnapshot	sync_from_cache ()
tuple[str, Literal["llm"]]	begin_load (str model_id)
None	finish_load (str slug, Literal["llm"] kind)
None	fail_load (str model_id, str error)
None	begin_unload ()
None	finish_unload ()
None	request_load_cancel ()
bool	is_load_cancelled ()
None	_emit (ProgressFn\|None progress, str message)
str	load_weights (str model_id, *, ProgressFn\|None progress=None)
None	unload_weights (*, ProgressFn\|None progress=None, bool clear_active=True)
bool	release_foreign_daemon ()
str\|None	vram_line ()
dict[str, Any]	health_payload (*, bool daemon, int port)

Variables
	ProgressFn = Callable[[str, float], None]
	_lock = threading.Lock()
	_load_serial = threading.RLock()
str	_phase = "idle"
str	_target_id = None
str	_resident_id = None
Literal	_kind = None
str	_error = None
float	_started_at = None
float	_loaded_at = None
str	_load_message = None
int	_load_epoch = 0
int	_load_epoch_at_start = 0

Function Documentation

◆ _emit()

None _emit	(	ProgressFn \| None	progress,
		str	message )

protected

Definition at line 215 of file model_runtime.py.

References snapshot().

Referenced by load_weights(), and unload_weights().

◆ _set_phase()

None _set_phase	(	str	phase,
		*	,
		str \| None	model_id = None,
		str \| None	target_id = None,
		Literal["llm"] \| None	kind = None,
		str \| None	error = None )

protected

Definition at line 74 of file model_runtime.py.

Referenced by begin_load(), begin_unload(), fail_load(), finish_load(), and finish_unload().

◆ begin_load()

tuple[str, Literal["llm"]] begin_load ( str model_id )

Definition at line 166 of file model_runtime.py.

References _set_phase(), and resolve_kind().

Referenced by load_weights().

◆ begin_unload()

None begin_unload ( )

Definition at line 190 of file model_runtime.py.

References _set_phase(), and snapshot().

Referenced by unload_weights().

◆ fail_load()

None fail_load	(	str	model_id,
		str	error )

Definition at line 183 of file model_runtime.py.

References _set_phase().

Referenced by load_weights().

◆ finish_load()

None finish_load	(	str	slug,
		Literal["llm"]	kind )

Definition at line 176 of file model_runtime.py.

References _set_phase().

Referenced by load_weights().

◆ finish_unload()

None finish_unload ( )

Definition at line 195 of file model_runtime.py.

References _set_phase().

Referenced by load_weights(), and unload_weights().

◆ health_payload()

dict[str, Any] health_payload	(	*	,
		bool	daemon,
		int	port )

JSON health block for the local engine server.

Definition at line 333 of file model_runtime.py.

References resident_from_cache(), snapshot(), sync_from_cache(), and vram_line().

◆ is_load_cancelled()

bool is_load_cancelled ( )

True if unload/cancel happened after this load began.

Definition at line 209 of file model_runtime.py.

Referenced by load_weights().

◆ load_weights()

str load_weights	(	str	model_id,
		*	,
		ProgressFn \| None	progress = None )

Load model_id into this process. Returns resolved slug. Updates lifecycle state.

Process-wide serialized :  concurrent builds on MPS exhaust unified memory.

Definition at line 224 of file model_runtime.py.

References _emit(), begin_load(), fail_load(), finish_load(), finish_unload(), and is_load_cancelled().

◆ release_foreign_daemon()

bool release_foreign_daemon ( )

Optionally unload a background engine so this process can claim VRAM.

Default is **never** :  the desktop app keeps a resident model in the daemon.
Headless CLI tools that try load_model in a short-lived process used to call
model_daemon.unload() here, which looked like the model “randomly disappearing”.

Opt-in only: set AQUIN_CLAIM_DAEMON_VRAM=1 when you intentionally want an
exclusive in-process load that frees the background engine first.

Definition at line 286 of file model_runtime.py.

◆ request_load_cancel()

None request_load_cancel ( )

Mark any in-flight build as cancelled (checked after from_pretrained).

Definition at line 202 of file model_runtime.py.

Referenced by unload_weights().

◆ reset_state()

None reset_state ( )

Clear lifecycle markers (tests / daemon shutdown).

Definition at line 107 of file model_runtime.py.

◆ resident_from_cache()

tuple[str | None, Literal["llm"] | None] resident_from_cache ( )

What is actually resident in this process's VRAM caches.

Definition at line 131 of file model_runtime.py.

Referenced by health_payload(), and sync_from_cache().

◆ resolve_kind()

tuple[str, Literal["llm"]] resolve_kind ( str model_id )

Resolve slug + model family for any supported id.

Definition at line 124 of file model_runtime.py.

Referenced by begin_load().

◆ snapshot()

RuntimeSnapshot snapshot ( )

Thread-safe view of the in-process lifecycle state.

Definition at line 57 of file model_runtime.py.

Referenced by _emit(), begin_unload(), health_payload(), and sync_from_cache().

◆ sync_from_cache()

RuntimeSnapshot sync_from_cache ( )

Reconcile lifecycle state with actual VRAM caches (health checks).

Definition at line 144 of file model_runtime.py.

References resident_from_cache(), and snapshot().

Referenced by health_payload().

◆ unload_weights()

None unload_weights	(	*	,
		ProgressFn \| None	progress = None,
		bool	clear_active = True )

Drop all resident weights from VRAM in this process.

Definition at line 269 of file model_runtime.py.

References _emit(), begin_unload(), finish_unload(), and request_load_cancel().

◆ vram_line()

str | None vram_line ( )

Short VRAM summary for status output.

Definition at line 315 of file model_runtime.py.

Referenced by health_payload().

Variable Documentation

◆ _error

str aquin.compute.model_runtime._error = None

protected

Definition at line 29 of file model_runtime.py.

◆ _kind

Literal aquin.compute.model_runtime._kind = None

protected

Definition at line 28 of file model_runtime.py.

◆ _load_epoch

int aquin.compute.model_runtime._load_epoch = 0

protected

Definition at line 34 of file model_runtime.py.

◆ _load_epoch_at_start

int aquin.compute.model_runtime._load_epoch_at_start = 0

protected

Definition at line 35 of file model_runtime.py.

◆ _load_message

str aquin.compute.model_runtime._load_message = None

protected

Definition at line 32 of file model_runtime.py.

◆ _load_serial

aquin.compute.model_runtime._load_serial = threading.RLock()

protected

Definition at line 24 of file model_runtime.py.

◆ _loaded_at

float aquin.compute.model_runtime._loaded_at = None

protected

Definition at line 31 of file model_runtime.py.

◆ _lock

aquin.compute.model_runtime._lock = threading.Lock()

protected

Definition at line 22 of file model_runtime.py.

◆ _phase

str aquin.compute.model_runtime._phase = "idle"

protected

Definition at line 25 of file model_runtime.py.

◆ _resident_id

str aquin.compute.model_runtime._resident_id = None

protected

Definition at line 27 of file model_runtime.py.

◆ _started_at

float aquin.compute.model_runtime._started_at = None

protected

Definition at line 30 of file model_runtime.py.

◆ _target_id

str aquin.compute.model_runtime._target_id = None

protected

Definition at line 26 of file model_runtime.py.

◆ ProgressFn

aquin.compute.model_runtime.ProgressFn = Callable[[str, float], None]

Definition at line 20 of file model_runtime.py.

Classes

Functions

Variables

Function Documentation

◆ _emit()

◆ _set_phase()

◆ begin_load()

◆ begin_unload()

◆ fail_load()

◆ finish_load()

◆ finish_unload()

◆ health_payload()

◆ is_load_cancelled()

◆ load_weights()

◆ release_foreign_daemon()

◆ request_load_cancel()

◆ reset_state()

◆ resident_from_cache()

◆ resolve_kind()

◆ snapshot()

◆ sync_from_cache()

◆ unload_weights()

◆ vram_line()

Variable Documentation

◆ _error

◆ _kind

◆ _load_epoch

◆ _load_epoch_at_start

◆ _load_message

◆ _load_serial

◆ _loaded_at

◆ _lock

◆ _phase

◆ _resident_id

◆ _started_at

◆ _target_id

◆ ProgressFn