AquinLabs

Methods · LLM / LoRA

Foundation-model training with Hugging Face + PEFT. model: is required. Devices: CUDA, MPS, ROCm, CPU. QLoRA is CUDA + bitsandbytes only.

Prerequisitefamily: llm · recipe.model set · kernel HF deps installed

method: lora | qlora | llm

LoRA / QLoRA / full LLM training. Objectives: next-token, sft, full-ft, lora, qlora, fim, mlm, span, continued-pretrain.

example
family: llm
method: lora
model: meta-llama/Llama-3.2-1B-Instruct
objective: sft
rank: 16
alpha: 32
steps: 100
lr: 2.0e-4
data:
  path: data.jsonl
  prompt: prompt
  completion: completion
eval:
  metric: loss

SFT masks prompt tokens (loss on completion). full-ft trains all non-pad tokens. mlm needs a MaskedLM-capable model. size: is a label only.

aq serve

Generate from the latest (or named) checkpoint after an LLM train.

FlagDescription
--ckpt <name> Checkpoint name.
--max-tokens n Max tokens.
example
$aq serve "hello" --max-tokens 32