locodellm.test_models.tiny_model#
Creates a tiny dummy LLM model directory for testing.
The model has uninitialized (zero) weights and produces meaningless output, but the ONNX graph and configuration files are valid enough for onnxruntime-genai to load and run a generation loop.
- locodellm.test_models.tiny_model.create_tiny_model(output_dir: str) str#
Creates a tiny dummy LLM model directory at output_dir.
The directory will contain:
model.onnx– a valid but zero-weight ONNX decoder graph.genai_config.json– onnxruntime-genai configuration.tokenizer.json– minimal BPE tokenizer.tokenizer_config.json– tokenizer metadata.
- Parameters:
output_dir – Path where the model directory is created. The directory and any missing parents are created automatically.
- Returns:
The absolute path to output_dir.
- locodellm.test_models.tiny_model.make_decoder_model()#
Builds a minimal ONNX decoder graph.
- Inputs:
input_ids [batch, seq_len] INT64 attention_mask [batch, total_seq_len] INT64 position_ids [batch, seq_len] INT64 past_key_values.L.key [batch, H, past_seq, D] FLOAT past_key_values.L.value [batch, H, past_seq, D] FLOAT
- Outputs:
logits [batch, seq_len, vocab] FLOAT present.L.key [batch, H, total_seq, D] FLOAT present.L.value [batch, H, total_seq, D] FLOAT
The logits are computed by a zero-weight embedding lookup followed by a zero-weight projection. The KV cache is grown by padding one zero slice on the sequence axis.
- locodellm.test_models.tiny_model.make_genai_config() dict#
Returns a minimal
genai_config.jsondict.