locodellm.test_models.mock_generate_model#
Creates a mock LLM model directory for plot_generate.py.
The model produces the exact same outputs as the
Qwen/Qwen2.5-Coder-0.5B-Instruct model for the two prompts
used in docs/examples/plot_generate.py, using a hardcoded
token lookup table instead of real transformer weights.
The two prompts (with do_sample=False, top_k=1):
'write a python function which returns "hello"'"change hello into bonjour"
- locodellm.test_models.mock_generate_model.create_mock_generate_model(output_dir: str) str#
Creates a mock model directory that reproduces plot_generate.py outputs.
The directory will contain:
model.onnx– a lookup-table-based ONNX decoder graph.genai_config.json– onnxruntime-genai configuration.tokenizer.json– the Qwen tokenizer (downloaded from HuggingFace).tokenizer_config.json– tokenizer metadata.
- Parameters:
output_dir – Path where the model directory is created.
- Returns:
The absolute path to output_dir.
- locodellm.test_models.mock_generate_model.make_decoder_model()#
Builds a minimal ONNX decoder graph with hardcoded token predictions.
- Inputs:
input_ids [batch, seq_len] INT64 attention_mask [batch, total_seq_len] INT64 past_key_values.0.key [batch, H, past_seq, D] FLOAT past_key_values.0.value [batch, H, past_seq, D] FLOAT
- Outputs:
logits [batch, seq_len, vocab] FLOAT present.0.key [batch, H, total_seq, D] FLOAT present.0.value [batch, H, total_seq, D] FLOAT
The logits are computed by looking up the expected next token from a hardcoded table indexed by
past_seq + seq_len(the total number of tokens seen so far). A one-hot vector scaled to 100.0 is returned so that greedy / near-greedy decoding always picks the right token.