ONNX runtime benchmarks

Compare execution backends over time

Explore model and operator performance across optimized, reference, and lightweight ONNX runtimes. Lower execution times are better.

ONNX Runtime Production runtime with optimized CPU execution providers. ONNX Reference ONNX's Python reference evaluator, focused on specification correctness. onnx-light Lightweight ONNX execution with a compact Python kernel implementation. onnx-light-cpu onnx-light with native CPU kernels and fallback to the lightweight runtime. ONNX Runtime GenAI Generation API for autoregressive models, used by the Tiny-LLM benchmark.

Benchmark processors

Logical cores and instruction sets available to each benchmark worker.

Processor Logical cores Instruction sets
commits
plot settings
log scale zoom y axis reference even commit spacing date scale legend