Processing time comparison between
onnxruntime and the
onnx-light reference evaluator backed by its C++ kernel
dispatch table (onnx_light.reference.ReferenceEvaluator)
on every ONNX backend node test.
Backends run in separate global phases with their default spin policies.
Each test runs up to CPU-scaled warm-up
iterations (not timed) and up to
CPU-scaled timed iterations. Each phase
stops after 1 second of cumulative
execution;
the average time per dataset is reported (a trimmed mean that
excludes the fastest and slowest timed iterations; hover a cell for the
raw min/max).
Speedup = onnxruntime avg /
onnx‑light avg:
values > 1 mean onnx‑light is faster. The
onnx‑light + cpu column runs the same
onnx‑light runtime with the
onnx‑light‑cpu
SIMD kernels layered on top, and its speedup uses the same
onnxruntime avg baseline.
Three averages are reported. Avg speedup is the unweighted mean of every
per-test speedup, so an O(1) kernel (e.g.
Shape/Reshape) counts as much as an O(n)
(Add) or O(n²) (Gemm) one; the
weighted avg speedup card instead weighs each test by a
symbolic kernel cost: linear in tensor size for ordinary unary/binary
operators and quadratic for matrix multiplication and attention
(sum(N × speedup) / sum(N)),
while sum latency speedup is
sum(baseline latency) / sum(backend latency).
Each row shows its symbolic weight, clipped to the range 1 – 64.