Skip to main content
Ctrl+K
onnx-light 0.1.31 documentation - Home onnx-light 0.1.31 documentation - Home
  • Quick tour
  • Start
  • Design
  • API
  • ByOp
  • How-To
  • Next Steps
  • Miscellaneous
  • GitHub
  • Quick tour
  • Start
  • Design
  • API
  • ByOp
  • How-To
  • Next Steps
  • Miscellaneous
  • GitHub

Section Navigation

  • How-to Python / C++
    • Command lines
      • fillshape
      • show
      • run
      • backend
      • kernel
    • How to install onnx-light
    • Replacing onnxruntime’s protobuf usage with onnx-light
    • How to link a C++ project against the pre-built onnx-light artifacts
    • Build and optimize a graph with GraphBuilder
    • How to replace onnx by onnx-light
    • How to load and save ONNX files
    • How to re-align external weights without loading them in memory
    • How to save a model that shares weights with another on-disk model
    • How to save and load a model in the ORT flatbuffer format
    • How to collect backend test cases (by op type or by name)
    • Run a backend test case with the reference evaluator
    • Benchmark methodology
    • How to use a custom kernel
    • Persistent input/output feedback
    • Quantizes tensors into encoded values
    • Tune kernel thresholds
    • How to use a custom optim shape inference function
    • How to register a built-in kernel, test case, shape inference, light op or peak memory function
    • How to add a custom graph-rewriting pattern and set its priority
    • How to use ONNX graph manipulation helpers
  • Proto Examples
    • Benchmark streaming vs in-memory alignment of external data
    • Inspect and edit nodes while parsing or serializing with a node callback
    • Load and save ONNX models with external data
    • Measures fresh-process ONNX model-load cold start
    • Measures loading and saving time for an ONNX model
    • Number of threads used to load and save ONNX models
    • Profiles ONNX external-data save time
    • Save an ONNX model in the ORT flatbuffer format and compare sizes
    • Statistics on the weights of an ONNX model
    • Track tensor weights while parsing with a raw_data callback
    • pretty_onnx: shape info, shape tags, inplace and release annotations
    • translate: turn an ONNX model back into Python code
  • Pattern Optimization Examples
    • Optimizing a model with graph-rewriting patterns
    • Replaying graph cleanup modifications
    • Replaying graph-rewriting patterns
  • Gradient Examples
    • Gradient and training loop for linear regression
  • Runtime, Backend Test, and Kernel Examples
    • Benchmark Abs: onnxruntime vs onnx-light
    • Benchmark a subset of backend test cases against onnxruntime
    • Benchmark the initialization steps: onnxruntime vs onnx-light
    • Calibrates quantization with graph kernels
    • Extend ReferenceEvaluator with a custom kernel
    • Profile the runtime memory of a model with the event log
    • Quantizes and dequantizes selected pages of a KV cache
    • Replace a built-in kernel with a Python one and prove it ran
    • Retrieve a backend test case and display its model and data
    • Run a model with the runtime and inspect intermediate results
    • Run an ONNX model casting a float tensor into an int2 tensor
    • Run the reference evaluator with tensor, sequence and dictionary inputs/outputs
    • Uses every portable quantization profile from Python
  • Tuning Examples
    • Inspect ParallelFor profiling from Python
    • Inspect, change, and calibrate kernel tuning from Python
  • Compute Examples
    • Benchmark GraphBuilder against onnxscript GraphBuilder
    • ComputeContext memory expressions
    • Computing shape, tag, constant, release and in-place information
    • Evaluating inferred shapes with concrete input dimensions
    • Optimized Shape inference
    • Qwen3-like ComputeContext memory profile
    • Shape inference with a custom operator
  • Expressions Examples
    • Symbolic expressions for dimensions
  • Examples with the C++ API
    • Standalone C++ example: combine onnx-light with Eclipse Aidge
    • Standalone C++ example: validate an ONNX model with onnx_light checker
    • Standalone C++ example: build, save and load an ONNX model with only lib_onnx_proto
    • Contiguous KV decode on CPU
    • Standalone C++ example: export an ONNX model to NNEF
    • Standalone C++ example: load an ONNX file with onnx_light
    • Standalone C++ example: measure ONNX loading time
    • Inspect ParallelFor profiling from C++
    • Standalone C++ example: print a proto for debugging
    • Standalone C++ example: register a new kernel for an existing operator
    • Standalone C++ example: run every backend test through onnxruntime
  • How-To
  • Tuning Examples

Tuning Examples#

Examples showing how to inspect parallel execution, calibrate kernel parameters, and persist machine-specific tuning profiles.

Inspect ParallelFor profiling from Python

Inspect ParallelFor profiling from Python

Inspect, change, and calibrate kernel tuning from Python

Inspect, change, and calibrate kernel tuning from Python

Download all examples in Python source code: auto_examples_tuning_python.zip

Download all examples in Jupyter notebooks: auto_examples_tuning_jupyter.zip

Gallery generated by Sphinx-Gallery

previous

Uses every portable quantization profile from Python

next

Inspect ParallelFor profiling from Python

Show Source

Created using Sphinx 9.1.0.

Built with the PyData Sphinx Theme 0.22.0.