Skip to main content
Ctrl+K
onnx-light 0.1.31 documentation - Home onnx-light 0.1.31 documentation - Home
  • Quick tour
  • Start
  • Design
  • API
  • ByOp
  • How-To
  • Next Steps
  • Miscellaneous
  • GitHub
  • Quick tour
  • Start
  • Design
  • API
  • ByOp
  • How-To
  • Next Steps
  • Miscellaneous
  • GitHub

Section Navigation

  • How-to Python / C++
    • Command lines
      • fillshape
      • show
      • run
      • backend
      • kernel
    • How to install onnx-light
    • Replacing onnxruntime’s protobuf usage with onnx-light
    • How to link a C++ project against the pre-built onnx-light artifacts
    • Build and optimize a graph with GraphBuilder
    • How to replace onnx by onnx-light
    • How to load and save ONNX files
    • How to re-align external weights without loading them in memory
    • How to save a model that shares weights with another on-disk model
    • How to save and load a model in the ORT flatbuffer format
    • How to collect backend test cases (by op type or by name)
    • Run a backend test case with the reference evaluator
    • Benchmark methodology
    • How to use a custom kernel
    • Persistent input/output feedback
    • Quantizes tensors into encoded values
    • Tune kernel thresholds
    • How to use a custom optim shape inference function
    • How to register a built-in kernel, test case, shape inference, light op or peak memory function
    • How to add a custom graph-rewriting pattern and set its priority
    • How to use ONNX graph manipulation helpers
  • Proto Examples
    • Benchmark streaming vs in-memory alignment of external data
    • Inspect and edit nodes while parsing or serializing with a node callback
    • Load and save ONNX models with external data
    • Measures fresh-process ONNX model-load cold start
    • Measures loading and saving time for an ONNX model
    • Number of threads used to load and save ONNX models
    • Profiles ONNX external-data save time
    • Save an ONNX model in the ORT flatbuffer format and compare sizes
    • Statistics on the weights of an ONNX model
    • Track tensor weights while parsing with a raw_data callback
    • pretty_onnx: shape info, shape tags, inplace and release annotations
    • translate: turn an ONNX model back into Python code
  • Pattern Optimization Examples
    • Optimizing a model with graph-rewriting patterns
    • Replaying graph cleanup modifications
    • Replaying graph-rewriting patterns
  • Gradient Examples
    • Gradient and training loop for linear regression
  • Runtime, Backend Test, and Kernel Examples
    • Benchmark Abs: onnxruntime vs onnx-light
    • Benchmark a subset of backend test cases against onnxruntime
    • Benchmark the initialization steps: onnxruntime vs onnx-light
    • Calibrates quantization with graph kernels
    • Extend ReferenceEvaluator with a custom kernel
    • Profile the runtime memory of a model with the event log
    • Quantizes and dequantizes selected pages of a KV cache
    • Replace a built-in kernel with a Python one and prove it ran
    • Retrieve a backend test case and display its model and data
    • Run a model with the runtime and inspect intermediate results
    • Run an ONNX model casting a float tensor into an int2 tensor
    • Run the reference evaluator with tensor, sequence and dictionary inputs/outputs
    • Uses every portable quantization profile from Python
  • Tuning Examples
    • Inspect ParallelFor profiling from Python
    • Inspect, change, and calibrate kernel tuning from Python
  • Compute Examples
    • Benchmark GraphBuilder against onnxscript GraphBuilder
    • ComputeContext memory expressions
    • Computing shape, tag, constant, release and in-place information
    • Evaluating inferred shapes with concrete input dimensions
    • Optimized Shape inference
    • Qwen3-like ComputeContext memory profile
    • Shape inference with a custom operator
  • Expressions Examples
    • Symbolic expressions for dimensions
  • Examples with the C++ API
    • Standalone C++ example: combine onnx-light with Eclipse Aidge
    • Standalone C++ example: validate an ONNX model with onnx_light checker
    • Standalone C++ example: build, save and load an ONNX model with only lib_onnx_proto
    • Contiguous KV decode on CPU
    • Standalone C++ example: export an ONNX model to NNEF
    • Standalone C++ example: load an ONNX file with onnx_light
    • Standalone C++ example: measure ONNX loading time
    • Inspect ParallelFor profiling from C++
    • Standalone C++ example: print a proto for debugging
    • Standalone C++ example: register a new kernel for an existing operator
    • Standalone C++ example: run every backend test through onnxruntime
  • How-To

How-To#

  • How-to Python / C++
  • Proto Examples
  • Pattern Optimization Examples
  • Gradient Examples
  • Runtime, Backend Test, and Kernel Examples
  • Tuning Examples
  • Compute Examples
  • Expressions Examples
  • Examples with the C++ API

previous

QuantizePagedCache

next

How-to Python / C++
Show Source

Created using Sphinx 9.1.0.

Built with the PyData Sphinx Theme 0.22.0.