Design

onnx-light replaces :epkg`protobuf` by a custom implementation but keeps the same ONNX to make it fully compatible. It offers more freedom to implement any custom loading, parsing scenario and speed up this first step.

It replicates the same Python API and the same C++ API to enable a smooth replacement.

Without Protobuf

• Differences between onnx and onnx_light
  • No protobuf dependency
  • Files larger than 2 GB
  • Buffered file I/O
  • Parallel tensor loading
  • Zero-copy parsing
  • C++ class generation via macros
  • External-data / multi-file models
    • Saving with external data
    • Loading with external data
    • Splitting external data across multiple files
  • Encrypted model save / load
    • File format
    • Python API (file-based)
    • Python API (in-memory / bytes)
    • C++ API
  • API compatibility
  • Summary
• Protobuf format applied to ONNX
  • The wire format in a nutshell
    • Tag encoding
    • Varints
    • ZigZag encoding
    • Length-prefixed values
    • Packed repeated fields
    • Default values and unknown fields
  • How ONNX uses the wire format
    • Tensors and raw_data
    • External data
    • The 2 GB protobuf limit
  • Further reading
• ModelProto creation and no-copy ownership
  • Options class hierarchy
  • Core objects and where ownership lives
  • When ownership is assigned during parsing
  • In-place consolidation with ConsolidateTensorsToBuffer
  • Loading scenarios summary
  • When memory is released
  • Model copy/move behavior
• Linking onnx-light in C++
  • Install and link model
  • Alternative without install
  • Excerpt from the example project
  • See also

Shape Inference is still under development as a new algorithm is being implemented to handle small tensor and other backward propagation. It supports simple expressions often used in models.

Shape Inference

• Comparison: onnx OpSchema vs onnx_light LightOpSchema
  • Summary
  • Per-operator comparison
  • See also
• Symbolic expression library (onnx_light.onnx_optim.expressions)
  • Motivation
  • Expression grammar
  • AST node types
  • Simplification pipeline
  • Unparser
  • Dimension operations
  • Renaming
  • Build layout
  • API reference

A C++ backend test is implemented enabling the testing of a runtime in both Python and C++. It does not store any big tensor to avoid having a huge package to publish.

Backend

• Using backend tests to evaluate a runtime
  • Defining a runtime function
  • Generating a test class
  • Filtering tests
  • Adjusting numerical tolerances
  • Filtering test cases by operator and opset
  • Full example: ONNXRuntime backend
  • How test cases are collected
  • Running backend tests in C++
  • See also
• Backend test-case coverage
  • What is measured
  • Computing the report
  • Listing operators without a test case
  • See also
• Runtime test coverage (onnxruntime and shape_inference)
  • Summary
  • Per-test-case status
  • See also