Linking onnx-light in C++

This page summarizes the design used to consume onnx-light as a standalone C++ library from another project.

Runnable examples are available in C++ onnx-light examples, including examples/load_onnx_light_time, examples/save_onnx_light_time, examples/build_save_load_onnx_proto, examples/check_onnx_light_model, examples/export_nnef, examples/run_add_node_test and examples/run_backend_test_ort. Each example’s CMakeLists.txt shows which subset of the exported targets is needed.

See How the C++ libraries are split for the full dependency graph of the C++ libraries and a per-target description.

CMake configure-time options

The top-level CMakeLists.txt exposes the following options. All of them can be set on the cmake command line with -D<NAME>=<VALUE>.

Option	Default	Description
ONNX_LIGHT_BUILD_PYTHON	ON	Build the nanobind Python extensions (_onnxpyprotoop, _onnxpyprotolib, _onnxpycore, _onnxpykernels and _onnxpybackend). Turn OFF for a pure C++ build; this also switches lib_onnx_proto from a shared library back to a static archive.
ONNX_LIGHT_INSTALL	ON	Install the C++ libraries, headers and the exported onnx_light CMake package on cmake --install. Turn OFF when building only for in-tree consumption (for example from a parent add_subdirectory).
ONNX_LIGHT_BUILD_KERNELS	ON	Build the operator-kernel runtime (lib_onnx_kernels) and the backend-test case registry (lib_onnx_backend_test). Turn OFF to install only the schema / checker / shape-inference / version-converter / proto libraries; incompatible with ONNX_LIGHT_BUILD_PYTHON=ON and ONNX_LIGHT_BUILD_TESTS=ON.
ONNX_ML	ON	Enable ai.onnx.ml (traditional ML) operator support.
ONNX_LIGHT_BUILD_TESTS	OFF	Build the C++ unit-test executable test_onnx_light and register the tests with CTest. Requires ONNX_LIGHT_BUILD_KERNELS=ON.
ONNX_LIGHT_BUILD_BENCHMARKS	OFF	Build the C++ benchmark executables from benchmarks/bench_*.cc.
ONNX_LIGHT_BENCH_GPROF	OFF	Compile the benchmark executables with -pg for gprof profiling. Only meaningful with ONNX_LIGHT_BUILD_BENCHMARKS=ON.
ONNX_LIGHT_BENCH_WITH_UPSTREAM_ONNX	OFF	Fetch and build upstream onnx (the protobuf-based reference implementation) via FetchContent to enable the BENCH_HAS_UPSTREAM_ONNX side-by-side comparison block in bench_load_file. Only meaningful with ONNX_LIGHT_BUILD_BENCHMARKS=ON.
ONNX_LIGHT_BUILD_FUZZERS	OFF	Build the libFuzzer-instrumented fuzz harnesses (fuzz/fuzz_*.cc) and the make_seed_corpus helper. Requires Clang and adds -fsanitize=fuzzer,address by default.
ONNX_LIGHT_FUZZER_SANITIZERS	address	Comma-separated sanitizer list passed to -fsanitize= for the fuzz harnesses (the fuzzer sanitizer is always added automatically). Only meaningful with ONNX_LIGHT_BUILD_FUZZERS=ON.
ONNX_LIGHT_WERROR	ON	Treat C/C++ compiler warnings as errors (/WX on MSVC, -Werror on GCC/Clang, on top of the -Wall -Wextra baseline those compilers always receive). Third-party sources built inside the tree (googletest, nanobind, vendored BLAKE3, upstream onnx) are excluded, and Clang’s -Wunused-lambda-capture stays a plain warning. Turn OFF when building with a toolchain that emits warnings onnx-light does not control.
ONNX_HARDENING	OFF	Opt in to the OpenSSF Compiler Options Hardening Guide for C and C++ baseline. When ON, every onnx-light library, Python extension, test, and benchmark target receives the recommended compile and link flags (_FORTIFY_SOURCE=3, _GLIBCXX_ASSERTIONS, -fstack-protector-strong, -fstack-clash-protection, -fcf-protection=full, -fstrict-flex-arrays=3, -ftrivial-auto-var-init=zero, -Wformat=2, -Werror=format-security, -z noexecstack, -z relro, -z now, on GCC/Clang and /GS, /guard:cf, /Qspectre, /sdl, /DYNAMICBASE, /NXCOMPAT, /CETCOMPAT on MSVC). When /Qspectre is enabled on MSVC, the configure step also prepends the matching VCToolsInstallDir/lib/spectre/<arch> directory to the linker search path so the Spectre-mitigated CRT/STL libraries are used when that Visual Studio component is installed. Each flag is probed by the configure step and silently skipped when the active toolchain does not accept it. See cmake/Hardening.cmake for the full list.

Build without the backend tests and kernels

The operator-kernel runtime (lib_onnx_kernels) and the backend-test case registry (lib_onnx_backend_test) are by far the largest libraries in the build. When downstream code only needs the schema / checker / shape-inference / version-converter / proto layer, pass -DONNX_LIGHT_BUILD_KERNELS=OFF at configure time to skip building and installing them:

cmake -S . -B build-install \
      -DCMAKE_BUILD_TYPE=Release \
      -DONNX_LIGHT_BUILD_PYTHON=OFF \
      -DONNX_LIGHT_BUILD_KERNELS=OFF \
      -DCMAKE_INSTALL_PREFIX=/usr/local
cmake --build build-install
cmake --install build-install

The exported CMake package then provides only the kernel-free targets — onnx_light::lib_onnx_lib, onnx_light::lib_onnx_manipulations, onnx_light::lib_onnx_op, onnx_light::lib_onnx_shape, onnx_light::lib_onnx_patterns and onnx_light::lib_onnx_proto. onnx_light::lib_onnx_kernels and onnx_light::lib_onnx_backend_test are not built and not part of the package.

ONNX_LIGHT_BUILD_KERNELS=OFF is incompatible with ONNX_LIGHT_BUILD_PYTHON=ON and ONNX_LIGHT_BUILD_TESTS=ON: the Python extensions and the C++ unit tests both require the kernels.

Install and link model

From the repository root, install the C++ library with CMake:

cmake -S . -B build-install \
      -DCMAKE_BUILD_TYPE=Release \
      -DONNX_LIGHT_BUILD_PYTHON=OFF \
      -DCMAKE_INSTALL_PREFIX=/usr/local
cmake --build build-install
cmake --install build-install

When downstream code only needs the schema / checker / shape-inference / version-converter / proto layer (onnx_light::lib_onnx_lib, onnx_light::lib_onnx_manipulations, onnx_light::lib_onnx_op, onnx_light::lib_onnx_shape, onnx_light::lib_onnx_patterns, onnx_light::lib_onnx_proto), pass -DONNX_LIGHT_BUILD_KERNELS=OFF at configure time to skip building and installing the much larger lib_onnx_kernels (operator-kernel runtime) and lib_onnx_backend_test (backend-test case registry) libraries. onnx_light::lib_onnx_kernels and onnx_light::lib_onnx_backend_test are then not part of the exported CMake package.

Then downstream projects can rely on the exported CMake targets:

find_package(onnx_light REQUIRED)
target_link_libraries(my_target PRIVATE onnx_light::lib_onnx_lib)

Use onnx_light::lib_onnx_lib when the code needs higher-level ONNX features implemented in onnx_light/onnx_lib such as operator schemas, checker, shape inference, or version conversion.

For protobuf-compatible message parsing/serialization only, downstream code can link just the lighter proto target:

find_package(onnx_light REQUIRED)
target_link_libraries(my_target PRIVATE onnx_light::lib_onnx_proto)

That is sufficient when the program only manipulates ModelProto / GraphProto data and does not need any notion of operators.

For manual registration of lightweight math operator schemas without shape inference support, downstream code can link:

find_package(onnx_light REQUIRED)
target_link_libraries(my_target PRIVATE onnx_light::lib_onnx_op)

To parse / print ONNX text models and manipulate ModelProto / GraphProto (attribute and tensor proto helpers, data-type name utilities, graph-input collection) without pulling in the operator schemas, link the manipulations target, which only depends on lib_onnx_proto:

find_package(onnx_light REQUIRED)
target_link_libraries(my_target PRIVATE onnx_light::lib_onnx_manipulations)

When the standalone shape-inference dispatch is needed without the full onnx_light::lib_onnx_lib checker/inliner/version converter, link:

find_package(onnx_light REQUIRED)
target_link_libraries(my_target PRIVATE onnx_light::lib_onnx_shape)

Registering and selecting graph patterns

Pattern integration has two independent choices: link lib_onnx_patterns and register the standard ONNX rewrites, or link only lib_onnx_core and provide application-specific rewrites.

Standard ONNX patterns

The generic graph optimizer and registry are part of lib_onnx_core. The concrete ONNX rewrites live in the optional lib_onnx_patterns extension. Registration is explicit and idempotent:

find_package(onnx_light REQUIRED)
target_link_libraries(my_target PRIVATE onnx_light::lib_onnx_patterns)

#include "onnx_core/builder/graph_graph.h"
#include "onnx_core/builder/pattern_registry.h"
#include "onnx_extensions/patterns/dispatch_table.h"

onnx_patterns::RegisterPatterns();
core::builder::GraphGraph optimizer(builder);
optimizer.Optimize();

GraphGraph(builder) instantiates every registered factory. Registration order is preserved by the registry, then the optimizer evaluates patterns in ascending priority order. Names are stable, non-empty, and unique; RegisterPattern throws PatternRegistrationError for invalid or duplicate registrations.

To inspect or filter the standard set, create it explicitly before constructing the optimizer:

onnx_patterns::RegisterPatterns();
const auto names = core::builder::RegisteredPatternNames();
auto patterns = core::builder::CreateRegisteredPatterns();
std::erase_if(patterns, [](const auto &pattern) {
  return pattern->Name() != "CastCast";
});
core::builder::GraphGraph optimizer(builder, std::move(patterns));
optimizer.Optimize();

Custom patterns

Applications that provide only custom rewrites link onnx_light::lib_onnx_core; they do not need lib_onnx_patterns:

find_package(onnx_light REQUIRED)
target_link_libraries(my_target PRIVATE onnx_light::lib_onnx_core)

Derive from PatternOptimization and implement FastOpType, Match, and Apply. Passing instances directly keeps selection local to one optimizer:

std::vector<std::unique_ptr<core::builder::PatternOptimization>> patterns;
patterns.push_back(std::make_unique<MyPattern>());
core::builder::GraphGraph optimizer(builder, std::move(patterns));
optimizer.Optimize();

Alternatively, register a factory once to make the pattern available to default GraphGraph instances:

core::builder::RegisterPattern(
    "my_extension.MyPattern",
    []() { return std::make_unique<MyPattern>(); });
core::builder::GraphGraph optimizer(builder);
optimizer.Optimize();

examples/register_custom_pattern contains a complete custom pattern, both selection modes, and standalone CMake files. Its build.sh and build.bat scripts first install the current source tree and then build against the exported onnx_light::lib_onnx_core target. Inside the source tree, the equivalent non-exported target name is lib_onnx_core.

To evaluate ONNX nodes / graphs / models in-process using the bundled C++ reference implementation of the ONNX operators (runtime struct Tensor, RunGraph / RunFunction / RunModel, the SplitMix64-based deterministic RNG, …), link the kernels target:

find_package(onnx_light REQUIRED)
target_link_libraries(my_target PRIVATE onnx_light::lib_onnx_kernels)

The kernels live under onnx_light/onnx_extensions/kernels/kernels/<group>/ and form a self-contained runtime that depends on onnx_light::lib_onnx_proto and onnx_light::lib_onnx_manipulations (for the graph-manipulation helpers). See onnx_kernels for the full C++ API reference.

To additionally pull in the backend-test infrastructure (struct TestCase, Expect() helper, per-operator RegisterXxxCases registries used by every CollectTestCases call), link:

find_package(onnx_light REQUIRED)
target_link_libraries(my_target PRIVATE onnx_light::lib_onnx_backend_test)

onnx_light::lib_onnx_backend_test publicly depends on onnx_light::lib_onnx_kernels (which transitively brings in onnx_light::lib_onnx_proto) and is intentionally independent from onnx_light::lib_onnx_lib / onnx_light::lib_onnx_op; it can be combined with onnx_light::lib_onnx_lib when both schema validation and execution are needed in the same binary.

To compute gradient functions (reverse-mode automatic differentiation of ONNX graphs via GradientOfNodes / GradientOfFunction), link the gradient target, which depends on lib_onnx_core (and transitively on lib_onnx_proto):

find_package(onnx_light REQUIRED)
target_link_libraries(my_target PRIVATE onnx_light::lib_onnx_gradient)

See onnx_gradient for the full C++ API reference.

This keeps downstream CMake files independent from hardcoded include paths and library file names. If onnx-light is installed to a non-standard prefix, configure the downstream project with -DCMAKE_PREFIX_PATH=<prefix>.

Alternative without install

For monorepos or local development, a downstream CMake project can also include onnx-light directly:

set(ONNX_LIGHT_BUILD_PYTHON OFF CACHE BOOL "" FORCE)
add_subdirectory(path/to/onnx-light)
target_link_libraries(my_target PRIVATE lib_onnx_lib)

Use the in-tree lib_onnx_proto target instead when only proto parsing/serialization is needed, or lib_onnx_op, lib_onnx_manipulations, lib_onnx_shape, lib_onnx_patterns, lib_onnx_kernels, lib_onnx_backend_test or lib_onnx_gradient for the corresponding feature subset. This uses the in-tree build targets directly instead of find_package.

Excerpt from the example project

The example CMake project in examples/load_onnx_light_time uses exactly that pattern:

cmake_minimum_required(VERSION 3.15)
project(load_onnx_light_time LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

find_package(onnx_light REQUIRED)

add_executable(load_onnx_light_time main.cc)
target_link_libraries(load_onnx_light_time PRIVATE onnx_light::lib_onnx_proto)

Python extension modules and proto duplication

The Python package ships up to six nanobind extension modules, onnx_light.onnx_py._onnxpyprotoop, onnx_light.onnx_py._onnxpyprotolib, onnx_light.onnx_py._onnxpycore, onnx_light.onnx_py._onnxpykernels, onnx_light.onnx_py._onnxpybackend and onnx_light.onnx_py._onnxpygradient. The first three are always built; the last three (_onnxpykernels, _onnxpybackend and _onnxpygradient) belong to the extended build variant and are only present when ONNX_LIGHT_BUILD_KERNELS=ON. They all need access to the proto classes (ModelProto, NodeProto, TensorProto, …) defined in onnx_light/onnx_proto. How do the extensions agree on a single nb::class_<ModelProto> registration so that values can flow between them without a serialise/parse round-trip?

When ONNX_LIGHT_BUILD_PYTHON=ON, CMakeLists.txt builds lib_onnx_proto as a shared library (liblib_onnx_proto.so / .dylib / .dll) instead of a static archive. All the extensions link against that single shared object (directly or transitively through lib_onnx_lib / lib_onnx_op / lib_onnx_shape / lib_onnx_kernels / lib_onnx_backend_test / lib_onnx_gradient), and the build installs every file side by side under onnx_light/onnx_py/. The extensions are linked with an $ORIGIN runtime path (@loader_path on macOS) so the dynamic loader finds liblib_onnx_proto.so next to them at import time without any LD_LIBRARY_PATH setup.

Pure C++ consumers (ONNX_LIGHT_BUILD_PYTHON=OFF) keep the lighter static variant they used to ship, so the existing find_package(onnx_light) -> onnx_light::lib_onnx_proto workflow is unchanged.

Because liblib_onnx_proto.so is loaded only once per process, the proto classes have a single set of out-of-line member definitions and a single std::type_info instance. Consequently &typeid(ModelProto) evaluates to the same pointer in every extension, and nanobind’s cross-module type registry resolves ModelProto references coming from _onnxpycore or _onnxpybackend against the nb::class_<ModelProto> that _onnxpyprotoop registered. In practice, only _onnxpyprotoop declares nb::class_<NodeProto> / nb::class_<ModelProto> / …; the _onnxpyprotolib, _onnxpycore, _onnxpykernels, _onnxpybackend and _onnxpygradient modules return proto values by reference (for example TestCase.model, see onnx_light/onnx_py/_onnxpy_backend_test.cc) and let the shared registry produce a Python object backed by the same binding. Every Python module that consumes a proto value imports the proto classes directly from _onnxpyprotoop (for example from ..onnx_py._onnxpyprotoop import ModelProto), which guarantees that _onnxpyprotoop is loaded — and its nb::class_<ModelProto> binding registered — before any _onnxpyprotolib / _onnxpycore / _onnxpykernels / _onnxpybackend / _onnxpygradient accessor is used.

See also

• How the C++ libraries are split

• Standalone C++ example: load an ONNX file with onnx_light