Getting Started#

Installation#

Install from source:

pip install .

Or with Pixi:

pixi install
pixi run install

Or build with CMake (C++ only):

cmake -S . -B build -DONNX_LIGHT_CPU_BUILD_TESTS=ON \
      -DONNX_LIGHT_CPU_BUILD_PYTHON=OFF \
      -DONNX_LIGHT_CPU_WITH_ONNX_LIGHT=ON
cmake --build build

The C++ quick start requires the onnx-light C++ package to be installed so find_package(onnx_light) can locate it.

Diagnosing native-extension import failures#

Before attributing an undefined symbol to import order, rebuild both projects from clean source trees with no existing native artifacts. Reusing an in-place build can mix extensions and shared libraries from different configurations. Use absolute source paths and fresh build directories; do not install either project into site-packages or use editable installs for this check.

For example, on Linux, with the Python build dependencies already available:

export ONNX_SOURCE=/absolute/path/to/onnx-light
export CPU_SOURCE=/absolute/path/to/onnx-light-cpu
export BUILD_ROOT="$(mktemp -d)"

cmake -S "$ONNX_SOURCE" -B "$BUILD_ROOT/onnx-light" \
    -DCMAKE_BUILD_TYPE=Release -DONNX_LIGHT_INSTALL=OFF \
    -DPython_EXECUTABLE="$(command -v python)"
cmake --build "$BUILD_ROOT/onnx-light" --parallel 4
cmake --install "$BUILD_ROOT/onnx-light" --prefix "$ONNX_SOURCE"

export PYTHONPATH="$CPU_SOURCE:$ONNX_SOURCE"
python "$CPU_SOURCE/setup.py" build_ext --inplace --onnx-light-source \
    --build-temp "$BUILD_ROOT/onnx-light-cpu" --parallel 4

cd "$CPU_SOURCE"
python -m unittest \
    unittests.python.test_kernels_e2e.TestExtensionImportOrder -v

Here cmake --install places the dependency’s freshly built libraries in its source tree, not in site-packages. --onnx-light-source links against those same runtime libraries rather than rebuilding a second copy.

In-place builds may place symbolic links in onnx_light/onnx_py pointing to native artifacts in the same checkout’s build directory. These are supported: the extension is located through the selected source package, and linking uses the resolved runtime-library paths.

Python builds that exchange nanobind-bound C++ objects must also use the same effective ABI mode. Release builds target onnx-light 0.2.0; its wheels must be published before the matching onnx-light-cpu release can be built. The release configuration uses nanobind’s native ABI even though ONNX_LIGHT_PYTHON_STABLE_ABI defaults to ON: the option requests stable ABI support, but the effective mode depends on the build frontend and Python configuration. The matching onnx-light-cpu release therefore sets ONNX_LIGHT_CPU_PYTHON_STABLE_ABI=OFF. In either mode, onnx-light-cpu must link the exact shared onnx-light runtime loaded by Python; linking a second copy creates a separate type and registry universe, so objects such as RuntimeContext cannot cross the module boundary. A downstream build must match the effective ABI reported by onnx-light rather than infer it from a CMake option or wheel tag.

With onnx-light 0.2.0 source checkouts, the standalone proto headers are under onnx_proto/include and their helpers under onnx_proto/helpers/include. --onnx-light-source uses those paths while still linking the exact core and proto shared libraries loaded by Python. Source checkouts with the standalone threading library also use onnx_threads/include and link the same threading shared library as Python, without rebuilding it. Direct CMake builds can select that library with ONNX_LIGHT_CPU_ONNX_LIGHT_THREADS_LIBRARY; Windows builds also need ONNX_LIGHT_CPU_ONNX_LIGHT_THREADS_IMPLIB. Installed Python wheels retain the packaged header layout under onnx_light. The selected threading library is also linked with that layout, without requiring the source-only onnx_threads/include directory. Both layouts are recognized when selecting the runtime through PYTHONPATH, including forwarding the threading import library on Windows.

The regression test imports _cpukernels then _cpuregister, and the reverse order, in separate Python subprocesses. It runs from both the checkout and a temporary working directory, verifies that the CPU extensions came from the checkout, and exercises kernel registration. A failure includes the child process’s output and traceback. If a clean build passes but reused artifacts fail, record both source revisions and build commands with the report rather than changing symbol visibility or preloading libraries to mask the failure.

For issue #698, this procedure did not reproduce the reported undefined symbol on Linux x86-64. The clean Release builds used CPU sources at 1ebe06e8b3e4 and onnx-light at e03a1ba56145, with Python 3.13.15, GCC 13.3, CMake 3.31.6, and nanobind 3.0.1. Both import orders passed from both working directories, as did the existing kernel-usage and SIMD-detection tests (nine tests total). Neither project was installed into site-packages, and no runtime-library preload was needed. The reported mixed-artifact failure therefore did not warrant a source linkage or export change.

The same regression subsequently exposed a separate Windows DLL-search failure: _cpuregister could not locate onnx-light’s dependent DLLs in a fresh process. On Windows, the bindings package registers the selected onnx-light package’s onnx_py directory with os.add_dll_directory and retains its handle. This makes the DLLs discoverable without preloading either project’s extensions; adding the directory to PATH alone is insufficient for Python’s extension loader.

Windows release-wheel repair excludes the runtime DLLs supplied by onnx-light from delvewheel bundling and renaming, just as Linux and macOS keep that runtime external. The installed onnx-light wheel supplies the shared runtime; bundling another copy would separate its types and kernel registries.

Quick Start#

Register the optimized kernels, then execute the graph through onnx-light’s runtime. Runtime dispatch selects the registered onnx-light-cpu kernel and its best available SIMD path:

from onnx_light_cpu import detect_simd_level, has_cpu_kernels, register_kernels

# Check what SIMD level is available:
print("CPU kernels available:", has_cpu_kernels())
print("SIMD level:", detect_simd_level())

# When onnx-light is installed, register the optimized kernels globally so
# ReferenceEvaluator uses them for supported ONNX operators.
register_kernels()

#include <onnx_light_cpu/kernels/register_kernels.h>

#include <onnx_core/runtime/kernels/kernel_context.h>
#include <onnx_core/runtime/memory/simple_tensor.h>
#include <onnx_core/runtime/runtime_context.h>
#include <onnx_core/runtime/runtime_session.h>
#include <onnx_proto/onnx_helper.h>

int main() {
    namespace rt = ONNX_LIGHT_NAMESPACE::core::runtime;

    onnx_light_cpu::RegisterAllKernels();

    ONNX_LIGHT_NAMESPACE::GraphProto graph;
    graph.ref_node().push_back(
        ONNX_LIGHT_NAMESPACE::MakeNode("Abs", {"x"}, {"y"}));

    rt::RuntimeContext context(rt::KernelContext(rt::DefaultOpset(18)));
    context.Set(
        "x", rt::Tensor::FromFloat("x", {4}, {-1.0f, 2.0f, -3.0f, 4.0f}));
    rt::RuntimeSession session(context.GetExecutionPlan(graph));
    session.Run(context);

    const float *output = context.Get("y").AsFloat();
    // output = {1.0f, 2.0f, 3.0f, 4.0f}
}