Run a backend test case with the reference evaluator#

The backend-test catalog installed with onnx-light contains both models and their reference inputs and outputs. This walkthrough retrieves one small, deterministic case by its exact name, renders it as onnx-compact Python, and runs it without downloading any data.

Retrieve and display the case#

onnx_light.onnx.backend.get_test_case() performs an exact-name lookup. The test_cc_abs case is compiled into the regular wheel and contains one node and a small, fixed float32 input. The public onnx_light.tools.translate() helper renders its model with the onnx-compact representation; onnx_light.tools.translate_header() adds the imports needed to execute the generated expression.

    import numpy as np
    import ml_dtypes
    import onnx_light.onnx as onnx
    import onnx_light.onnx.helper as oh
    import onnx_light.onnx.numpy_helper as onh
    model = oh.make_model(
        oh.make_graph(
            [
                oh.make_node('Abs', ['x'], ['y']),
            ],
            'test_cc_abs',
            [
                oh.make_tensor_value_info('x', onnx.TensorProto.FLOAT, (2, 3)),
            ],
            [
                oh.make_tensor_value_info('y', onnx.TensorProto.FLOAT, (2, 3)),
            ],
        ),
        opset_imports=[oh.make_opsetid('', 13)],
        ir_version=13,
    )

Execute and compare the supplied values#

data_sets contains NumPy values in graph-input order and expected values in graph-output order. onnx_light.onnx.reference.ReferenceEvaluator exposes those names through input_names. The checks below deliberately test output count, shape, and dtype before applying the backend case’s own rtol and atol numerical tolerances.

    test_cc_abs: 1 output(s) match (rtol=0.001, atol=1e-07).

Enable runtime diagnostics and intermediate release#

The evaluator accepts execution options when the session is created:

  • verbose=1 prints one line for every dispatched node. Keep it at 0 for normal silent execution.

  • events_enabled=True records value-map changes and node dispatches in the session’s RuntimeContext; events() returns the records after a run.

  • release_intermediates=True removes an intermediate value after its last consumer. Setting it to False keeps intermediates until the run ends, which can aid debugging but increases peak memory. This one-node case has no intermediate tensors to release, but the option has this effect on larger graphs.

The same supplied inputs and expected values can therefore exercise the diagnostic path:

    Recorded 3 events with actions ['add', 'run_node'].

The runtime design overview explains how these Python calls map to session preparation, kernel dispatch, runtime storage, and CPU execution. To discover other cases, see html_theme.sidebar_secondary.remove.