.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples_backend/plot_run_cast_to_int2.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_backend_plot_run_cast_to_int2.py: .. _l-example-plot-run-cast-to-int2: Run an ONNX model casting a float tensor into an int2 tensor ============================================================ This example shows how to take one backend test case from the suite shipped with ``onnx-light`` (:mod:`onnx_light.onnx.backend`), run its ONNX model with the reference runtime, and then re-run the *same* case as a backend test. The case we use is ``test_cc_cast_FLOAT_to_INT2``: a single ``Cast`` node that converts a ``float32`` tensor into a 2-bit signed integer tensor (``INT2``). ``INT2`` is a sub-byte dtype: its representable range is ``[-2, 1]`` and values outside that range saturate, which is visible in the runtime output below. This example: * retrieves the ``test_cc_cast_FLOAT_to_INT2`` case via :func:`onnx_light.onnx.backend.collect_test_case`, * displays its single-node ``Cast`` ``ModelProto``, * runs the model with :class:`onnx_light.onnx.reference.ReferenceEvaluator` and prints the resulting ``INT2`` tensor, * shows how to run the corresponding backend test by passing a tiny runtime function to the case's ``assert_allclose`` method (and notes the :func:`onnx_light.onnx.backend.make_test_class` helper for running the whole registry). .. GENERATED FROM PYTHON SOURCE LINES 31-40 .. code-block:: Python from __future__ import annotations import numpy as np from onnx_light.onnx.backend import collect_test_case from onnx_light.onnx.reference import ReferenceEvaluator from onnx_light.tools import pretty_onnx .. GENERATED FROM PYTHON SOURCE LINES 41-47 Retrieve the float-to-int2 cast case ++++++++++++++++++++++++++++++++++++ ``collect_test_case`` returns every registered backend test case as a ``dict`` mapping each case name to its ``TestCase``. We pick the ``test_cc_cast_FLOAT_to_INT2`` entry from it. .. GENERATED FROM PYTHON SOURCE LINES 47-56 .. code-block:: Python all_cases = collect_test_case() print(f"Total number of backend test cases: {len(all_cases)}") tc = all_cases["test_cc_cast_FLOAT_to_INT2"] print(f"name : {tc.name}") print(f"model_name: {tc.model_name}") print(f"kind : {tc.kind}") .. rst-class:: sphx-glr-script-out .. code-block:: none Total number of backend test cases: 2107 name : test_cc_cast_FLOAT_to_INT2 model_name: test_cc_cast_FLOAT_to_INT2 kind : node .. GENERATED FROM PYTHON SOURCE LINES 57-63 Display the model +++++++++++++++++ The model is a single ``Cast`` node. The ``to`` attribute is ``26``, the ONNX ``TensorProto.INT2`` data type. The graph output is declared with ``elem_type: 26`` accordingly. .. GENERATED FROM PYTHON SOURCE LINES 63-66 .. code-block:: Python print(pretty_onnx(tc.model)) .. rst-class:: sphx-glr-script-out .. code-block:: none opset: domain='' version=25 graph: name='test_cc_cast_FLOAT_to_INT2' input: float[7,1] input 0: Cast(input) -> output output: dtype26[7,1] output .. GENERATED FROM PYTHON SOURCE LINES 67-76 Run the model with the reference runtime ++++++++++++++++++++++++++++++++++++++++ :class:`~onnx_light.onnx.reference.ReferenceEvaluator` runs the model with the C++ reference kernels. The input is the same ``np.arange(-3, 4)`` float32 sweep the test case uses, reshaped to the model's ``(7, 1)`` input shape. The runtime returns an ``INT2`` numpy array (backed by ``ml_dtypes.int2``); values below ``-2`` or above ``1`` saturate to the representable range. .. GENERATED FROM PYTHON SOURCE LINES 76-91 .. code-block:: Python session = ReferenceEvaluator(tc.model) x = np.arange(-3, 4, dtype=np.float32).reshape(7, 1) print("input (float32):") print(x.ravel()) output = session.run(None, {"input": x})[0] print(f"output type: {type(output)}") print(f"output dtype: {output.dtype}") print(f"output shape: {output.shape}") print("output (int2, saturated to [-2, 1]):") # Cast to int8 only for a readable decimal print of the 2-bit values. print(output.astype(np.int8).ravel()) .. rst-class:: sphx-glr-script-out .. code-block:: none input (float32): [-3. -2. -1. 0. 1. 2. 3.] output type: output dtype: int2 output shape: (7, 1) output (int2, saturated to [-2, 1]): [ 1 -2 -1 0 1 -2 -1] .. GENERATED FROM PYTHON SOURCE LINES 92-102 Run the corresponding backend test ++++++++++++++++++++++++++++++++++ The same case retrieved with ``collect_test_case`` *is* the backend test: every :class:`TestCase` carries the reference input/output data sets and an :meth:`~onnx_light.onnx_lib.backend.test.case.base.TestCase.assert_allclose` method. A backend test only needs a runtime callable with the signature ``rt(model, *inputs) -> list[np.ndarray]``; ``assert_allclose`` feeds each data set through it and compares the outputs against the expected tensors (using the case ``atol`` / ``rtol``). .. GENERATED FROM PYTHON SOURCE LINES 102-122 .. code-block:: Python def reference_runtime(model, *inputs: np.ndarray) -> list[np.ndarray]: """Runs *model* on *inputs* with the reference runtime. Returns: The model outputs as a list of numpy arrays, in graph-output order, as expected by ``TestCase.assert_allclose``. """ sess = ReferenceEvaluator(model) feeds = {i.name: arr for i, arr in zip(model.graph.input, inputs)} return sess.run(None, feeds) # ``tc`` was retrieved above from ``collect_test_case()``; run its # backend test directly. ``assert_allclose`` raises an ``AssertionError`` # on a mismatch and returns ``None`` on success. tc.assert_allclose(reference_runtime) print(f"Backend test {tc.name!r} passed.") .. rst-class:: sphx-glr-script-out .. code-block:: none Backend test 'test_cc_cast_FLOAT_to_INT2' passed. .. GENERATED FROM PYTHON SOURCE LINES 123-140 Running every backend test from the command line +++++++++++++++++++++++++++++++++++++++++++++++++ To turn the whole registry into a :class:`unittest.TestCase` (one ``test_`` method per collected case), pass the same runtime to :func:`~onnx_light.onnx.backend.make_test_class`, which calls ``collect_test_case`` internally. In practice you would place this in its own test file and run it with pytest or unittest, optionally narrowing to this case with ``-k``:: from onnx_light.onnx.backend import make_test_class MyBackendTests = make_test_class(reference_runtime) # python -m pytest my_backend_tests.py -v -k cast_FLOAT_to_INT2 See :ref:`l-design-backend-tests` for the full backend-test workflow. .. GENERATED FROM PYTHON SOURCE LINES 142-147 Gallery thumbnail +++++++++++++++++ Render a simple text figure used as the sphinx-gallery thumbnail for this example. .. GENERATED FROM PYTHON SOURCE LINES 147-154 .. code-block:: Python import matplotlib.pyplot as plt # noqa: E402 fig, ax = plt.subplots(figsize=(4, 3)) ax.text(0.5, 0.5, "float\n\u2192\nint2", ha="center", va="center", fontsize=28) ax.set_axis_off() fig.tight_layout() .. image-sg:: /auto_examples_backend/images/sphx_glr_plot_run_cast_to_int2_001.png :alt: plot run cast to int2 :srcset: /auto_examples_backend/images/sphx_glr_plot_run_cast_to_int2_001.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 0.155 seconds) .. _sphx_glr_download_auto_examples_backend_plot_run_cast_to_int2.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_run_cast_to_int2.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_run_cast_to_int2.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_run_cast_to_int2.zip ` .. include:: plot_run_cast_to_int2.recommendations .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_