:orphan:

Optimization Examples
=====================

Examples demonstrating the optimization features of *onnx-light*
exposed by :mod:`onnx_light.onnx_optim` (shape inference, expression
helpers, ...).


.. raw:: html

  <div id='sg-tag-list' class='sphx-glr-tag-list'></div>


.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="ComputeContext reports, for each node, how much memory is already live before execution, how much extra output allocation is still needed, and the resulting total. When input shapes are symbolic, these quantities stay symbolic as well.">

.. only:: html

  .. image:: /auto_examples_optimization/images/thumb/sphx_glr_plot_compute_context_memory_thumb.png
    :alt:

  :doc:`/auto_examples_optimization/plot_compute_context_memory`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">ComputeContext memory expressions</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="ONNX models commonly use symbolic names — sometimes called tokens — for the dimensions of their inputs (``&quot;batch&quot;``, &quot;seq_length&quot;, &quot;d_model&quot;, ...).  After running shape inference, every intermediate tensor&#x27;s shape is expressed in terms of those input tokens, either as a plain dim_param (``&quot;batch&quot;``) or as a symbolic arithmetic expression (``&quot;2d_model&quot;``, &quot;batchseq_length&quot;, ...).">

.. only:: html

  .. image:: /auto_examples_optimization/images/thumb/sphx_glr_plot_evaluate_shapes_thumb.png
    :alt:

  :doc:`/auto_examples_optimization/plot_evaluate_shapes`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Evaluating inferred shapes with concrete input dimensions</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example compares three approaches to shape inference on a graph that mirrors a common transformer pattern:">

.. only:: html

  .. image:: /auto_examples_optimization/images/thumb/sphx_glr_plot_shape_inference_thumb.png
    :alt:

  :doc:`/auto_examples_optimization/plot_shape_inference`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Optimized Shape inference</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Most ONNX shape inference relies on the built-in dispatch table that knows the schemas of standard operators. Real-world models often include custom operators from a private domain. For those, the built-in inference has no way to compute output shapes and stops with an error.">

.. only:: html

  .. image:: /auto_examples_optimization/images/thumb/sphx_glr_plot_shape_inference_custom_op_thumb.png
    :alt:

  :doc:`/auto_examples_optimization/plot_shape_inference_custom_op`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Shape inference with a custom operator</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="ONNX models frequently use symbolic tensor shapes: instead of a concrete integer such as 128, a dimension carries a name like &quot;batch&quot; or &quot;seq_length&quot;.  During shape inference and graph optimisation it is necessary to compare, simplify, evaluate, and rename those symbolic expressions.">

.. only:: html

  .. image:: /auto_examples_optimization/images/thumb/sphx_glr_plot_expressions_thumb.png
    :alt:

  :doc:`/auto_examples_optimization/plot_expressions`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Symbolic expressions for dimensions</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="pretty_onnx renders any ONNX proto as a compact, human-readable text listing.  Beyond the basic node-by-node view it supports four optional annotation layers:">

.. only:: html

  .. image:: /auto_examples_optimization/images/thumb/sphx_glr_plot_pretty_onnx_thumb.png
    :alt:

  :doc:`/auto_examples_optimization/plot_pretty_onnx`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">pretty_onnx: shape info, shape tags, inplace and release annotations</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples_optimization/plot_compute_context_memory
   /auto_examples_optimization/plot_evaluate_shapes
   /auto_examples_optimization/plot_shape_inference
   /auto_examples_optimization/plot_shape_inference_custom_op
   /auto_examples_optimization/plot_expressions
   /auto_examples_optimization/plot_pretty_onnx


.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-gallery

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download all examples in Python source code: auto_examples_optimization_python.zip </auto_examples_optimization/auto_examples_optimization_python.zip>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download all examples in Jupyter notebooks: auto_examples_optimization_jupyter.zip </auto_examples_optimization/auto_examples_optimization_jupyter.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_