.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples_core/plot_pattern_optimization.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_core_plot_pattern_optimization.py: .. _l-example-plot-pattern-optimization: Optimizing a model with graph-rewriting patterns ================================================= *onnx-light* optimizes a model by repeatedly matching small :class:`~onnx_light.onnx_core.optimization.PatternOptimization` subgraphs and replacing them with a simplified equivalent. This example walks through the whole workflow on a tiny model: 1. Build a model containing redundant nodes (two useless ``Cast`` and two consecutive ``Neg``). 2. Run the standard patterns and inspect the optimization *statistics* (:class:`~onnx_light.onnx_core.optimization.OptimizationReport`). 3. List every applied modification (:class:`~onnx_light.onnx_core.optimization.LocalRewriting`) and *replay* them from the original model to reproduce the optimized graph. 4. Write a custom pattern. 5. Inspect a candidate rejected by that pattern and the recorded failure reason. 6. Apply the custom pattern together with the standard ones. See :ref:`l-howto-add-custom-pattern` for a companion how-to that focuses on writing and registering a pattern (including how priorities order patterns), in both Python and C++. .. GENERATED FROM PYTHON SOURCE LINES 27-41 .. code-block:: Python from __future__ import annotations import onnx_light.onnx.helper as oh from onnx_light.onnx_lib import parser from onnx_light.onnx_core.optimization import ( GraphBuilder, GraphGraph, PatternOptimization, replay, standard_patterns, ) from onnx_light.tools import pretty_onnx .. GENERATED FROM PYTHON SOURCE LINES 42-49 Build a model with redundant nodes ++++++++++++++++++++++++++++++++++ ``x`` is cast to ``float`` (a no-op, since it is already ``float``) and then negated twice before being cast back. The standard ``Cast`` pattern removes the two ``Cast`` nodes; the custom pattern added further below removes the two ``Neg`` nodes. .. GENERATED FROM PYTHON SOURCE LINES 49-61 .. code-block:: Python model = parser.parse_model( '\n' "agraph (float[4] x) => (float[4] y) {\n" " casted = Cast (x)\n" " middle = Neg(casted)\n" " negated = Neg(middle)\n" " y = Cast (negated)\n" "}\n" ) print(pretty_onnx(model)) .. rst-class:: sphx-glr-script-out .. code-block:: none opset: domain='' version=18 graph: name='agraph' input: float[4] x 0: Cast(x) -> casted 1: Neg(casted) -> middle 2: Neg(middle) -> negated 3: Cast(negated) -> y output: float[4] y .. GENERATED FROM PYTHON SOURCE LINES 62-69 Run the standard patterns and read the statistics +++++++++++++++++++++++++++++++++++++++++++++++++ :meth:`~onnx_light.onnx_core.optimization.GraphGraph.optimize` returns the list of applied rewrites; passing ``report=True`` additionally returns an :class:`~onnx_light.onnx_core.optimization.OptimizationReport` with timing and match/no-match counters for every pattern that was tried. .. GENERATED FROM PYTHON SOURCE LINES 69-84 .. code-block:: Python builder = GraphBuilder(model) graph = GraphGraph(builder, standard_patterns(["Cast"])) rewrites, report = graph.optimize(report=True) optimized_graph = builder.build_graph() print(pretty_onnx(builder.to_onnx("model"))) print(report) for pattern_stats in report.patterns: print( f"{pattern_stats.pattern_name}: {pattern_stats.matches} match(es) over " f"{pattern_stats.attempts} attempt(s)" ) .. rst-class:: sphx-glr-script-out .. code-block:: none opset: domain='ai.onnx' version=18 graph: name='agraph' input: float[4] x 0: Neg(x) -> middle 1: Neg(middle) -> negated 2: Identity(negated) -> y output: float[4] y OptimizationReport(iterations=5, rewrites=3, total_time_ns=125049, phases={matching: 34953, rewriting: 22573, cleanup: 39040, constant_folding: 28483, subgraph_optimization: 0}, patterns=[Cast(attempts=2, matches=2, match_time_ns=2493, apply_time_ns=5168), ConvBiasNull(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), Identity(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), PadConv(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ConcatEmpty(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ConcatGather(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ConcatTwiceUnary(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), GatherConcat(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), GatherGather(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), GatherShape(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), GathersSplit(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), SliceSlice(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), SlicesSplit(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), SplitConcat(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), SequenceConstructAt(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), SplitToSequenceSequenceAt(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), NotWhere(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), UnsqueezeEqual(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), WhereAdd(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), Expand(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ExpandBroadcast(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ShapeBasedConcatExpand(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ShapeBasedExpandBroadcast(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ShapeBasedExpandBroadcastMatMul(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ShapeBasedStaticExpand(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ShapeBasedExpandSwap(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ShapeBasedExpandCastWhereSwap(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ExpandSwap(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), SwapExpandUnsqueeze(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ExpandUnsqueezeExpand(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ConcatReshape(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), Reshape(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ReduceReshape(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), Reshape2Of3(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ReshapeReshapeBinary(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ReshapeReshape(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ReshapeSqueeze(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ShapeBasedEditDistanceReshape(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ShapeBasedReshapeIsSqueeze(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ShapedBasedReshape(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), StaticConcatReshape(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), UnsqueezeOrSqueezeReshape(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), UnsqueezeReshape(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), MulUnsqueezeUnsqueeze(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), SqueezeAdd(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), SqueezeBinaryUnsqueeze(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), SwapUnsqueezeTranspose(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), TransposeEqualReshape(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), TransposeReshapeTranspose(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), MulMulMulScalar(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), SwitchOrderBinary(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), SwapRangeAddScalar(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ReduceArgTopK(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ReduceSumNormalize(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), Sub1Mul(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), SwapUnary(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), SameChildren(attempts=16, matches=0, match_time_ns=3745, apply_time_ns=0, no_matches=[/home/runner/work/xadupre.github.io/xadupre.github.io/onnx-light/onnx_light/onnx_extensions/patterns/algebra/common_pattern.cc:339(occurrences=16, reason=no equivalent sibling nodes found)]), SameChildrenFromInput(attempts=16, matches=0, match_time_ns=1993, apply_time_ns=0, no_matches=[/home/runner/work/xadupre.github.io/xadupre.github.io/onnx-light/onnx_light/onnx_extensions/patterns/algebra/common_pattern.cc:387(occurrences=5, reason=the graph input has fewer than two consumers), /home/runner/work/xadupre.github.io/xadupre.github.io/onnx-light/onnx_light/onnx_extensions/patterns/algebra/common_pattern.cc:383(occurrences=11, reason=the candidate first input is not a graph input)]), ShapeBasedIdentity(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ShapeBasedSameChildren(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ShapeBasedShapeShapeAdd(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), BatchNormalization(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), BatchNormalizationTraining(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), Gelu(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), LeakyRelu(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), SoftmaxCrossEntropyLossCast(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), FunctionAttention(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), SwapExpandReshape(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), TransposeTranspose(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), TransposeGather(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), UnsqueezeUnsqueeze(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), SqueezeUnsqueeze(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ShapeTranspose(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), UnsqueezeShape(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), CastCast(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), CastCastBinary(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), CastOpCast(attempts=6, matches=0, match_time_ns=3053, apply_time_ns=0, no_matches=[/home/runner/work/xadupre.github.io/xadupre.github.io/onnx-light/onnx_light/onnx_extensions/patterns/canonicalization/cast_pattern.cc:321(occurrences=6, reason=the operation output must feed one unary Cast node)]), ClipClip(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ConstantToInitializer(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), Dropout(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), NotNot(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), GemmTranspose(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), MatMulReshape2Of3(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), MulMulMatMul(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ReshapeMatMulReshape(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), ShapeBasedMatMulToMul(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), SwitchReshapeActivation(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), TransposeMatMul(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), TransposeReshapeMatMul(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), CastLayerNormalizationCast(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), LayerNormalization(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), LayerNormalizationScale(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), RMSNormalization(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), RMSNormalizationMul(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), MaxRelu(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), RotaryEmbedding(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), RotaryConcatPart(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), FunctionCausalMask(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), FunctionCausalMaskMulAdd(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), FunctionCosSinCache(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), FunctionHalfRotaryEmbedding(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0), FunctionAttentionGQA(attempts=9, matches=0, match_time_ns=2355, apply_time_ns=0, no_matches=[/home/runner/work/xadupre.github.io/xadupre.github.io/onnx-light/onnx_light/onnx_extensions/patterns/attention/attention_pattern.cc:773(occurrences=9, reason=FunctionAttentionGQAPattern expects a LocalAttention call.)]), AttentionGQA(attempts=6, matches=0, match_time_ns=1153, apply_time_ns=0, no_matches=[/home/runner/work/xadupre.github.io/xadupre.github.io/onnx-light/onnx_light/onnx_extensions/patterns/attention/attention_pattern.cc:871(occurrences=6, reason=AttentionGQAPattern requires opset 23 and one output.)]), MatMulAdd(attempts=0, matches=0, match_time_ns=0, apply_time_ns=0)], subgraphs=[]) Cast: 2 match(es) over 2 attempt(s) ConvBiasNull: 0 match(es) over 0 attempt(s) Identity: 0 match(es) over 0 attempt(s) PadConv: 0 match(es) over 0 attempt(s) ConcatEmpty: 0 match(es) over 0 attempt(s) ConcatGather: 0 match(es) over 0 attempt(s) ConcatTwiceUnary: 0 match(es) over 0 attempt(s) GatherConcat: 0 match(es) over 0 attempt(s) GatherGather: 0 match(es) over 0 attempt(s) GatherShape: 0 match(es) over 0 attempt(s) GathersSplit: 0 match(es) over 0 attempt(s) SliceSlice: 0 match(es) over 0 attempt(s) SlicesSplit: 0 match(es) over 0 attempt(s) SplitConcat: 0 match(es) over 0 attempt(s) SequenceConstructAt: 0 match(es) over 0 attempt(s) SplitToSequenceSequenceAt: 0 match(es) over 0 attempt(s) NotWhere: 0 match(es) over 0 attempt(s) UnsqueezeEqual: 0 match(es) over 0 attempt(s) WhereAdd: 0 match(es) over 0 attempt(s) Expand: 0 match(es) over 0 attempt(s) ExpandBroadcast: 0 match(es) over 0 attempt(s) ShapeBasedConcatExpand: 0 match(es) over 0 attempt(s) ShapeBasedExpandBroadcast: 0 match(es) over 0 attempt(s) ShapeBasedExpandBroadcastMatMul: 0 match(es) over 0 attempt(s) ShapeBasedStaticExpand: 0 match(es) over 0 attempt(s) ShapeBasedExpandSwap: 0 match(es) over 0 attempt(s) ShapeBasedExpandCastWhereSwap: 0 match(es) over 0 attempt(s) ExpandSwap: 0 match(es) over 0 attempt(s) SwapExpandUnsqueeze: 0 match(es) over 0 attempt(s) ExpandUnsqueezeExpand: 0 match(es) over 0 attempt(s) ConcatReshape: 0 match(es) over 0 attempt(s) Reshape: 0 match(es) over 0 attempt(s) ReduceReshape: 0 match(es) over 0 attempt(s) Reshape2Of3: 0 match(es) over 0 attempt(s) ReshapeReshapeBinary: 0 match(es) over 0 attempt(s) ReshapeReshape: 0 match(es) over 0 attempt(s) ReshapeSqueeze: 0 match(es) over 0 attempt(s) ShapeBasedEditDistanceReshape: 0 match(es) over 0 attempt(s) ShapeBasedReshapeIsSqueeze: 0 match(es) over 0 attempt(s) ShapedBasedReshape: 0 match(es) over 0 attempt(s) StaticConcatReshape: 0 match(es) over 0 attempt(s) UnsqueezeOrSqueezeReshape: 0 match(es) over 0 attempt(s) UnsqueezeReshape: 0 match(es) over 0 attempt(s) MulUnsqueezeUnsqueeze: 0 match(es) over 0 attempt(s) SqueezeAdd: 0 match(es) over 0 attempt(s) SqueezeBinaryUnsqueeze: 0 match(es) over 0 attempt(s) SwapUnsqueezeTranspose: 0 match(es) over 0 attempt(s) TransposeEqualReshape: 0 match(es) over 0 attempt(s) TransposeReshapeTranspose: 0 match(es) over 0 attempt(s) MulMulMulScalar: 0 match(es) over 0 attempt(s) SwitchOrderBinary: 0 match(es) over 0 attempt(s) SwapRangeAddScalar: 0 match(es) over 0 attempt(s) ReduceArgTopK: 0 match(es) over 0 attempt(s) ReduceSumNormalize: 0 match(es) over 0 attempt(s) Sub1Mul: 0 match(es) over 0 attempt(s) SwapUnary: 0 match(es) over 0 attempt(s) SameChildren: 0 match(es) over 16 attempt(s) SameChildrenFromInput: 0 match(es) over 16 attempt(s) ShapeBasedIdentity: 0 match(es) over 0 attempt(s) ShapeBasedSameChildren: 0 match(es) over 0 attempt(s) ShapeBasedShapeShapeAdd: 0 match(es) over 0 attempt(s) BatchNormalization: 0 match(es) over 0 attempt(s) BatchNormalizationTraining: 0 match(es) over 0 attempt(s) Gelu: 0 match(es) over 0 attempt(s) LeakyRelu: 0 match(es) over 0 attempt(s) SoftmaxCrossEntropyLossCast: 0 match(es) over 0 attempt(s) FunctionAttention: 0 match(es) over 0 attempt(s) SwapExpandReshape: 0 match(es) over 0 attempt(s) TransposeTranspose: 0 match(es) over 0 attempt(s) TransposeGather: 0 match(es) over 0 attempt(s) UnsqueezeUnsqueeze: 0 match(es) over 0 attempt(s) SqueezeUnsqueeze: 0 match(es) over 0 attempt(s) ShapeTranspose: 0 match(es) over 0 attempt(s) UnsqueezeShape: 0 match(es) over 0 attempt(s) CastCast: 0 match(es) over 0 attempt(s) CastCastBinary: 0 match(es) over 0 attempt(s) CastOpCast: 0 match(es) over 6 attempt(s) ClipClip: 0 match(es) over 0 attempt(s) ConstantToInitializer: 0 match(es) over 0 attempt(s) Dropout: 0 match(es) over 0 attempt(s) NotNot: 0 match(es) over 0 attempt(s) GemmTranspose: 0 match(es) over 0 attempt(s) MatMulReshape2Of3: 0 match(es) over 0 attempt(s) MulMulMatMul: 0 match(es) over 0 attempt(s) ReshapeMatMulReshape: 0 match(es) over 0 attempt(s) ShapeBasedMatMulToMul: 0 match(es) over 0 attempt(s) SwitchReshapeActivation: 0 match(es) over 0 attempt(s) TransposeMatMul: 0 match(es) over 0 attempt(s) TransposeReshapeMatMul: 0 match(es) over 0 attempt(s) CastLayerNormalizationCast: 0 match(es) over 0 attempt(s) LayerNormalization: 0 match(es) over 0 attempt(s) LayerNormalizationScale: 0 match(es) over 0 attempt(s) RMSNormalization: 0 match(es) over 0 attempt(s) RMSNormalizationMul: 0 match(es) over 0 attempt(s) MaxRelu: 0 match(es) over 0 attempt(s) RotaryEmbedding: 0 match(es) over 0 attempt(s) RotaryConcatPart: 0 match(es) over 0 attempt(s) FunctionCausalMask: 0 match(es) over 0 attempt(s) FunctionCausalMaskMulAdd: 0 match(es) over 0 attempt(s) FunctionCosSinCache: 0 match(es) over 0 attempt(s) FunctionHalfRotaryEmbedding: 0 match(es) over 0 attempt(s) FunctionAttentionGQA: 0 match(es) over 9 attempt(s) AttentionGQA: 0 match(es) over 6 attempt(s) MatMulAdd: 0 match(es) over 0 attempt(s) .. GENERATED FROM PYTHON SOURCE LINES 85-93 List the modifications and replay them ++++++++++++++++++++++++++++++++++++++ Each :class:`~onnx_light.onnx_core.optimization.LocalRewriting` records which pattern fired, the positions of the matched nodes, and the nodes it added. :func:`~onnx_light.onnx_core.optimization.replay` reconstructs the optimized graph by reapplying that captured sequence to a **fresh copy** of the original model, without running the pattern matcher again. .. GENERATED FROM PYTHON SOURCE LINES 93-101 .. code-block:: Python for rewrite in rewrites: print(rewrite) replayed_graph = replay(model, rewrites) assert replayed_graph.SerializeToString() == optimized_graph.SerializeToString() print("replay reproduced the optimized graph") .. rst-class:: sphx-glr-script-out .. code-block:: none LocalRewriting(pattern=Cast, graph_path=[], matched_nodes=[0], added_nodes=[Identity(outputs=[casted])], added_nodes_positions=[0], added_initializers=[], added_initializer_positions=[], removed_initializers=[], value_renames=[], iteration=0, match_time_ns=2313, apply_time_ns=4667) LocalRewriting(pattern=Cast, graph_path=[], matched_nodes=[3], added_nodes=[Identity(outputs=[y])], added_nodes_positions=[3], added_initializers=[], added_initializer_positions=[], removed_initializers=[], value_renames=[], iteration=0, match_time_ns=180, apply_time_ns=501) LocalRewriting(pattern=RemoveIdentityNodes, graph_path=[], matched_nodes=[0, 1, 2, 3], added_nodes=[Neg(outputs=[middle]), Neg(outputs=[negated]), Identity(outputs=[y])], added_nodes_positions=[0, 1, 2], added_initializers=[], added_initializer_positions=[], removed_initializers=[], value_renames=[casted->x], iteration=1, match_time_ns=0, apply_time_ns=0) replay reproduced the optimized graph .. GENERATED FROM PYTHON SOURCE LINES 102-112 Add a custom pattern ++++++++++++++++++++ A pattern derives from :class:`~onnx_light.onnx_core.optimization.PatternOptimization` and implements ``fast_op_type`` (the operator types it may start from), ``match`` (returns ``self.result(...)`` on success or ``self.no_match(...)`` with a diagnostic otherwise) and ``apply`` (builds the replacement nodes). Passing it to ``GraphGraph`` alongside the standard patterns runs both together, in ascending ``priority`` order. .. GENERATED FROM PYTHON SOURCE LINES 112-135 .. code-block:: Python class NegNegPattern(PatternOptimization): """Replaces two consecutive Neg nodes with Identity.""" def __init__(self): super().__init__(priority=1, name="NegNeg") def fast_op_type(self): return {"Neg"} def match(self, graph, node): previous = graph.node_before(node.input[0]) if previous is None or previous.op_type != "Neg": return self.no_match(node, "the input is not produced by Neg") return self.result([previous, node], insert_at=node) def apply(self, graph, nodes): del graph previous, node = nodes return [oh.make_node("Identity", [previous.input[0]], list(node.output))] .. GENERATED FROM PYTHON SOURCE LINES 136-143 Inspect a failed pattern candidate ++++++++++++++++++++++++++++++++++ The same custom pattern rejects a single ``Neg`` because its input is not produced by another ``Neg``. Returning ``self.no_match(candidate, reason)`` keeps that rejection out of normal output, but the optional report stores the aggregated reason so the pattern author can understand why nothing changed. .. GENERATED FROM PYTHON SOURCE LINES 143-171 .. code-block:: Python failure_model = parser.parse_model( '\n' "agraph (float[4] x) => (float[4] y) {\n" " y = Neg(x)\n" "}\n" ) builder = GraphBuilder(failure_model) graph = GraphGraph(builder, [NegNegPattern()]) failed_rewrites, failed_report = graph.optimize(report=True) print(f"failed rewrite count: {len(failed_rewrites)}") for pattern_stats in failed_report.patterns: for no_match in pattern_stats.no_matches: print( f"{pattern_stats.pattern_name} rejected " f"{no_match.occurrences} candidate(s): {no_match.reason}" ) assert not failed_rewrites failed_no_match_reasons = { no_match.reason for pattern_stats in failed_report.patterns if pattern_stats.pattern_name == "NegNeg" for no_match in pattern_stats.no_matches } assert "the input is not produced by Neg" in failed_no_match_reasons .. rst-class:: sphx-glr-script-out .. code-block:: none failed rewrite count: 0 SameChildren rejected 4 candidate(s): no equivalent sibling nodes found SameChildrenFromInput rejected 4 candidate(s): the graph input has fewer than two consumers CastOpCast rejected 3 candidate(s): the operation output must feed one unary Cast node FunctionAttentionGQA rejected 3 candidate(s): FunctionAttentionGQAPattern expects a LocalAttention call. NegNeg rejected 3 candidate(s): the input is not produced by Neg AttentionGQA rejected 2 candidate(s): AttentionGQAPattern requires opset 23 and one output. .. GENERATED FROM PYTHON SOURCE LINES 172-174 Apply the custom pattern with the standard patterns +++++++++++++++++++++++++++++++++++++++++++++++++++ .. GENERATED FROM PYTHON SOURCE LINES 174-183 .. code-block:: Python builder = GraphBuilder(model) graph = GraphGraph(builder, [*standard_patterns(["Cast"]), NegNegPattern()]) rewrites = graph.optimize() fully_optimized = builder.to_onnx("model") print(pretty_onnx(fully_optimized)) print([rewrite.pattern_name for rewrite in rewrites]) assert [node.op_type for node in fully_optimized.graph.node] == ["Identity"] .. rst-class:: sphx-glr-script-out .. code-block:: none opset: domain='ai.onnx' version=18 graph: name='agraph' input: float[4] x 0: Identity(x) -> y output: float[4] y ['Cast', 'Cast', 'RemoveIdentityNodes', 'NegNeg', 'RemoveIdentityNodes'] .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 0.006 seconds) .. _sphx_glr_download_auto_examples_core_plot_pattern_optimization.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_pattern_optimization.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_pattern_optimization.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_pattern_optimization.zip ` .. include:: plot_pattern_optimization.recommendations .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_