.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples_runtime/plot_kernel_tuning.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_runtime_plot_kernel_tuning.py: .. _l-example-plot-kernel-tuning: Inspect, change, and calibrate kernel tuning from Python ======================================================== This example uses :mod:`onnx_light.kernel_tuning` to discover every tuning parameter used by one exact kernel, compare its portable and local values, write a validated local profile, and run a bounded calibration. The example writes only to a temporary cache. Real applications may omit ``path`` to use :func:`~onnx_light.kernel_tuning.default_kernel_tuning_cache_path`. .. GENERATED FROM PYTHON SOURCE LINES 14-28 .. code-block:: Python from __future__ import annotations import tempfile from pathlib import Path from pprint import pprint import numpy as np from onnx_light import kernel_tuning from onnx_light.onnx import TensorProto from onnx_light.onnx.reference import ReferenceEvaluator from onnx_light.onnx_lib import parser .. GENERATED FROM PYTHON SOURCE LINES 29-35 Discover the parameters and defaults ++++++++++++++++++++++++++++++++++++ A tuning schema is registered for every exact combination of library, implementation, element type, device, and tuning ABI. ``Abs`` uses one parallel crossover threshold. .. GENERATED FROM PYTHON SOURCE LINES 35-42 .. code-block:: Python element_type = int(TensorProto.FLOAT) initial = kernel_tuning.kernel_tuning_parameters(kernel="Abs", element_type=element_type) (abs_parameters,) = initial["kernels"] print(f"default cache: {initial['cache_path']}") pprint(abs_parameters) .. rst-class:: sphx-glr-script-out .. code-block:: none default cache: /home/runner/.cache/onnx-light/kernel_tuning.cache {'active_source': 'portable_default', 'active_values': {'parallel.minimum_elements': 32768}, 'cached_values': None, 'calibratable': True, 'defaults': {'parallel.minimum_elements': 32768}, 'device': -1, 'element_type': 1, 'implementation': 'portable', 'kernel': 'Abs', 'library': 'onnx_light', 'parameter_names': ['parallel.minimum_elements'], 'tuning_abi': 2} .. GENERATED FROM PYTHON SOURCE LINES 43-49 Propose missing profiles ++++++++++++++++++++++++ A proposal is read-only. It compares the requested exact keys with the local cache and separates keys that can be calibrated automatically from those without callbacks. .. GENERATED FROM PYTHON SOURCE LINES 49-59 .. code-block:: Python temporary = tempfile.TemporaryDirectory() missing_cache = Path(temporary.name) / "missing_tuning.cache" proposal = kernel_tuning.propose_kernel_tuning_updates( kernels=["Abs"], element_types=[element_type], path=str(missing_cache) ) assert len(proposal["calibratable"]) == 1 print("proposed calibrations:") pprint(proposal["calibratable"]) .. rst-class:: sphx-glr-script-out .. code-block:: none proposed calibrations: [{'active_source': 'portable_default', 'active_values': {'parallel.minimum_elements': 32768}, 'cached_values': None, 'calibratable': True, 'defaults': {'parallel.minimum_elements': 32768}, 'device': -1, 'element_type': 1, 'implementation': 'portable', 'kernel': 'Abs', 'library': 'onnx_light', 'parameter_names': ['parallel.minimum_elements'], 'tuning_abi': 2}] .. GENERATED FROM PYTHON SOURCE LINES 60-67 Write a validated profile +++++++++++++++++++++++++ ``set_kernel_tuning_parameters`` accepts a partial dictionary. It fills omitted names from an existing matching cache profile or the portable defaults, validates the complete set, persists it atomically, and loads it into the current process by default. .. GENERATED FROM PYTHON SOURCE LINES 67-79 .. code-block:: Python cache_path = Path(temporary.name) / "kernel_tuning.cache" portable_minimum = abs_parameters["defaults"]["parallel.minimum_elements"] chosen_minimum = max(1, portable_minimum // 2) update = kernel_tuning.set_kernel_tuning_parameters( "Abs", element_type, {"parallel.minimum_elements": chosen_minimum}, path=str(cache_path) ) assert update["status"] == "updated", update["diagnostics"] print("updated profile:") pprint(update) .. rst-class:: sphx-glr-script-out .. code-block:: none updated profile: {'diagnostics': [], 'load': {'diagnostics': [], 'incompatible': [], 'invalid': [], 'loaded': [{'device': -1, 'element_type': 1, 'implementation': 'portable', 'kernel': 'Abs', 'library': 'onnx_light', 'tuning_abi': 2}], 'missing': [], 'path': '/tmp/tmpglcmoxin/kernel_tuning.cache', 'published_generation': 185, 'stale': [], 'status': 'loaded'}, 'path': '/tmp/tmpglcmoxin/kernel_tuning.cache', 'preserved': [], 'pruned': [], 'status': 'updated', 'updated': [{'device': -1, 'element_type': 1, 'implementation': 'portable', 'kernel': 'Abs', 'library': 'onnx_light', 'tuning_abi': 2}], 'values': {'parallel.minimum_elements': 16384}} .. GENERATED FROM PYTHON SOURCE LINES 80-86 Compare cache and active values +++++++++++++++++++++++++++++++ Inspection reads every persisted profile without changing the registry. ``kernel_tuning_parameters`` separately reports the matching local cache values and the values currently published in this process. .. GENERATED FROM PYTHON SOURCE LINES 86-101 .. code-block:: Python inspection = kernel_tuning.inspect_kernel_tuning_cache(str(cache_path)) assert inspection["status"] == "loaded" assert inspection["profiles"][0]["local"] print("cache profiles:") pprint(inspection["profiles"]) current = kernel_tuning.kernel_tuning_parameters( kernel="Abs", element_type=element_type, path=str(cache_path) ) (abs_tuning,) = current["kernels"] assert abs_tuning["cached_values"]["parallel.minimum_elements"] == chosen_minimum assert abs_tuning["active_values"]["parallel.minimum_elements"] == chosen_minimum print("active source:", abs_tuning["active_source"]) .. rst-class:: sphx-glr-script-out .. code-block:: none cache profiles: [{'architecture': 'x86_64', 'device': -1, 'effective_threads': 2, 'element_type': 1, 'implementation': 'portable', 'kernel': 'Abs', 'library': 'onnx_light', 'local': True, 'microarchitecture': '', 'tuning_abi': 2, 'values': {'parallel.minimum_elements': 16384}, 'vendor': 'amd'}] active source: published_profile .. GENERATED FROM PYTHON SOURCE LINES 102-108 Use the active value ++++++++++++++++++++ A session created after the profile is loaded resolves it once and copies the typed value into its ``Abs`` kernel. Steady-state calls do not read the cache or registry again. .. GENERATED FROM PYTHON SOURCE LINES 108-119 .. code-block:: Python model = parser.parse_model( '' "agraph (float[4] x) => (float[4] y) { y = Abs(x) }" ) session = ReferenceEvaluator(model) x = np.array([-1.0, 2.0, -3.5, 0.0], dtype=np.float32) (y,) = session.run(None, {"x": x}) np.testing.assert_array_equal(y, np.abs(x)) print("Abs output:", y) .. rst-class:: sphx-glr-script-out .. code-block:: none Abs output: [1. 2. 3.5 0. ] .. GENERATED FROM PYTHON SOURCE LINES 120-127 Calibrate the kernel ++++++++++++++++++++ Calibration compares deterministic candidate runs with the forced serial implementation, validates every output, and searches for a stable crossover. ``save=False`` publishes the result only in this process. Set ``save=True`` (the default) to merge it into the selected cache. .. GENERATED FROM PYTHON SOURCE LINES 127-142 .. code-block:: Python calibration = kernel_tuning.calibrate_kernel_tuning( "Abs", element_types=[element_type], maximum_duration_ms=100, maximum_memory_bytes=16 << 20, save=False, ) assert len(calibration["calibrated"]) == 1 print("calibrated profile:") pprint(calibration["calibrated"][0]) print("diagnostics:") pprint(calibration["diagnostics"]) temporary.cleanup() .. rst-class:: sphx-glr-script-out .. code-block:: none calibrated profile: {'device': -1, 'element_type': 1, 'implementation': 'portable', 'kernel': 'Abs', 'library': 'onnx_light', 'tuning_abi': 2, 'values': {'parallel.minimum_elements': 32768}} diagnostics: [{'device': -1, 'element_type': 1, 'implementation': 'portable', 'kernel': 'Abs', 'library': 'onnx_light', 'message': 'Abs selected parallel.minimum_elements=32768.', 'tuning_abi': 2}] .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 0.021 seconds) .. _sphx_glr_download_auto_examples_runtime_plot_kernel_tuning.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_kernel_tuning.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_kernel_tuning.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_kernel_tuning.zip ` .. include:: plot_kernel_tuning.recommendations .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_