Benchmarking and profiling Tutorial

Next sections shows how to measure performance of an ONNX graph when executing with onnxruntime.

• Study behavior of one operator
  • Benchmark operator LeakyRelu
  • Benchmark operator Slice
  • Benchmark and profile of operator Slice
• Inference
  • Benchmark onnxruntime API: run or …
  • Benchmark inference for scikit-learn models
  • Benchmark inference for a linear regression
  • Benchmark onnxruntime optimization
• Profiling
  • Profile onnxruntime execution
  • Profiling of ONNX graph with onnxruntime
• Training
  • Benchmark, comparison scikit-learn - onnxruntime-training
  • Benchmark, comparison scikit-learn - forward-backward
  • Benchmark, comparison sklearn - forward-backward - classification
  • Compares numpy to onnxruntime on simple functions

The tutorial was tested with following version:

<<<

import sys
import numpy
import scipy
import onnx
import onnxruntime
import lightgbm
import xgboost
import sklearn
import onnxconverter_common
import onnxmltools
import skl2onnx
import pyquickhelper
import mlprodict
import onnxcustom

print("python {}".format(sys.version_info))
mods = [numpy, scipy, sklearn, lightgbm, xgboost,
        onnx, onnxmltools, onnxruntime, onnxcustom,
        onnxconverter_common,
        skl2onnx, mlprodict, pyquickhelper]
mods = [(m.__name__, m.__version__) for m in mods]
mx = max(len(_[0]) for _ in mods) + 1
for name, vers in sorted(mods):
    print("{}{}{}".format(name, " " * (mx - len(name)), vers))

>>>

    python sys.version_info(major=3, minor=9, micro=1, releaselevel='final', serial=0)
    lightgbm             3.3.2
    mlprodict            0.8.1747
    numpy                1.22.3
    onnx                 1.11.0
    onnxconverter_common 1.10.0
    onnxcustom           0.4.293
    onnxmltools          1.10.0
    onnxruntime          1.11.993+cpu
    pyquickhelper        1.11.3711
    scipy                1.8.0
    skl2onnx             1.11
    sklearn              1.0.2
    xgboost              1.5.2

previous

Compares numpy to onnxruntime on simple functions

next

Study behavior of one operator