Common errors with onnxruntime

This example looks into several common situations in which onnxruntime does not return the model prediction but raises an exception instead. It starts by loading the model trained in example Step 1: Train a model using your favorite framework which produced a logistic regression trained on Iris datasets. The model takes a vector of dimension 2 and returns a class among three.

import numpy

import onnxruntime as rt
from onnxruntime.capi.onnxruntime_pybind11_state import InvalidArgument
from onnxruntime.datasets import get_example

example2 = get_example("logreg_iris.onnx")
sess = rt.InferenceSession(example2, providers=rt.get_available_providers())

input_name = sess.get_inputs()[0].name
output_name = sess.get_outputs()[0].name

The first example fails due to bad types. onnxruntime only expects single floats (4 bytes) and cannot handle any other kind of floats.

try:
    x = numpy.array([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], dtype=numpy.float64)
    sess.run([output_name], {input_name: x})
except Exception as e:
    print("Unexpected type")
    print("{0}: {1}".format(type(e), e))

Unexpected type
<class 'onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument'>: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Unexpected input data type. Actual: (tensor(double)) , expected: (tensor(float))

The model fails to return an output if the name is misspelled.

try:
    x = numpy.array([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], dtype=numpy.float32)
    sess.run(["misspelled"], {input_name: x})
except Exception as e:
    print("Misspelled output name")
    print("{0}: {1}".format(type(e), e))

Misspelled output name
<class 'onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument'>: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid Output Name:misspelled

The output name is optional, it can be replaced by None and onnxruntime will then return all the outputs.

x = numpy.array([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], dtype=numpy.float32)
try:
    res = sess.run(None, {input_name: x})
    print("All outputs")
    print(res)
except (RuntimeError, InvalidArgument) as e:
    print(e)

All outputs
[array([0, 0, 0], dtype=int64), [{0: 0.9505997896194458, 1: 0.027834143489599228, 2: 0.021566055715084076}, {0: 0.9974970817565918, 1: 5.6270167988259345e-05, 2: 0.0024466365575790405}, {0: 0.9997311234474182, 1: 1.787709464906584e-07, 2: 0.0002686927327886224}]]

The same goes if the input name is misspelled.

try:
    x = numpy.array([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], dtype=numpy.float32)
    sess.run([output_name], {"misspelled": x})
except Exception as e:
    print("Misspelled input name")
    print("{0}: {1}".format(type(e), e))

Misspelled input name
<class 'onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument'>: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid Feed Input Name:misspelled

onnxruntime does not necessarily fail if the input dimension is a multiple of the expected input dimension.

for x in [
    numpy.array([1.0, 2.0, 3.0, 4.0], dtype=numpy.float32),
    numpy.array([[1.0, 2.0, 3.0, 4.0]], dtype=numpy.float32),
    numpy.array([[1.0, 2.0], [3.0, 4.0]], dtype=numpy.float32),
    numpy.array([1.0, 2.0, 3.0], dtype=numpy.float32),
    numpy.array([[1.0, 2.0, 3.0]], dtype=numpy.float32),
]:
    try:
        r = sess.run([output_name], {input_name: x})
        print("Shape={0} and predicted labels={1}".format(x.shape, r))
    except (RuntimeError, InvalidArgument) as e:
        print("ERROR with Shape={0} - {1}".format(x.shape, e))

for x in [
    numpy.array([1.0, 2.0, 3.0, 4.0], dtype=numpy.float32),
    numpy.array([[1.0, 2.0, 3.0, 4.0]], dtype=numpy.float32),
    numpy.array([[1.0, 2.0], [3.0, 4.0]], dtype=numpy.float32),
    numpy.array([1.0, 2.0, 3.0], dtype=numpy.float32),
    numpy.array([[1.0, 2.0, 3.0]], dtype=numpy.float32),
]:
    try:
        r = sess.run(None, {input_name: x})
        print("Shape={0} and predicted probabilities={1}".format(x.shape, r[1]))
    except (RuntimeError, InvalidArgument) as e:
        print("ERROR with Shape={0} - {1}".format(x.shape, e))

ERROR with Shape=(4,) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid rank for input: float_input Got: 1 Expected: 2 Please fix either the inputs or the model.
ERROR with Shape=(1, 4) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: float_input for the following indices
 index: 0 Got: 1 Expected: 3
 index: 1 Got: 4 Expected: 2
 Please fix either the inputs or the model.
ERROR with Shape=(2, 2) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: float_input for the following indices
 index: 0 Got: 2 Expected: 3
 Please fix either the inputs or the model.
ERROR with Shape=(3,) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid rank for input: float_input Got: 1 Expected: 2 Please fix either the inputs or the model.
ERROR with Shape=(1, 3) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: float_input for the following indices
 index: 0 Got: 1 Expected: 3
 index: 1 Got: 3 Expected: 2
 Please fix either the inputs or the model.
ERROR with Shape=(4,) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid rank for input: float_input Got: 1 Expected: 2 Please fix either the inputs or the model.
ERROR with Shape=(1, 4) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: float_input for the following indices
 index: 0 Got: 1 Expected: 3
 index: 1 Got: 4 Expected: 2
 Please fix either the inputs or the model.
ERROR with Shape=(2, 2) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: float_input for the following indices
 index: 0 Got: 2 Expected: 3
 Please fix either the inputs or the model.
ERROR with Shape=(3,) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid rank for input: float_input Got: 1 Expected: 2 Please fix either the inputs or the model.
ERROR with Shape=(1, 3) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: float_input for the following indices
 index: 0 Got: 1 Expected: 3
 index: 1 Got: 3 Expected: 2
 Please fix either the inputs or the model.

It does not fail either if the number of dimension is higher than expects but produces a warning.

for x in [
    numpy.array([[[1.0, 2.0], [3.0, 4.0]]], dtype=numpy.float32),
    numpy.array([[[1.0, 2.0, 3.0]]], dtype=numpy.float32),
    numpy.array([[[1.0, 2.0]], [[3.0, 4.0]]], dtype=numpy.float32),
]:
    try:
        r = sess.run([output_name], {input_name: x})
        print("Shape={0} and predicted labels={1}".format(x.shape, r))
    except (RuntimeError, InvalidArgument) as e:
        print("ERROR with Shape={0} - {1}".format(x.shape, e))

ERROR with Shape=(1, 2, 2) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid rank for input: float_input Got: 3 Expected: 2 Please fix either the inputs or the model.
ERROR with Shape=(1, 1, 3) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid rank for input: float_input Got: 3 Expected: 2 Please fix either the inputs or the model.
ERROR with Shape=(2, 1, 2) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid rank for input: float_input Got: 3 Expected: 2 Please fix either the inputs or the model.