Load and predict with ONNX Runtime and a very simple model

This example demonstrates how to load a model and compute the output for an input vector. It also shows how to retrieve the definition of its inputs and outputs.

import numpy

import onnxruntime as rt
from onnxruntime.datasets import get_example

Let’s load a very simple model. The model is available on github onnx…test_sigmoid.

example1 = get_example("sigmoid.onnx")
sess = rt.InferenceSession(example1, providers=rt.get_available_providers())

Let’s see the input name and shape.

input_name = sess.get_inputs()[0].name
print("input name", input_name)
input_shape = sess.get_inputs()[0].shape
print("input shape", input_shape)
input_type = sess.get_inputs()[0].type
print("input type", input_type)

input name x
input shape [3, 4, 5]
input type tensor(float)

Let’s see the output name and shape.

output_name = sess.get_outputs()[0].name
print("output name", output_name)
output_shape = sess.get_outputs()[0].shape
print("output shape", output_shape)
output_type = sess.get_outputs()[0].type
print("output type", output_type)

output name y
output shape [3, 4, 5]
output type tensor(float)

Let’s compute its outputs (or predictions if it is a machine learned model).

import numpy.random

x = numpy.random.random((3, 4, 5))
x = x.astype(numpy.float32)
res = sess.run([output_name], {input_name: x})
print(res)

[array([[[0.5121713 , 0.5650853 , 0.6802028 , 0.61690974, 0.6358885 ],
        [0.55241215, 0.6055912 , 0.52588457, 0.6217706 , 0.63186777],
        [0.6922743 , 0.55451953, 0.72307765, 0.6290897 , 0.7098243 ],
        [0.6762531 , 0.56139696, 0.6198987 , 0.68655896, 0.6428313 ]],

       [[0.7158093 , 0.5204219 , 0.66138065, 0.6042783 , 0.5939287 ],
        [0.53244114, 0.55207145, 0.66423714, 0.7054734 , 0.5450796 ],
        [0.6722973 , 0.66944003, 0.60254616, 0.6478344 , 0.6227298 ],
        [0.72135854, 0.6804575 , 0.72911936, 0.71352696, 0.73097146]],

       [[0.63974607, 0.56598544, 0.56790644, 0.6143429 , 0.67127275],
        [0.60251653, 0.6885378 , 0.53336734, 0.6208148 , 0.66633767],
        [0.63168377, 0.6576307 , 0.555976  , 0.63650405, 0.7152079 ],
        [0.56185985, 0.6577897 , 0.6504953 , 0.5192938 , 0.5771852 ]]],
      dtype=float32)]