Training

Differences between onnxruntime and onnxruntime-training

onnxruntime-training is an extension of onnxruntime that supports training. Version 1.10 is obtained by compiling onnxruntime from the sources with different flags. One example:

python ./tools/ci_build/build.py --build_dir ./build/debian \\
       --config Release --build_wheel --numpy_version= \\
       --skip_tests --build_shared_lib --enable_training \\
       --enable_training_ops --enable_training_torch_interop \\
       --parallel

Python Wrapper for TrainingSession

class onnxruntime.TrainingSession(path_or_bytes, parameters, sess_options=None, providers=None, provider_options=None)

Parameters

• path_or_bytes – filename or serialized ONNX or ORT format model in a byte string

• sess_options – session options

• providers – Optional sequence of providers in order of decreasing precedence. Values can either be provider names or tuples of (provider name, options dict). If not provided, then all available providers are used with the default precedence.

• provider_options – Optional sequence of options dicts corresponding to the providers listed in ‘providers’.

The model type will be inferred unless explicitly set in the SessionOptions. To explicitly set:

so = onnxruntime.SessionOptions()
# so.add_session_config_entry('session.load_model_format', 'ONNX') or
so.add_session_config_entry('session.load_model_format', 'ORT')

A file extension of ‘.ort’ will be inferred as an ORT format model. All other filenames are assumed to be ONNX format models.

‘providers’ can contain either names or names and options. When any options are given in ‘providers’, ‘provider_options’ should not be used.

The list of providers is ordered by precedence. For example [‘CUDAExecutionProvider’, ‘CPUExecutionProvider’] means execute a node using CUDAExecutionProvider if capable, otherwise execute using CPUExecutionProvider.

__class__

alias of type

__del__()

__delattr__(name, /)

Implement delattr(self, name).

__dir__()

Default dir() implementation.

__eq__(value, /)

Return self==value.

__format__(format_spec, /)

Default object formatter.

__ge__(value, /)

Return self>=value.

__getattribute__(name, /)

Return getattr(self, name).

__gt__(value, /)

Return self>value.

__hash__()

Return hash(self).

__init__(path_or_bytes, parameters, sess_options=None, providers=None, provider_options=None)

Parameters

• path_or_bytes – filename or serialized ONNX or ORT format model in a byte string

• sess_options – session options

• providers – Optional sequence of providers in order of decreasing precedence. Values can either be provider names or tuples of (provider name, options dict). If not provided, then all available providers are used with the default precedence.

• provider_options – Optional sequence of options dicts corresponding to the providers listed in ‘providers’.

The model type will be inferred unless explicitly set in the SessionOptions. To explicitly set:

so = onnxruntime.SessionOptions()
# so.add_session_config_entry('session.load_model_format', 'ONNX') or
so.add_session_config_entry('session.load_model_format', 'ORT')

A file extension of ‘.ort’ will be inferred as an ORT format model. All other filenames are assumed to be ONNX format models.

‘providers’ can contain either names or names and options. When any options are given in ‘providers’, ‘provider_options’ should not be used.

The list of providers is ordered by precedence. For example [‘CUDAExecutionProvider’, ‘CPUExecutionProvider’] means execute a node using CUDAExecutionProvider if capable, otherwise execute using CPUExecutionProvider.

__init_subclass__()

This method is called when a class is subclassed.

The default implementation does nothing. It may be overridden to extend subclasses.

__le__(value, /)

Return self<=value.

__lt__(value, /)

Return self<value.

__ne__(value, /)

Return self!=value.

__new__(**kwargs)

__reduce__()

Helper for pickle.

__reduce_ex__(protocol, /)

Helper for pickle.

__repr__()

Return repr(self).

__setattr__(name, value, /)

Implement setattr(self, name, value).

__sizeof__()

Size of object in memory, in bytes.

__str__()

Return str(self).

__subclasshook__()

Abstract classes can override this to customize issubclass().

This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).

_create_inference_session(providers, provider_options, disabled_optimizers=None)

_reset_session(providers, provider_options)

release underlying session object.

disable_fallback()

Disable session.run() fallback mechanism.

enable_fallback()

Enable session.Run() fallback mechanism. If session.Run() fails due to an internal Execution Provider failure, reset the Execution Providers enabled for this session. If GPU is enabled, fall back to CUDAExecutionProvider. otherwise fall back to CPUExecutionProvider.

end_profiling()

End profiling and return results in a file.

The results are stored in a filename if the option onnxruntime.SessionOptions.enable_profiling().

get_inputs()

Return the inputs metadata as a list of onnxruntime.NodeArg.

get_model_state(include_mixed_precision_weights=False)

get_modelmeta()

Return the metadata. See onnxruntime.ModelMetadata.

get_optimizer_state()

get_outputs()

Return the outputs metadata as a list of onnxruntime.NodeArg.

get_overridable_initializers()

Return the inputs (including initializers) metadata as a list of onnxruntime.NodeArg.

get_partition_info_map()

get_profiling_start_time_ns()

Return the nanoseconds of profiling’s start time Comparable to time.monotonic_ns() after Python 3.3 On some platforms, this timer may not be as precise as nanoseconds For instance, on Windows and MacOS, the precision will be ~100ns

get_provider_options()

Return registered execution providers’ configurations.

get_providers()

Return list of registered execution providers.

get_session_options()

Return the session options. See onnxruntime.SessionOptions.

get_state()

io_binding()

Return an onnxruntime.IOBinding object`.

is_output_fp32_node(output_name)

load_state(dict, strict=False)

run(output_names, input_feed, run_options=None)

Compute the predictions.

Parameters

• output_names – name of the outputs

• input_feed – dictionary { input_name: input_value }

• run_options – See onnxruntime.RunOptions.

sess.run([output_name], {input_name: x})

run_with_iobinding(iobinding, run_options=None)

Compute the predictions.

Parameters

• iobinding – the iobinding object that has graph inputs/outputs bind.

• run_options – See onnxruntime.RunOptions.

run_with_ort_values(output_names, input_dict_ort_values, run_options=None)

Compute the predictions.

Parameters

• output_names – name of the outputs

• input_dict_ort_values – dictionary { input_name: input_ort_value } See OrtValue class how to create OrtValue from numpy array or SparseTensor

• run_options – See onnxruntime.RunOptions.

Returns

an array of OrtValue

sess.run([output_name], {input_name: x})

set_providers(providers=None, provider_options=None)

Register the input list of execution providers. The underlying session is re-created.

Parameters

• providers – Optional sequence of providers in order of decreasing precedence. Values can either be provider names or tuples of (provider name, options dict). If not provided, then all available providers are used with the default precedence.

• provider_options – Optional sequence of options dicts corresponding to the providers listed in ‘providers’.

‘providers’ can contain either names or names and options. When any options are given in ‘providers’, ‘provider_options’ should not be used.

The list of providers is ordered by precedence. For example [‘CUDAExecutionProvider’, ‘CPUExecutionProvider’] means execute a node using CUDAExecutionProvider if capable, otherwise execute using CPUExecutionProvider.

C Class TrainingSession

class onnxruntime.capi._pybind_state.TrainingSession(*args, **kwargs)

Overloaded function.

1. __init__(self: onnxruntime.capi.onnxruntime_pybind11_state.TrainingSession, arg0: onnxruntime.capi.onnxruntime_pybind11_state.SessionOptions) -> None

2. __init__(self: onnxruntime.capi.onnxruntime_pybind11_state.TrainingSession) -> None

__init__(*args, **kwargs)

Overloaded function.

1. __init__(self: onnxruntime.capi.onnxruntime_pybind11_state.TrainingSession, arg0: onnxruntime.capi.onnxruntime_pybind11_state.SessionOptions) -> None

2. __init__(self: onnxruntime.capi.onnxruntime_pybind11_state.TrainingSession) -> None

finalize(self: object) → None

get_model_state(self: onnxruntime.capi.onnxruntime_pybind11_state.TrainingSession, arg0: bool) → Dict[str, Dict[str, object]]

get_optimizer_state(self: onnxruntime.capi.onnxruntime_pybind11_state.TrainingSession) → Dict[str, Dict[str, object]]

get_partition_info_map(self: onnxruntime.capi.onnxruntime_pybind11_state.TrainingSession) → Dict[str, Dict[str, List[int]]]

get_state(self: onnxruntime.capi.onnxruntime_pybind11_state.TrainingSession) → Dict[str, object]

is_output_fp32_node(self: onnxruntime.capi.onnxruntime_pybind11_state.TrainingSession, arg0: str) → bool

load_model(self: onnxruntime.capi.onnxruntime_pybind11_state.TrainingSession, arg0: str, arg1: onnxruntime.capi.onnxruntime_pybind11_state.TrainingParameters, arg2: List[str], arg3: List[Dict[str, str]]) → onnxruntime.capi.onnxruntime_pybind11_state.TrainingConfigurationResult

load_state(self: onnxruntime.capi.onnxruntime_pybind11_state.TrainingSession, arg0: Dict[str, object], arg1: bool) → None

read_bytes(self: onnxruntime.capi.onnxruntime_pybind11_state.TrainingSession, arg0: bytes, arg1: onnxruntime.capi.onnxruntime_pybind11_state.TrainingParameters, arg2: List[str], arg3: List[Dict[str, str]]) → onnxruntime.capi.onnxruntime_pybind11_state.TrainingConfigurationResult

TrainingParameters

class onnxruntime.capi._pybind_state.TrainingParameters(self: onnxruntime.capi.onnxruntime_pybind11_state.TrainingParameters) → None

Configuration information for training.

__init__(self: onnxruntime.capi.onnxruntime_pybind11_state.TrainingParameters) → None

property allow_layer_norm_mod_precision

property allreduce_post_accumulation

property attn_dropout_recompute

property data_parallel_size

property deepspeed_zero_stage

property enable_adasum

property enable_grad_norm_clip

property gelu_recompute

property gradient_accumulation_steps

property horizontal_parallel_size

property immutable_weights

property loss_output_name

property loss_scale

property lr_params_feed_name

property model_after_graph_transforms_path

property model_with_gradient_graph_path

property model_with_training_graph_path

property num_pipeline_micro_batches

property number_recompute_layers

property optimizer_attributes_map

property optimizer_int_attributes_map

property pipeline_cut_info_string

property pipeline_parallel_size

property propagate_cast_ops_allow

property propagate_cast_ops_level

property set_gradients_as_graph_outputs

set_optimizer_initial_state(self: onnxruntime.capi.onnxruntime_pybind11_state.TrainingParameters, arg0: Dict[str, Dict[str, object]]) → None

property sliced_axes

property sliced_schema

property sliced_tensor_names

property training_optimizer_name

property transformer_layer_recompute

property use_fp16_moments

property use_memory_efficient_gradient

property use_mixed_precision

property weights_not_to_train

property weights_to_train

property world_rank

property world_size

GradientGraphBuilder

class onnxruntime.capi._pybind_state.GradientGraphBuilder(self: onnxruntime.capi.onnxruntime_pybind11_state.GradientGraphBuilder, arg0: bytes, arg1: Set[str], arg2: Set[str], arg3: str) → None

A utility for making a gradient graph that can be used to help train a model.

__init__(self: onnxruntime.capi.onnxruntime_pybind11_state.GradientGraphBuilder, arg0: bytes, arg1: Set[str], arg2: Set[str], arg3: str) → None

build(self: onnxruntime.capi.onnxruntime_pybind11_state.GradientGraphBuilder) → None

save(self: onnxruntime.capi.onnxruntime_pybind11_state.GradientGraphBuilder, arg0: str) → None

previous

Inference

next

Partial Training