optim_tensor.h#

Lightweight, non-owning tensor description used by ONNX graph optimisation passes.

The onnx_optim namespace exposes three small value types:

:cpp:class:OptimDim — a single shape dimension, either a concrete int64_t or a symbolic string expression.
:cpp:class:OptimShape — an ordered, bounded-rank collection of :cpp:class:OptimDim.
:cpp:class:OptimTensor — a non-owning view over a contiguous buffer carrying a :cpp:type:TensorType, an :cpp:class:OptimShape, and an optional shape annotation when the tensor itself represents a shape (e.g. the shape input of Reshape).

These types are intentionally header-only friendly and never allocate the tensor data they describe; callers are responsible for the lifetime of the underlying buffer.

namespace onnx_light

Alias that makes onnx-light headers compatible with code that references ONNX_LIGHT_NAMESPACE (the macro used in the standard onnx package).

Set to ONNX_LIGHT_NAMESPACE so both names resolve to the same namespace.

Symbol-visibility attribute for the public onnx-light C++ API.

Defined as empty because onnx-light does not require explicit __declspec(dllexport) or __attribute__((visibility("default"))) annotations — visibility is controlled at the shared-library level. The macro is provided so that vendored ONNX headers that decorate their declarations with ONNX_API compile without modification.

Namespace alias so that ONNX C++ code (and consumers such as onnxruntime) that refers to the literal onnx namespace — rather than the ONNX_NAMESPACE macro — resolves to the onnx-light namespace. The standard onnx package lives in namespace onnx; onnx-light uses onnx_light (via ONNX_LIGHT_NAMESPACE), so this alias keeps onnx-light a true drop-in. It is only introduced when the onnx-light namespace differs from onnx.

namespace onnx_optim

Typedefs

using TensorType = onnx_light::onnx_op::TensorType #: Reuse the TensorType enumeration defined by onnx_op so that onnx_optim is fully aligned with the rest of the operator stack.

Enums

enum class OptimCmpResult#

Result of comparing two :cpp:class:OptimTensor descriptors with :cpp:func:OptimTensor::Cmp. The four outcomes describe the relative precision of the two descriptors when they are interpreted as statements about the same logical tensor.

:cpp:enumerator:OptimCmpResult::kConflict — the two descriptors carry contradictory information (e.g. different known element types, different ranks, or incompatible concrete dimension values) and cannot both be true at the same time.
:cpp:enumerator:OptimCmpResult::kMorePrecise — *this is at least as precise as other on every field and strictly more precise on at least one (or both are equivalent — the equal case is reported as kMorePrecise).
:cpp:enumerator:OptimCmpResult::kLessPrecise — other is strictly more precise than *this.
:cpp:enumerator:OptimCmpResult::kComplementary — neither descriptor dominates: each one carries some information the other one is missing, but the two are mutually compatible.

Values:

enumerator kConflict#

enumerator kMorePrecise#

enumerator kLessPrecise#

enumerator kComplementary#

enum class Device : int32_t#

Logical device on which an :cpp:class:OptimTensor resides.

The enumeration is intentionally compact: kUndefined denotes the “no information” state (the default), kCPU denotes the host CPU, and the contiguous range [kGPU0, kGPU<kMaxGPUIndex>] enumerates up to kMaxGPUIndex + 1 distinct GPU devices. The numeric value of kGPU<i> is static_cast<int32_t>(kGPU0) + i; the helpers :cpp:func:MakeGPUDevice, :cpp:func:IsGPU and :cpp:func:GPUIndex should be preferred over direct casts.

Values:

enumerator kUndefined#

enumerator kCPU#

enumerator kGPU0#: First GPU device. kGPU<i> corresponds to static_cast<Device>(static_cast<int32_t>(kGPU0) + i) for 0 <= i <= kMaxGPUIndex.

enumerator kGPU8191#: Last addressable GPU device (index kMaxGPUIndex).

Functions

TensorType DataTypeToTensorType(TensorProto::DataType dtype)#: Maps a TensorProto::DataType to the matching :cpp:type:TensorType enumerator used by :cpp:class:OptimTensor. Returns :cpp:enumerator:TensorType::kUndefined for any data type that is not representable in the onnx_optim stack (e.g. UNDEFINED).

TensorProto::DataType TensorTypeToDataType(TensorType t)#

Maps a :cpp:type:TensorType enumerator back to the matching TensorProto::DataType. This is the inverse of :cpp:func:DataTypeToTensorType and is used to write inferred element types into ValueInfoProto / TypeProto::Tensor.

Only the scalar tensor types are supported (the sequence- and optional-typed enumerators do not have a single matching scalar DataType). For sequence/optional types or for :cpp:enumerator:TensorType::kUndefined, the function returns TensorProto::DataType::UNDEFINED.

bool IsIntegerTensorType(TensorType t)#

Returns whether a tensor element type can be interpreted as shape dimensions for ValueAsShape.

Accepted integer types are signed/unsigned 8, 16, 32 and 64-bit integers.

Parameters:: t – Tensor element type to evaluate.
Returns:: true if t is one of the supported integer types.

OptimShape ShapeFromTensorProtoDims(const TensorProto &tensor_proto)#

Builds an :cpp:class:OptimShape from tensor_proto.dims().

ONNX stores tensor dimensions as non-negative 64-bit integers in TensorProto::dims; this helper converts every dimension into an :cpp:class:OptimDim and appends them in order.

Parameters:: tensor_proto – Tensor whose dims field is converted.
Returns:: Converted shape with the same rank and dimension values.

Device MakeGPUDevice(int index)#

Returns the :cpp:enumerator:Device::kGPU0 + index enumerator.

Parameters:: index – GPU index in the inclusive range [0, kMaxGPUIndex].
Throws:: std::out_of_range – if index is outside the supported range.

constexpr bool IsGPU(Device d) noexcept#: Returns true when d is one of the kGPU0..kGPU8191 enumerators.

constexpr int GPUIndex(Device d) noexcept#: Returns the GPU index of d (0 for kGPU0, kMaxGPUIndex for kGPU8191) or -1 when d is not a GPU device.

std::string DeviceName(Device d)#: Returns a human-readable name for d: "Undefined", "CPU", "GPU<i>" for GPU devices, or "Unknown" for any out-of-range value.

Device DeviceFromName(const std::string &name)#

Parses a device name back into a :cpp:enum:Device enumerator.

Recognises the exact strings produced by :cpp:func:DeviceName: "Undefined", "CPU" and "GPU<i>" where i is a decimal integer in [0, kMaxGPUIndex]. Any other input — including the empty string, "Unknown", a different case, a leading sign or whitespace, or an out-of-range GPU index — yields :cpp:enumerator:Device::kUndefined.

bool OptimTensorFromValueInfo(const ValueInfoProto &vi, OptimTensor &out)#

Populates out from a ValueInfoProto describing a tensor.

The element type and (optional) shape are read from vi.type().tensor_type(). When vi.metadata_props() contains an entry whose key matches :cpp:var:kValueInfoDeviceMetadataKey, its value is parsed with :cpp:func:DeviceFromName and assigned to the resulting tensor; otherwise the device is left as :cpp:enumerator:Device::kUndefined.

Parameters:

vi – ValueInfoProto to read from.
out – Tensor to overwrite on success.

Returns:

true when vi wraps a tensor type; false for sequence/map/optional/sparse types (which OptimTensor does not model), in which case out is left untouched.

bool OptimTensorFromTensorProto(const TensorProto &tp, OptimTensor &out)#

Populates out from a TensorProto (typically a graph initializer).

The element type is read from tp.data_type() and the shape is built from tp.dims() via :cpp:func:ShapeFromTensorProtoDims (every dimension becomes a concrete integer :cpp:class:OptimDim). The data pointer is left null and the device is left :cpp:enumerator:Device::kUndefined.

When the tensor payload is readable, the function also annotates the resulting descriptor with:

the min/max value bounds derived from the actual content. Both integer (INT8/16/32/64, UINT8/16/32/64) and floating-point (FLOAT, DOUBLE) element types are supported. The bounds are skipped for empty tensors and for tensors whose payload cannot be decoded (no typed field and no raw_data, unsupported dtype, etc.);
the :cpp:func:OptimTensor::ValueAsShape annotation when the element type is integer, the rank is at most one, and the element count is strictly less than :cpp:var:kOptimValueAsShapeMaxElements (kMaxOptimRank + 1, i.e., at most kMaxOptimRank elements). This covers every 1-D integer constant whose values fit inside an :cpp:class:OptimShape, enabling Reshape, Unsqueeze, and similar operators to infer fully-concrete output shapes from large axes or target-shape initializers.

Parameters:

tp – TensorProto to read from.
out – Tensor to overwrite on success.

Returns:

false (and leaves out untouched) when tp.data_type() is TensorProto::DataType::UNDEFINED; true otherwise.

bool OptimTensorToValueInfo(const OptimTensor &tensor, ValueInfoProto &vi)#

Writes the (dtype, shape, device) triple carried by tensor into vi.

Any pre-existing type on vi is overwritten so that the inferred descriptor takes precedence. The device is encoded as a metadata_props entry keyed by :cpp:var:kValueInfoDeviceMetadataKey; if such an entry already exists it is updated in place, and if the tensor’s device is :cpp:enumerator:Device::kUndefined the existing entry (when any) is removed. The name and doc_string fields of vi are never touched.

Parameters:

tensor – Source tensor.
vi – Destination ValueInfoProto.

Returns:

false (and leaves vi unchanged) when tensor has an undefined element type, since TensorProto::DataType provides no meaningful encoding for it; true otherwise.

Variables

constexpr std::size_t kMaxOptimRank = 16#: Maximum number of dimensions an OptimShape can describe inline. Shapes in practice rarely exceed this rank, so storing the dimensions in a small container keeps the structure compact and cache-friendly.

constexpr int kMaxGPUIndex = 8191#: Maximum GPU index supported by :cpp:enum:Device. The enumeration exposes one enumerator per GPU from kGPU0 up to kGPU<kMaxGPUIndex> (inclusive).

constexpr const char *kValueInfoMinMetadataKey = "min"#: Well-known key used to round-trip the :cpp:func:OptimTensor::Min bound through the ValueInfoProto::metadata_props field.

constexpr const char *kValueInfoMaxMetadataKey = "max"#: Well-known key used to round-trip the :cpp:func:OptimTensor::Max bound through the ValueInfoProto::metadata_props field.

constexpr const char *kValueInfoDeviceMetadataKey = "device"#: Well-known key used to round-trip :cpp:enum:Device through the ValueInfoProto::metadata_props field. Exposed so that callers outside onnx_optim can inspect or remove the entry.

constexpr int64_t kOptimValueAsShapeMaxElements = static_cast<int64_t>(kMaxOptimRank) + 1#

Maximum element count of a small integer tensor for which :cpp:func:OptimTensorFromTensorProto (and friends) populate the :cpp:func:OptimTensor::ValueAsShape annotation. Tensors beyond this threshold are not data-propagated (the dtype, shape and min/max bounds are still recorded normally).

The limit is set to kMaxOptimRank + 1 so that any 1-D integer constant whose length fits within an :cpp:class:OptimShape (i.e., up to kMaxOptimRank elements) can have its values propagated. This enables operators like Unsqueeze to infer fully-concrete output shapes even when the axes input is a large initializer (e.g., rank 15 or 16 models).

class OptimDim#

#include <optim_tensor.h>

A single shape dimension that is either a concrete non-negative integer or a symbolic expression represented as a string. OptimDim is used by OptimShape to describe both fully-known and partially-symbolic shapes.

Public Functions

inline OptimDim()#: Default constructs a zero-valued integer dimension.

inline OptimDim(int64_t value)#: Constructs an integer-valued dimension.

inline OptimDim(std::string expr)#: Constructs a symbolic dimension expressed as a string expression.

inline OptimDim(const char *expr)#: Convenience overload for C string literals.

inline bool IsInt() const noexcept#: Returns true when the dimension holds a concrete integer value.

inline bool IsExpr() const noexcept#: Returns true when the dimension holds a symbolic expression string.

inline int64_t AsInt() const#: Returns the integer value. Throws std::bad_variant_access if the dimension is not an integer.

inline const std::string &AsExpr() const#: Returns the symbolic expression. Throws std::bad_variant_access if the dimension is not a string expression.

inline bool operator==(const OptimDim &other) const noexcept#: Equality compares both the alternative and the underlying value.

inline bool operator!=(const OptimDim &other) const noexcept#

std::string ToString() const#: Returns a human-readable representation of the dimension: the integer value for concrete dimensions, or the symbolic expression for string dimensions.

Private Members

std::variant<int64_t, std::string> value_#

class OptimShape#

#include <optim_tensor.h>

A short, value-typed shape composed of OptimDim entries. The rank is bounded by kMaxOptimRank so that the structure fits comfortably on the stack. Symbolic and concrete dimensions can be mixed freely.

Public Types

using value_type = OptimDim #

Public Functions

OptimShape() = default#

OptimShape(std::initializer_list<OptimDim> dims)#: Constructs a shape from an initializer list of dimensions.

explicit OptimShape(const std::vector<OptimDim> &dims)#: Constructs a shape from any iterable container of OptimDim.

inline std::size_t Rank() const noexcept#: Number of dimensions.

inline bool Empty() const noexcept#: true when the shape contains no dimensions (scalar / rank-0).

inline const OptimDim &operator[](std::size_t i) const#: Access a dimension by index. Throws std::out_of_range if invalid.

inline OptimDim &operator[](std::size_t i)#

void PushBack(OptimDim dim)#: Appends a new dimension. Throws std::length_error when the maximum rank kMaxOptimRank would be exceeded.

bool IsFullyKnown() const noexcept#: Returns true when every dimension is a concrete integer.

int64_t NumElements() const#: Computes the product of all integer dimensions. Returns 1 for a rank-0 (empty) shape, matching the standard scalar-element-count semantic. Throws std::runtime_error if any dimension is symbolic.

inline bool operator==(const OptimShape &other) const noexcept#: Equality compares the dimensions element-wise.

inline bool operator!=(const OptimShape &other) const noexcept#

inline const std::vector<OptimDim> &Dims() const noexcept#: Read-only access to the underlying dimensions.

std::string ToString() const#: Returns a human-readable representation of the shape as a comma-separated list of dimensions enclosed in square brackets, e.g. "[2,3,N]". A rank-0 shape returns "[]".

Private Members

std::vector<OptimDim> dims_#

class OptimTensor#

#include <optim_tensor.h>

A non-owning view over a contiguous tensor buffer. OptimTensor never allocates: the caller is responsible for the lifetime of the underlying memory referenced by data. The shape may contain symbolic dimensions which is useful for representing intermediate values during optimisation passes where the concrete shape is not yet known.

Public Functions

OptimTensor() = default#: Constructs an empty (null) tensor.

inline OptimTensor(void *data, TensorType dtype, OptimShape shape)#

Constructs an OptimTensor referencing an external buffer.

Parameters:

data – Pointer to the first element of the externally-owned buffer. May be nullptr only when shape is empty or all integer dimensions are zero.
dtype – Element type of the data referenced by data.
shape – Shape describing the layout of the data.

inline void *Data() const noexcept#: Pointer to the externally-owned buffer. The view itself is const but the buffer it references is not, mirroring std::span semantics.

inline TensorType Dtype() const noexcept#: Element type of the referenced buffer.

inline Device GetDevice() const noexcept#: Logical device on which the buffer resides. Defaults to :cpp:enumerator:Device::kUndefined (“no information”).

inline void SetDevice(Device device) noexcept#: Sets the logical device on which the buffer resides.

inline bool HasMin() const noexcept#: true when a lower bound on the tensor’s values is known.

inline bool HasMax() const noexcept#: true when an upper bound on the tensor’s values is known.

inline double Min() const#: Lower bound on the tensor’s values. Throws std::bad_optional_access when :cpp:func:HasMin is false.

inline double Max() const#: Upper bound on the tensor’s values. Throws std::bad_optional_access when :cpp:func:HasMax is false.

inline void SetMin(double value) noexcept#: Sets the lower bound on the tensor’s values.

inline void SetMax(double value) noexcept#: Sets the upper bound on the tensor’s values.

void SetMinMax(double min, double max)#: Sets both bounds in a single call. min must be <= max; otherwise std::invalid_argument is thrown.

inline void ClearMin() noexcept#: Clears the stored lower bound.

inline void ClearMax() noexcept#: Clears the stored upper bound.

inline void ClearMinMax() noexcept#: Clears both bounds.

inline bool IsNullConstant() const noexcept#: Returns true when the tensor is known to be a “null constant”, i.e. when both bounds are known and equal to zero. This is the canonical condition exploited by optimisation passes that need to detect all-zero tensors without inspecting the buffer.

inline const OptimShape &Shape() const noexcept#: Shape of the tensor (may contain symbolic dimensions).

inline OptimShape &Shape() noexcept#

inline bool IsNull() const noexcept#: true when the tensor has no associated data pointer.

inline void SetValueAsShape(OptimShape shape)#: Tags the tensor as carrying a shape value (e.g. the shape input of a Reshape node) and stores that shape. An empty OptimShape is permitted: it denotes a rank-0 / scalar shape value.

inline void ClearValueAsShape() noexcept#: Clears the value-as-shape annotation so that HasValueAsShape becomes false again.

inline bool HasValueAsShape() const noexcept#: true when the tensor’s value is interpreted as a shape. An empty stored shape still returns true — use ValueAsShape().Empty() to distinguish the empty case.

inline const OptimShape &ValueAsShape() const#: Returns the shape value carried by this tensor. Throws std::bad_optional_access if HasValueAsShape() is false.

inline OptimShape &ValueAsShape()#

inline bool operator==(const OptimTensor &other) const noexcept#: Equality compares the data pointer, dtype, device, shape, the optional value-as-shape annotation, and the optional min/max value bounds. Because :cpp:class:OptimTensor is a non-owning view, two tensors are considered equal only when they refer to the same external buffer.

inline bool operator!=(const OptimTensor &other) const noexcept#

std::string ToString() const#

Returns a human-readable representation of the tensor of the form

"OptimTensor(dtype=<name>, shape=<shape>[, device=<name>][, value_as_shape=<shape>][,


min=<v>][, max=<v>][, data=<ptr>])"

. The device component is omitted when the device is :cpp:enumerator:Device::kUndefined. The data component is omitted when the tensor holds no buffer. The value_as_shape component is omitted when no shape annotation is attached. The min and max components are each omitted when the corresponding bound is not set. The <name> is the unqualified TensorType enumerator name (e.g. "Float", "Int64", "Undefined").

OptimCmpResult Cmp(const OptimTensor &other) const noexcept#

Compares the information carried by *this and other and reports which descriptor is more precise (see :cpp:enum:OptimCmpResult).

The comparison covers, in order:

the element type (an unknown :cpp:enumerator:TensorType::kUndefined is treated as “no information”; two different known types yield :cpp:enumerator:OptimCmpResult::kConflict);
the device (an unknown :cpp:enumerator:Device::kUndefined is treated as “no information”; two different known devices yield :cpp:enumerator:OptimCmpResult::kConflict);
the shape rank (different ranks yield kConflict);
each dimension (two different concrete integers or two different symbolic expressions yield kConflict; an integer is considered more precise than a symbolic expression at the same position);
the optional :cpp:func:ValueAsShape annotation (handled with the same rules as the main shape; a present annotation is more precise than an absent one);
the optional min and max bounds: a known bound is more precise than an absent one; two known bounds that produce a tighter interval (higher min or lower max) are more precise; intervals that are provably disjoint (a.min > b.max or b.min > a.max) yield kConflict;
the data-pointer presence (a non-null pointer is more precise than a null one; two distinct non-null pointers carry no precision signal because :cpp:class:OptimTensor is a non-owning view and the buffer contents are not inspected).

When *this and other are equivalent on every field, the result is :cpp:enumerator:OptimCmpResult::kMorePrecise (*this is at least as precise as other).

Private Members

void *data_ = nullptr#

TensorType dtype_ = TensorType::kUndefined#

Device device_ = Device::kUndefined #

OptimShape shape_ = {}#

std::optional<OptimShape> value_as_shape_ = {}#

std::optional<double> min_ = {}#

std::optional<double> max_ = {}#