include_quantization_kernels.h

namespace onnx_light

Alias that makes onnx-light headers compatible with code that references ONNX_LIGHT_NAMESPACE (the macro used in the standard onnx package).

Set to ONNX_LIGHT_NAMESPACE so both names resolve to the same namespace.

Symbol-visibility attribute for the public onnx-light C++ API.

Defined as empty because onnx-light does not require explicit __declspec(dllexport) or __attribute__((visibility("default"))) annotations — visibility is controlled at the shared-library level. The macro is provided so that vendored ONNX headers that decorate their declarations with ONNX_API compile without modification.

Namespace alias so that ONNX C++ code (and consumers such as onnxruntime) that refers to the literal onnx namespace — rather than the ONNX_NAMESPACE macro — resolves to the onnx-light namespace. The standard onnx package lives in namespace onnx; onnx-light uses onnx_light (via ONNX_LIGHT_NAMESPACE), so this alias keeps onnx-light a true drop-in. It is only introduced when the onnx-light namespace differs from onnx.

namespace onnx_kernels

namespace kernel

class DequantizeLinear : public onnx_light::onnx_kernels::kernel::KernelBase

#include <include_quantization_kernels.h>

Per-tensor or per-axis linear dequantization of an integer or float8 input x to a FLOAT output y using y = (x - x_zero_point) * x_scale. When x_zero_point is omitted the zero point defaults to 0. When x_scale has more than one element, per-axis dequantization is performed along axis (only FLOAT x_scale is supported for per-axis).

Public Functions

Tensor operator()(const Tensor &x, const Tensor &x_scale) const

Omitted x_zero_point: zero point defaults to 0.

void operator()(const Tensor &x, const Tensor &x_scale, Tensor &output) const

Tensor operator()(const Tensor &x, const Tensor &x_scale, const Tensor &x_zero_point) const

Explicit x_zero_point: must have the same element type as x.

void operator()(const Tensor &x, const Tensor &x_scale, const Tensor &x_zero_point, Tensor &output) const

Tensor operator()(const Tensor &x, const Tensor &x_scale, const Tensor &x_zero_point, int64_t axis) const

Per-axis overloads: x_scale and x_zero_point have one entry per slice along axis. Delegates to the per-tensor overload when x_scale is scalar.

void operator()(const Tensor &x, const Tensor &x_scale, const Tensor &x_zero_point, int64_t axis, Tensor &output) const

Tensor operator()(const Tensor &x, const Tensor &x_scale, int64_t axis) const

Per-axis overloads with the x_zero_point omitted (defaults to 0). Delegates to the per-tensor overload when x_scale is scalar.

void operator()(const Tensor &x, const Tensor &x_scale, int64_t axis, Tensor &output) const

inline explicit KernelBase(const KernelContext &ctx)

Public Static Functions

static inline constexpr bool CanRunInPlace() noexcept

Output element type (FLOAT) differs from the integer/float8 input element type, so storage can never be shared with an input.

class DynamicQuantizeLinear : public onnx_light::onnx_kernels::kernel::KernelBase

#include <include_quantization_kernels.h>

Per-tensor dynamic linear quantization of a FLOAT input x to a UINT8 output y together with the derived scalar y_scale (FLOAT) and scalar y_zero_point (UINT8). Matches the ONNX DynamicQuantizeLinear operator (opset 11) for the canonical uint8 case: y_scale = (max(0, max(x)) - min(0, min(x))) / 255 y_zero_point = saturate(round(-min(0, min(x)) / y_scale)) y = saturate(round(x / y_scale) + y_zero_point)

Public Functions

std::tuple<Tensor, Tensor, Tensor> operator()(const Tensor &x) const

Returning overload: produces {y, y_scale, y_zero_point}.

void operator()(const Tensor &x, Tensor &y, Tensor &y_scale, Tensor &y_zero_point) const

In-place overload writing into caller-allocated outputs.

inline explicit KernelBase(const KernelContext &ctx)

Public Static Functions

static inline constexpr bool CanRunInPlace() noexcept

Output element type (UINT8) differs from the FLOAT input element type, so storage can never be shared with the input.

class QLinearConv : public onnx_light::onnx_kernels::kernel::KernelBase

#include <include_quantization_kernels.h>

Reference N-D QLinearConv kernel for the per-tensor (or per-output-channel w_scale / w_zero_point) case, restricted to INT8/UINT8 x, w and y and FLOAT scales. Implements y = saturate(round(((x - x_zp) * x_scale) * ((w - w_zp) * w_scale) / y_scale + bias * x_scale * w_scale / y_scale) + y_zp) using the standard :cpp:class:Conv shape/padding/dilation rules.

Public Functions

Tensor operator()(const Tensor &x, const Tensor &x_scale, const Tensor &x_zero_point, const Tensor &w, const Tensor &w_scale, const Tensor &w_zero_point, const Tensor &y_scale, const Tensor &y_zero_point, const Tensor &B, const Attributes &attrs) const

Returning overload. B may be a default-constructed (empty) Tensor to indicate the optional bias is missing.

void operator()(const Tensor &x, const Tensor &x_scale, const Tensor &x_zero_point, const Tensor &w, const Tensor &w_scale, const Tensor &w_zero_point, const Tensor &y_scale, const Tensor &y_zero_point, const Tensor &B, const Attributes &attrs, Tensor &output) const

inline explicit KernelBase(const KernelContext &ctx)

Public Static Functions

static inline constexpr bool CanRunInPlace() noexcept

Output dtype/shape generally differs from any input, so storage cannot be shared.

struct Attributes

#include <include_quantization_kernels.h>

Attributes carried by the ONNX QLinearConv operator.

Public Members

std::vector<int64_t> kernel_shape

std::vector<int64_t> strides

std::vector<int64_t> pads

std::vector<int64_t> dilations

int64_t group = 1

std::string auto_pad = "NOTSET"

class QLinearMatMul : public onnx_light::onnx_kernels::kernel::KernelBase

#include <include_quantization_kernels.h>

Reference QLinearMatMul kernel for the per-tensor (scalar scales and scalar zero-points) case, restricted to INT8/UINT8 inputs and outputs and FLOAT scales. Implements y = saturate(round(((a - a_zp) * a_scale) * ((b - b_zp) * b_scale) / y_scale) + y_zp). Matrix multiplication follows the standard :cpp:class:MatMul broadcasting rules.

Public Functions

Tensor operator()(const Tensor &a, const Tensor &a_scale, const Tensor &a_zero_point, const Tensor &b, const Tensor &b_scale, const Tensor &b_zero_point, const Tensor &y_scale, const Tensor &y_zero_point) const

Returning overload. Output element type is taken from y_zero_point.

void operator()(const Tensor &a, const Tensor &a_scale, const Tensor &a_zero_point, const Tensor &b, const Tensor &b_scale, const Tensor &b_zero_point, const Tensor &y_scale, const Tensor &y_zero_point, Tensor &output) const

In-place overload writing into a caller-allocated output.

inline explicit KernelBase(const KernelContext &ctx)

Public Static Functions

static inline constexpr bool CanRunInPlace() noexcept

Output dtype/shape generally differs from any input, so storage cannot be shared.

class QuantizeLinear : public onnx_light::onnx_kernels::kernel::KernelBase

#include <include_quantization_kernels.h>

Per-tensor and per-axis linear quantization of a FLOAT input x to an integer or sub-byte output. The output element type is taken from y_zero_point (UINT8, INT8, UINT16, INT16, FLOAT8E4M3FN, FLOAT8E5M2, INT4, UINT4, INT2, UINT2, or FLOAT4E2M1); if y_zero_point is omitted the output defaults to UINT8 with a zero point of 0.

Public Functions

Tensor operator()(const Tensor &x, const Tensor &y_scale) const

Omitted y_zero_point: output is UINT8 with zero point 0.

void operator()(const Tensor &x, const Tensor &y_scale, Tensor &output) const

Tensor operator()(const Tensor &x, const Tensor &y_scale, const Tensor &y_zero_point) const

Explicit y_zero_point: its data_type drives the output element type. Per-tensor (scalar y_scale) quantization.

void operator()(const Tensor &x, const Tensor &y_scale, const Tensor &y_zero_point, Tensor &output) const

Tensor operator()(const Tensor &x, const Tensor &y_scale, const Tensor &y_zero_point, int64_t axis) const

Per-axis quantization: y_scale (and y_zero_point) may have one entry per slice along axis. When y_scale.element_count() == 1 the call is forwarded to the scalar overload above.

void operator()(const Tensor &x, const Tensor &y_scale, const Tensor &y_zero_point, int64_t axis, Tensor &output) const

Tensor operator()(const Tensor &x, const Tensor &y_scale, int64_t axis, int32_t output_dtype) const

Per-axis/blocked quantization without explicit y_zero_point. output_dtype specifies the output element type; zero point is 0.

void operator()(const Tensor &x, const Tensor &y_scale, int64_t axis, int32_t output_dtype, Tensor &output) const

inline explicit KernelBase(const KernelContext &ctx)

Public Static Functions

static inline constexpr bool CanRunInPlace() noexcept

Output element type differs from the FLOAT input element type, so storage can never be shared with an input.