QLinearMatMul#

  • Domain: ai.onnx

  • Since version: 21

Matrix product that behaves like numpy.matmul. It consumes two quantized input tensors, their scales and zero points, scale and zero point of output, and computes the quantized output. The quantization formula is y = saturate((x / y_scale) + y_zero_point). For (x / y_scale), it is rounding to nearest ties to even. Refer to https://en.wikipedia.org/wiki/Rounding for details. Scale and zero point must have same shape. They must be either scalar (per tensor) or N-D tensor (per row for ‘a’ and per column for ‘b’). Scalar refers to per tensor quantization whereas N-D refers to per row or per column quantization. If the input is 2D of shape [M, K] then zero point and scale tensor may be an M element vector [v_1, v_2, …, v_M] for per row quantization and K element vector of shape [v_1, v_2, …, v_K] for per column quantization. If the input is N-D tensor with shape [D1, D2, M, K] then zero point and scale tensor may have shape [D1, D2, M, 1] for per row quantization and shape [D1, D2, 1, K] for per column quantization. Production must never overflow, and accumulation may overflow if and only if in 32 bits.

Inputs

  • a (T1): N-dimensional quantized matrix a

  • a_scale (TS): scale of quantized input a

  • a_zero_point (T1): zero point of quantized input a

  • b (T2): N-dimensional quantized matrix b

  • b_scale (TS): scale of quantized input b

  • b_zero_point (T2): zero point of quantized input b

  • y_scale (TS): scale of quantized output y

  • y_zero_point (T3): zero point of quantized output y

Outputs

  • y (T3): Quantized matrix multiply results from a * b

Type Constraints

  • TS: Constrain scales. Allowed types: tensor(bfloat16), tensor(float), tensor(float16).

  • T1: The type of input a and its zeropoint. Allowed types: tensor(float8e4m3fn), tensor(float8e4m3fnuz), tensor(float8e5m2), tensor(float8e5m2fnuz), tensor(int8), tensor(uint8).

  • T2: The type of input b and its zeropoint. Allowed types: tensor(float8e4m3fn), tensor(float8e4m3fnuz), tensor(float8e5m2), tensor(float8e5m2fnuz), tensor(int8), tensor(uint8).

  • T3: The type of the output and its zeropoint. Allowed types: tensor(float8e4m3fn), tensor(float8e4m3fnuz), tensor(float8e5m2), tensor(float8e5m2fnuz), tensor(int8), tensor(uint8).

Differences with previous version (10)#

SchemaDiff: QLinearMatMul (domain 'ai.onnx')

  • old version: 10

  • new version: 21

  • breaking: yes

Breaking reasons:

  • input ‘a_scale’ (changed): type_str changed ‘tensor(float)’ -> ‘TS’

  • input ‘b_scale’ (changed): type_str changed ‘tensor(float)’ -> ‘TS’

  • input ‘y_scale’ (changed): type_str changed ‘tensor(float)’ -> ‘TS’

Inputs:

  • [BREAKING] changed ‘a_scale’: type_str changed ‘tensor(float)’ -> ‘TS’

  • [BREAKING] changed ‘b_scale’: type_str changed ‘tensor(float)’ -> ‘TS’

  • [BREAKING] changed ‘y_scale’: type_str changed ‘tensor(float)’ -> ‘TS’

Type constraints:

  • added ‘TS’: added types: [‘tensor(bfloat16)’, ‘tensor(float)’, ‘tensor(float16)’]

  • changed ‘T1’: added types: [‘tensor(float8e4m3fn)’, ‘tensor(float8e4m3fnuz)’, ‘tensor(float8e5m2)’, ‘tensor(float8e5m2fnuz)’]

  • changed ‘T2’: added types: [‘tensor(float8e4m3fn)’, ‘tensor(float8e4m3fnuz)’, ‘tensor(float8e5m2)’, ‘tensor(float8e5m2fnuz)’]

  • changed ‘T3’: added types: [‘tensor(float8e4m3fn)’, ‘tensor(float8e4m3fnuz)’, ‘tensor(float8e5m2)’, ‘tensor(float8e5m2fnuz)’]

Version History#