MatMulInteger

• Domain: ai.onnx

• Since version: 10

Matrix product that behaves like numpy.matmul. The production MUST never overflow. The accumulation may overflow if and only if in 32 bits.

Inputs

• A (T1): N-dimensional matrix A

• B (T2): N-dimensional matrix B

• a_zero_point (T1): Zero point tensor for input ‘A’. It’s optional and default value is 0. It could be a scalar or N-D tensor. Scalar refers to per tensor quantization whereas N-D refers to per row quantization. If the input is 2D of shape [M, K] then zero point tensor may be an M element vector [zp_1, zp_2, …, zp_M]. If the input is N-D tensor with shape [D1, D2, M, K] then zero point tensor may have shape [D1, D2, M, 1].

• b_zero_point (T2): Zero point tensor for input ‘B’. It’s optional and default value is 0. It could be a scalar or a N-D tensor, Scalar refers to per tensor quantization whereas N-D refers to per col quantization. If the input is 2D of shape [K, N] then zero point tensor may be an N element vector [zp_1, zp_2, …, zp_N]. If the input is N-D tensor with shape [D1, D2, K, N] then zero point tensor may have shape [D1, D2, 1, N].

Outputs

• Y (T3): Matrix multiply results from A * B

Type Constraints

• T1: Constrain input A data type to 8-bit integer tensor. Allowed types: tensor(int8), tensor(uint8).

• T2: Constrain input B data type to 8-bit integer tensor. Allowed types: tensor(int8), tensor(uint8).

• T3: Constrain output Y data type as 32-bit integer tensor. Allowed types: tensor(int32).

Examples

test_cc_matmulinteger

Node:
  MatMulInteger(A, B, a_zero_point, b_zero_point) -> (Y)

Inputs:
  A: shape=(4, 3), dtype=uint8
    [[11,  7,  3],
     [10,  6,  2],
     [ 9,  5,  1],
     [ 8,  4,  0]]
  B: shape=(3, 2), dtype=uint8
    [[1, 4],
     [2, 5],
     [3, 6]]
  a_zero_point: shape=(1,), dtype=uint8
    [12]
  b_zero_point: shape=(1,), dtype=uint8
    [0]

Outputs:
  Y: shape=(4, 2), dtype=int32
    [[ -38,  -83],
     [ -44,  -98],
     [ -50, -113],
     [ -56, -128]]

test_cc_matmulinteger_int8

Node:
  MatMulInteger(A, B, a_zero_point, b_zero_point) -> (Y)

Inputs:
  A: shape=(2, 3), dtype=int8
    [[ 1, -2,  3],
     [-4,  5, -6]]
  B: shape=(3, 2), dtype=int8
    [[ 1,  2],
     [-3,  4],
     [ 5, -6]]
  a_zero_point: shape=(), dtype=int8
    1
  b_zero_point: shape=(), dtype=int8
    -1

Outputs:
  Y: shape=(2, 2), dtype=int32
    [[ 18, -25],
     [-60,  40]]

test_cc_matmulinteger_per_col_b_zp

Node:
  MatMulInteger(A, B, "", b_zero_point) -> (Y)

Inputs:
  A: shape=(2, 3), dtype=uint8
    [[11,  7,  3],
     [10,  6,  2]]
  B: shape=(3, 2), dtype=uint8
    [[1, 4],
     [2, 5],
     [3, 6]]
  b_zero_point: shape=(2,), dtype=uint8
    [1, 2]

Outputs:
  Y: shape=(2, 2), dtype=int32
    [[13, 55],
     [10, 46]]

test_cc_matmulinteger_per_row_a_zp

Node:
  MatMulInteger(A, B, a_zero_point, "") -> (Y)

Inputs:
  A: shape=(2, 3), dtype=uint8
    [[11,  7,  3],
     [10,  6,  2]]
  B: shape=(3, 2), dtype=uint8
    [[1, 4],
     [2, 5],
     [3, 6]]
  a_zero_point: shape=(2,), dtype=uint8
    [1, 2]

Outputs:
  Y: shape=(2, 2), dtype=int32
    [[28, 82],
     [16, 52]]