.. _op_ai_onnx_MatMulInteger:

MatMulInteger
=============

- **Domain**: ``ai.onnx``
- **Since version**: 10

Matrix product that behaves like `numpy.matmul <https://numpy.org/doc/stable/reference/generated/numpy.matmul.html>`_.
The production MUST never overflow. The accumulation may overflow if and only if in 32 bits.

**Inputs**

- **A** (*T1*): N-dimensional matrix A
- **B** (*T2*): N-dimensional matrix B
- **a_zero_point** (*T1*): Zero point tensor for input 'A'. It's optional and default value is 0. It could be a scalar or N-D tensor. Scalar refers to per tensor quantization whereas N-D refers to per row quantization. If the input is 2D of shape [M, K] then zero point tensor may be an M element vector [zp_1, zp_2, ..., zp_M]. If the input is N-D tensor with shape [D1, D2, M, K] then zero point tensor may have shape [D1, D2, M, 1].
- **b_zero_point** (*T2*): Zero point tensor for input 'B'. It's optional and default value is 0. It could be a scalar or a N-D tensor, Scalar refers to per tensor quantization whereas N-D refers to per col quantization. If the input is 2D of shape [K, N] then zero point tensor may be an N element vector [zp_1, zp_2, ..., zp_N]. If the input is N-D tensor with shape [D1, D2, K, N] then zero point tensor may have shape [D1, D2, 1, N].

**Outputs**

- **Y** (*T3*): Matrix multiply results from A \* B

**Type Constraints**

- **T1**: Constrain input A data type to 8-bit integer tensor.
  Allowed types: tensor(int8), tensor(uint8).
- **T2**: Constrain input B data type to 8-bit integer tensor.
  Allowed types: tensor(int8), tensor(uint8).
- **T3**: Constrain output Y data type as 32-bit integer tensor.
  Allowed types: tensor(int32).

Examples
--------

**test_cc_matmulinteger**

.. code-block:: text

    Node:
      MatMulInteger(A, B, a_zero_point, b_zero_point) -> (Y)

.. code-block:: text

    Inputs:
      A: shape=(4, 3), dtype=uint8
        [[11,  7,  3],
         [10,  6,  2],
         [ 9,  5,  1],
         [ 8,  4,  0]]
      B: shape=(3, 2), dtype=uint8
        [[1, 4],
         [2, 5],
         [3, 6]]
      a_zero_point: shape=(1,), dtype=uint8
        [12]
      b_zero_point: shape=(1,), dtype=uint8
        [0]

    Outputs:
      Y: shape=(4, 2), dtype=int32
        [[ -38,  -83],
         [ -44,  -98],
         [ -50, -113],
         [ -56, -128]]

**test_cc_matmulinteger_int8**

.. code-block:: text

    Node:
      MatMulInteger(A, B, a_zero_point, b_zero_point) -> (Y)

.. code-block:: text

    Inputs:
      A: shape=(2, 3), dtype=int8
        [[ 1, -2,  3],
         [-4,  5, -6]]
      B: shape=(3, 2), dtype=int8
        [[ 1,  2],
         [-3,  4],
         [ 5, -6]]
      a_zero_point: shape=(), dtype=int8
        1
      b_zero_point: shape=(), dtype=int8
        -1

    Outputs:
      Y: shape=(2, 2), dtype=int32
        [[ 18, -25],
         [-60,  40]]

**test_cc_matmulinteger_per_col_b_zp**

.. code-block:: text

    Node:
      MatMulInteger(A, B, "", b_zero_point) -> (Y)

.. code-block:: text

    Inputs:
      A: shape=(2, 3), dtype=uint8
        [[11,  7,  3],
         [10,  6,  2]]
      B: shape=(3, 2), dtype=uint8
        [[1, 4],
         [2, 5],
         [3, 6]]
      b_zero_point: shape=(2,), dtype=uint8
        [1, 2]

    Outputs:
      Y: shape=(2, 2), dtype=int32
        [[13, 55],
         [10, 46]]

**test_cc_matmulinteger_per_row_a_zp**

.. code-block:: text

    Node:
      MatMulInteger(A, B, a_zero_point, "") -> (Y)

.. code-block:: text

    Inputs:
      A: shape=(2, 3), dtype=uint8
        [[11,  7,  3],
         [10,  6,  2]]
      B: shape=(3, 2), dtype=uint8
        [[1, 4],
         [2, 5],
         [3, 6]]
      a_zero_point: shape=(2,), dtype=uint8
        [1, 2]

    Outputs:
      Y: shape=(2, 2), dtype=int32
        [[28, 82],
         [16, 52]]