:nosearch:

.. _op_ai_onnx_QuantizeLinear-25:

QuantizeLinear - version 25
===========================

This page documents version **25** of operator **QuantizeLinear**. See :doc:`QuantizeLinear` for the latest version (since version 28).

- **Domain**: ``ai.onnx``
- **Since version**: 25

The linear quantization operator consumes a high-precision tensor, a scale, and a zero point to compute the
low-precision/quantized tensor. The scale factor and zero point must have the same shape, determining the quantization
granularity. The quantization formula is ``y = saturate((x / y_scale) + y_zero_point)``.

Saturation is done according to:

- uint16: [0, 65535]
- int16: [-32768, 32767]
- uint8: [0, 255]
- int8: [-128, 127]
- uint4: [0, 15]
- int4: [-8, 7]
- uint2: [0, 3]
- int2: [-2, 1]

For ``(x / y_scale)``, it rounds to the nearest even. Refer to https://en.wikipedia.org/wiki/Rounding for details.

``y_zero_point`` and ``y`` must have the same type. ``y_zero_point`` is usually not used for quantization to float8 and 4bit types, but the quantization
formula remains the same for consistency, and the type of the attribute ``y_zero_point`` still determines the quantization type.
``x`` and ``y_scale`` are allowed to have different types. The type of ``y_scale`` determines the precision of the division operation between ``x`` and
``y_scale``, unless the ``precision`` attribute is specified.

There are three supported quantization granularities, determined by the shape of ``y_scale``.
In all cases, ``y_zero_point`` must have the same shape as ``y_scale``.

- Per-tensor (per-layer) quantization: ``y_scale`` is a scalar.
- Per-axis quantization: The scale must be a 1-D tensor, with the length of the quantization axis. For an input shape
  ``(D0, ..., Di, ..., Dn)`` and ``axis=i``, ``y_scale`` is a 1-D tensor of length ``Di``.
- Blocked quantization: The scale's shape is identical to the input's shape, except for one dimension, in which
  blocking is performed. Given ``x`` shape ``(D0, ..., Di, ..., Dn)``, ``axis=i``, and block size ``B``: ``y_scale`` shape is
  ``(D0, ..., ceil(Di/B), ..., Dn)``.

**Inputs**

- **x** (*T1*): N-D full precision Input tensor to be quantized.
- **y_scale** (*T2*): Scale for doing quantization to get ``y``. For per-tensor/layer quantization the scale is a scalar, for per-axis quantization it is a 1-D Tensor and for blocked quantization it has the same shape as the input, except for one dimension in which blocking is performed.
- **y_zero_point** (*T3*): Zero point for doing quantization to get ``y``. Shape must match ``y_scale``. Default is uint8 with zero point of 0 if it's not specified.

**Outputs**

- **y** (*T3*): N-D quantized output tensor. It has same shape as input ``x``.

**Type Constraints**

- **T1**: The type of the input 'x'.
  Allowed types: tensor(bfloat16), tensor(float), tensor(float16), tensor(int32).
- **T2**: The type of the input 'y_scale'.
  Allowed types: tensor(bfloat16), tensor(float), tensor(float16), tensor(float8e8m0), tensor(int32).
- **T3**: The type of the input ``y_zero_point`` and the output ``y``.
  Allowed types: tensor(float4e2m1), tensor(float8e4m3fn), tensor(float8e4m3fnuz), tensor(float8e5m2), tensor(float8e5m2fnuz), tensor(int16), tensor(int2), tensor(int4), tensor(int8), tensor(uint16), tensor(uint2), tensor(uint4), tensor(uint8).

Examples
--------

**test_cc_quantizelinear**

.. code-block:: text

    Node:
      QuantizeLinear(x, y_scale) -> (y)

.. code-block:: text

    Inputs:
      x: shape=(6,), dtype=float32
        [    0.,     2.,     3.,  1000.,  -254., -1000.]
      y_scale: shape=(), dtype=float32
        2.

    Outputs:
      y: shape=(6,), dtype=uint8
        [  0,   1,   2, 255,   0,   0]

**test_cc_quantizelinear_int8**

.. code-block:: text

    Node:
      QuantizeLinear(x, y_scale, y_zero_point) -> (y)

.. code-block:: text

    Inputs:
      x: shape=(6,), dtype=float32
        [    0.,     2.,     3.,  1000.,  -254., -1000.]
      y_scale: shape=(), dtype=float32
        2.
      y_zero_point: shape=(), dtype=int8
        -10

    Outputs:
      y: shape=(6,), dtype=int8
        [ -10,   -9,   -8,  127, -128, -128]

**test_quantizelinear**

.. code-block:: text

    Node:
      QuantizeLinear(x, y_scale, y_zero_point) -> (y)

.. code-block:: text

    Inputs:
      x: shape=(6,), dtype=float32
        [    0.,     2.,     3.,  1000.,  -254., -1000.]
      y_scale: shape=(), dtype=float32
        2.
      y_zero_point: shape=(), dtype=uint8
        128

    Outputs:
      y: shape=(6,), dtype=uint8
        [128, 129, 130, 255,   1,   0]

**test_quantizelinear_axis**

.. code-block:: text

    Node:
      QuantizeLinear(x, y_scale, y_zero_point) -> (y)
      Attributes:
        axis = 1

.. code-block:: text

    Inputs:
      x: shape=(1, 3, 3, 2), dtype=float32
        [[[[-162.,   10.],
           [-100.,  232.],
           [ -20.,  -50.]],
        
          [[ -76.,    0.],
           [   0.,  252.],
           [  32.,  -44.]],
        
          [[ 245., -485.],
           [-960., -270.],
           [-375., -470.]]]]
      y_scale: shape=(3,), dtype=float32
        [2., 4., 5.]
      y_zero_point: shape=(3,), dtype=uint8
        [ 84,  24, 196]

    Outputs:
      y: shape=(1, 3, 3, 2), dtype=uint8
        [[[[  3,  89],
           [ 34, 200],
           [ 74,  59]],
        
          [[  5,  24],
           [ 24,  87],
           [ 32,  13]],
        
          [[245,  99],
           [  4, 142],
           [121, 102]]]]

**test_quantizelinear_blocked_asymmetric**

.. code-block:: text

    Node:
      QuantizeLinear(x, y_scale, y_zero_point) -> (y)
      Attributes:
        axis = 1
        block_size = 2

.. code-block:: text

    Inputs:
      x: shape=(3, 4), dtype=float32
        [[ 6., 12., 50.,  5.],
         [ 1.,  8.,  4.,  5.],
         [ 0., 20., 10.,  4.]]
      y_scale: shape=(3, 2), dtype=float32
        [[1.5, 2.5],
         [3. , 4.9],
         [5.1, 6.9]]
      y_zero_point: shape=(3, 2), dtype=uint8
        [[0, 1],
         [1, 0],
         [2, 3]]

    Outputs:
      y: shape=(3, 4), dtype=uint8
        [[ 4,  8, 21,  3],
         [ 1,  4,  1,  1],
         [ 2,  6,  4,  4]]

**test_quantizelinear_blocked_symmetric**

.. code-block:: text

    Node:
      QuantizeLinear(x, y_scale) -> (y)
      Attributes:
        axis = 1
        block_size = 2
        output_dtype = 5

.. code-block:: text

    Inputs:
      x: shape=(3, 4), dtype=float32
        [[  6.,  -8., -10.,   5.],
         [  1.,   8.,   4.,   5.],
         [  0.,  20.,  10.,   4.]]
      y_scale: shape=(3, 2), dtype=float32
        [[1.5, 2.5],
         [3. , 4.9],
         [5.1, 6.9]]

    Outputs:
      y: shape=(3, 4), dtype=int16
        [[ 4, -5, -4,  2],
         [ 0,  3,  1,  1],
         [ 0,  4,  1,  1]]

**test_quantizelinear_e4m3fn**

.. code-block:: text

    Node:
      QuantizeLinear(x, y_scale, y_zero_point) -> (y)

.. code-block:: text

    Inputs:
      x: shape=(5,), dtype=float32
        [0.e+00, 1.e+00, 2.e+00, 1.e+05, 2.e+02]
      y_scale: shape=(), dtype=float32
        2.
      y_zero_point: shape=(1,), dtype=float8_e4m3fn
        [0]

    Outputs:
      y: shape=(5,), dtype=float8_e4m3fn
        [0, 0.5, 1, 448, 96]

**test_quantizelinear_e5m2**

.. code-block:: text

    Node:
      QuantizeLinear(x, y_scale, y_zero_point) -> (y)

.. code-block:: text

    Inputs:
      x: shape=(5,), dtype=float32
        [0.e+00, 1.e+00, 2.e+00, 1.e+05, 2.e+02]
      y_scale: shape=(), dtype=float32
        2.
      y_zero_point: shape=(1,), dtype=float8_e5m2
        [0]

    Outputs:
      y: shape=(5,), dtype=float8_e5m2
        [0, 0.5, 1, 49152, 96]

**test_quantizelinear_float4e2m1**

.. code-block:: text

    Node:
      QuantizeLinear(x, y_scale, y_zero_point) -> (y)
      Attributes:
        axis = 0

.. code-block:: text

    Inputs:
      x: shape=(3, 4), dtype=float32
        [[  0. ,   2.5,   4.8,   8.6],
         [-30. , -20. ,   6. ,   9. ],
         [ -0. ,  -2.5,  -4.8,  -8.6]]
      y_scale: shape=(3,), dtype=float32
        [2., 3., 4.]
      y_zero_point: shape=(3,), dtype=float4_e2m1fn
        [0, 0, 0]

    Outputs:
      y: shape=(3, 4), dtype=float4_e2m1fn
        [[0, 1, 2, 4],
         [-6, -6, 2, 3],
         [0, -0.5, -1, -2]]

**test_quantizelinear_float4e2m1_signed_zero**

.. code-block:: text

    Node:
      QuantizeLinear(x, y_scale) -> (y)
      Attributes:
        output_dtype = 23

.. code-block:: text

    Inputs:
      x: shape=(2,), dtype=float32
        [-0.,  0.]
      y_scale: shape=(), dtype=float32
        1.

    Outputs:
      y: shape=(2,), dtype=float4_e2m1fn
        [-0, 0]

**test_quantizelinear_int16**

.. code-block:: text

    Node:
      QuantizeLinear(x, y_scale, y_zero_point) -> (y)

.. code-block:: text

    Inputs:
      x: shape=(4,), dtype=float32
        [ 0.e+00,  2.e+00,  3.e+00, -1.e+05]
      y_scale: shape=(), dtype=float32
        2.
      y_zero_point: shape=(), dtype=int16
        -1024

    Outputs:
      y: shape=(4,), dtype=int16
        [ -1024,  -1023,  -1022, -32768]

**test_quantizelinear_int2**

.. code-block:: text

    Node:
      QuantizeLinear(x, y_scale, y_zero_point) -> (y)
      Attributes:
        axis = 0

.. code-block:: text

    Inputs:
      x: shape=(3, 4), dtype=float32
        [[ 0. ,  2.5,  4.8,  8.6],
         [-4. , -3. ,  1. ,  2. ],
         [-0. , -2.5, -4.8, -8.6]]
      y_scale: shape=(3,), dtype=float32
        [2., 3., 4.]
      y_zero_point: shape=(3,), dtype=int2
        [0, 0, 0]

    Outputs:
      y: shape=(3, 4), dtype=int2
        [[0, 1, 1, 1],
         [-1, -1, 0, 1],
         [0, -1, -1, -2]]

**test_quantizelinear_int4**

.. code-block:: text

    Node:
      QuantizeLinear(x, y_scale, y_zero_point) -> (y)
      Attributes:
        axis = 0

.. code-block:: text

    Inputs:
      x: shape=(3, 4), dtype=float32
        [[  0. ,   2.5,   4.8,   8.6],
         [-30. , -20. ,   6. ,   9. ],
         [ 12. ,  15. ,  16. ,  40. ]]
      y_scale: shape=(3,), dtype=float32
        [2., 3., 4.]
      y_zero_point: shape=(3,), dtype=int4
        [1, 1, 1]

    Outputs:
      y: shape=(3, 4), dtype=int4
        [[1, 2, 3, 5],
         [-8, -6, 3, 4],
         [4, 5, 5, 7]]

**test_quantizelinear_uint16**

.. code-block:: text

    Node:
      QuantizeLinear(x, y_scale, y_zero_point) -> (y)

.. code-block:: text

    Inputs:
      x: shape=(4,), dtype=float32
        [0.e+00, 2.e+00, 3.e+00, 2.e+05]
      y_scale: shape=(), dtype=float32
        2.
      y_zero_point: shape=(), dtype=uint16
        32767

    Outputs:
      y: shape=(4,), dtype=uint16
        [32767, 32768, 32769, 65535]

**test_quantizelinear_uint2**

.. code-block:: text

    Node:
      QuantizeLinear(x, y_scale, y_zero_point) -> (y)
      Attributes:
        axis = 0

.. code-block:: text

    Inputs:
      x: shape=(3, 4), dtype=float32
        [[ 0. ,  2.5,  4.8,  8.6],
         [-2. , -1. ,  1. ,  3. ],
         [ 4. ,  5. ,  6. ,  7. ]]
      y_scale: shape=(3,), dtype=float32
        [2., 3., 4.]
      y_zero_point: shape=(3,), dtype=uint2
        [0, 0, 0]

    Outputs:
      y: shape=(3, 4), dtype=uint2
        [[0, 1, 2, 3],
         [0, 0, 0, 1],
         [1, 1, 2, 2]]

**test_quantizelinear_uint4**

.. code-block:: text

    Node:
      QuantizeLinear(x, y_scale, y_zero_point) -> (y)
      Attributes:
        axis = 0

.. code-block:: text

    Inputs:
      x: shape=(3, 4), dtype=float32
        [[  0. ,   2.5,   4.8,   8.6],
         [-30. , -20. ,   6. ,   9. ],
         [ 12. ,  15. ,  16. ,  40. ]]
      y_scale: shape=(3,), dtype=float32
        [2., 3., 4.]
      y_zero_point: shape=(3,), dtype=uint4
        [1, 1, 1]

    Outputs:
      y: shape=(3, 4), dtype=uint4
        [[1, 2, 3, 5],
         [0, 0, 3, 4],
         [4, 5, 5, 11]]

Differences with previous version (24)
--------------------------------------

**SchemaDiff**: ``QuantizeLinear`` (domain ``'ai.onnx'``)

* old version: 24
* new version: 25
* breaking: no

**Type constraints:**

* changed 'T3': added types: ['tensor(int2)', 'tensor(uint2)']

**Documentation:**

* line similarity: 0.97 (+2/-0 lines)

.. code-block:: diff

    --- QuantizeLinear v24
    +++ QuantizeLinear v25
    @@ -10,6 +10,8 @@
     - int8: [-128, 127]
     - uint4: [0, 15]
     - int4: [-8, 7]
    +- uint2: [0, 3]
    +- int2: [-2, 1]
     
     For `(x / y_scale)`, it rounds to the nearest even. Refer to https://en.wikipedia.org/wiki/Rounding for details.