.. _op_ai_rt_Quantize:

Quantize
========

- **Domain**: ``ai.rt``
- **Since version**: 1

Encodes a finite floating tensor in the storage layout specified by type (a TYPE_PROTO attribute containing StructTypeProto). Optional inputs are scales, zero_points, offsets, codebooks, permutation, forward, inverse and outliers. Omitted scales are calibrated per block from the source range; cast blocks use one. Scalar numerical parameters broadcast, otherwise one value is required per block. Fixed scalar codebooks have profile defaults. Learned codebooks, vector/additive codebook scales and nonempty transforms/permutations/outlier indices must be supplied. This does not train GPTQ/AWQ/codebooks. Output preserves the source logical shape and dtype. With parameter_ref, a model-local fixed parameter set replaces all optional inputs and calibration; the output carries only local codes/outlier values in a compact storage schema plus the reference. Otherwise physical storage uses the requested type.

**Inputs**

- **X** (*T*): Floating tensor to encode.
- **scales** (*P1*): Optional scalar or per-block scales.
- **zero_points** (*P2*): Optional scalar or per-block zero points.
- **offsets** (*P3*): Optional scalar or per-block offsets.
- **codebooks** (*P4*): Optional concatenated per-block codebooks.
- **permutation** (*I*): Optional gather permutation.
- **forward** (*P5*): Optional forward transform matrix.
- **inverse** (*P6*): Optional inverse transform matrix.
- **outliers** (*I*): Optional original-source outlier indices.

**Outputs**

- **Y** (*E*): EncodedValueProto carrying its local storage type and parameter ownership.

**Attributes**

- **parameter_ref** (*string*): Model-local fixed numerical parameter set; excludes optional inputs.
- **type** (*type_proto*): Required quantization StructTypeProto wrapped in TypeProto.

**Type Constraints**

- **T**
  Allowed types: tensor(bfloat16), tensor(double), tensor(float), tensor(float16).
- **P1**
  Allowed types: tensor(bfloat16), tensor(double), tensor(float), tensor(float16).
- **P2**
  Allowed types: tensor(bfloat16), tensor(double), tensor(float), tensor(float16).
- **P3**
  Allowed types: tensor(bfloat16), tensor(double), tensor(float), tensor(float16).
- **P4**
  Allowed types: tensor(bfloat16), tensor(double), tensor(float), tensor(float16).
- **P5**
  Allowed types: tensor(bfloat16), tensor(double), tensor(float), tensor(float16).
- **P6**
  Allowed types: tensor(bfloat16), tensor(double), tensor(float), tensor(float16).
- **I**
  Allowed types: tensor(int64).
- **E**: Structured encoded value.
  Allowed types: struct.

Examples
--------

**test_cc_quantize_int4**

.. code-block:: text

    Node:
      ai.rt.Quantize(X) -> (Q)
      Attributes:
        type = <attribute type 13>

.. code-block:: text

    Inputs:
      X: shape=(3,), dtype=float32
        [-1.,  0.,  1.]

    Outputs:
      Q: shape=(3,), dtype=float64
        [-1.,  0.,  1.]

**test_cc_quantize_nf4**

.. code-block:: text

    Node:
      ai.rt.Quantize(X) -> (Q)
      Attributes:
        type = <attribute type 13>

.. code-block:: text

    Inputs:
      X: shape=(3,), dtype=float32
        [-1.,  0.,  1.]

    Outputs:
      Q: shape=(3,), dtype=float64
        [-1.,  0.,  1.]

**test_cc_quantize_ternary**

.. code-block:: text

    Node:
      ai.rt.Quantize(X) -> (Q)
      Attributes:
        type = <attribute type 13>

.. code-block:: text

    Inputs:
      X: shape=(3,), dtype=float32
        [-1.,  0.,  1.]

    Outputs:
      Q: shape=(3,), dtype=float64
        [-1.,  0.,  1.]

**test_cc_quantize_tiled_float**

.. code-block:: text

    Node:
      ai.rt.Quantize(X) -> (Q)
      Attributes:
        type = <attribute type 13>

.. code-block:: text

    Inputs:
      X: shape=(3,), dtype=float32
        [-1.,  0.,  1.]

    Outputs:
      Q: shape=(3,), dtype=float64
        [-1.,  0.,  1.]