GatherBlockQuantized (CPU)
==========================

* Domain: ``com.microsoft``
* Device: ``CPU``
* Kernel: ``onnx_light_cpu::GatherBlockQuantized``
* Supported types: ``UINT8``
* Opset bounds: since opset 1, until opset 1

Operator support
----------------

.. list-table::
   :header-rows: 1
   :widths: 30 70

   * - Capability
     - Implementation
   * - Shape inference
     - :cpp:func:`onnx_light_cpu::ComputeShapeGatherBlockQuantized`
   * - Peak memory
     - :cpp:func:`onnx_light_cpu::ComputePeakMemoryGatherBlockQuantized`
   * - Fusion patterns
     - None
   * - Gradient
     - Not provided

LightOpSchema
~~~~~~~~~~~~~

**Version:** 1

Gathers rows from a block-quantized UINT8 tensor of rank at least two and dequantizes them. Only gather_axis=0 and last-axis quantization are supported, including equivalent negative axes. The logical last dimension is the packed last dimension times 8/bits.

**Inputs**

- **data** (``T1``): UINT8 packed data of rank at least two.
- **indices** (``Tind``): INT32 or INT64 indices of any rank, including scalar.
- **scales** (``T2``): FLOAT or FLOAT16 scales, matching data except the last dimension is ceil(logical_last / block_size).
- **zero_points** (``T1``): Optional UINT8 zero points, matching scales except the last dimension is ceil(scales_last / (8/bits)).

**Outputs**

- **output** (``T2``): indices.shape + logical_data.shape[1:], with scales' element type.

**Attributes**

- **bits** (``INT``): Packed bit width: 2, 4, or 8. (optional, default: ``4``)
- **block_size** (``INT``): Power of two, at least 16. (optional, default: ``128``)
- **gather_axis** (``INT``): Only normalized axis zero is supported. (optional, default: ``0``)
- **quantize_axis** (``INT``): Must normalize to the last data axis. (optional, default: ``1``)

**Type constraints**

- **T1**: Packed unsigned data and zero points. Allowed types: tensor(uint8).
- **Tind**: Gather indices. Allowed types: tensor(int32), tensor(int64).
- **T2**: Scale and output types. Allowed types: tensor(float), tensor(float16).