GatherBlockQuantized (CPU)#

  • Domain: com.microsoft

  • Device: CPU

  • Kernel: onnx_light_cpu::GatherBlockQuantized

  • Supported types: UINT8

  • Opset bounds: since opset 1, until opset 1

Operator support#

Capability

Implementation

Shape inference

onnx_light_cpu::ComputeShapeGatherBlockQuantized()

Peak memory

onnx_light_cpu::ComputePeakMemoryGatherBlockQuantized()

Fusion patterns

None

Gradient

Not provided

LightOpSchema#

Version: 1

Gathers rows from a block-quantized UINT8 tensor of rank at least two and dequantizes them. Only gather_axis=0 and last-axis quantization are supported, including equivalent negative axes. The logical last dimension is the packed last dimension times 8/bits.

Inputs

  • data (T1): UINT8 packed data of rank at least two.

  • indices (Tind): INT32 or INT64 indices of any rank, including scalar.

  • scales (T2): FLOAT or FLOAT16 scales, matching data except the last dimension is ceil(logical_last / block_size).

  • zero_points (T1): Optional UINT8 zero points, matching scales except the last dimension is ceil(scales_last / (8/bits)).

Outputs

  • output (T2): indices.shape + logical_data.shape[1:], with scales’ element type.

Attributes

  • bits (INT): Packed bit width: 2, 4, or 8. (optional, default: 4)

  • block_size (INT): Power of two, at least 16. (optional, default: 128)

  • gather_axis (INT): Only normalized axis zero is supported. (optional, default: 0)

  • quantize_axis (INT): Must normalize to the last data axis. (optional, default: 1)

Type constraints

  • T1: Packed unsigned data and zero points. Allowed types: tensor(uint8).

  • Tind: Gather indices. Allowed types: tensor(int32), tensor(int64).

  • T2: Scale and output types. Allowed types: tensor(float), tensor(float16).