LinearAttention (CPU)#

  • Domain: com.microsoft

  • Device: CPU

  • Kernel: onnx_light_cpu::MicrosoftLinearAttention

  • Supported types: FLOAT

  • Opset bounds: since opset 1

Operator support#

Capability

Implementation

Shape inference

onnx_light_cpu::ComputeShapeLinearAttention()

Peak memory

onnx_light_cpu::ComputePeakMemoryLinearAttention()

Fusion patterns

onnx_light_cpu::LinearAttentionFusionPattern

Gradient

Not provided

LightOpSchema#

Version: 1

Recurrent linear attention with optional decay, delta updates, and persistent state.

Inputs

  • query (T): Query tensor with shape (batch, sequence, q_num_heads * key_head_size).

  • key (T): Key tensor with shape (batch, sequence, key_heads * key_head_size).

  • value (T): Value tensor with shape (batch, sequence, kv_num_heads * value_head_size).

  • past_state (T): Optional state tensor with shape (batch, kv_num_heads, key_head_size, value_head_size).

  • decay (T): Optional log-space decay tensor used by gated update rules.

  • beta (T): Optional update-rate tensor used by delta update rules.

Outputs

  • output (T): Output tensor with shape (batch, sequence, max(q_num_heads, kv_num_heads) * value_head_size).

  • present_state (T): Updated state with shape (batch, kv_num_heads, key_head_size, value_head_size).

Attributes

  • chunk_size (INT): CPU scheduling hint with no semantic effect. (optional, default: 64)

  • kv_num_heads (INT): Number of value/state heads. (required)

  • q_num_heads (INT): Number of query heads. (required)

  • scale (FLOAT): Query readout scale; zero selects 1/sqrt(key_head_size). (optional, default: 0)

  • state_window (INT): State window; the CPU implementation supports only zero. (optional, default: 0)

  • update_rule (STRING): linear, gated, delta, or gated_delta. (optional, default: gated_delta)

Type constraints

  • T: Constrain the CPU implementation to FLOAT tensors. Allowed types: tensor(float).