LinearAttention (CPU)
=====================

* Domain: ``com.microsoft``
* Device: ``CPU``
* Kernel: ``onnx_light_cpu::MicrosoftLinearAttention``
* Supported types: ``FLOAT``
* Opset bounds: since opset 1

Operator support
----------------

.. list-table::
   :header-rows: 1
   :widths: 30 70

   * - Capability
     - Implementation
   * - Shape inference
     - :cpp:func:`onnx_light_cpu::ComputeShapeLinearAttention`
   * - Peak memory
     - :cpp:func:`onnx_light_cpu::ComputePeakMemoryLinearAttention`
   * - Fusion patterns
     - :cpp:class:`onnx_light_cpu::LinearAttentionFusionPattern`
   * - Gradient
     - Not provided

LightOpSchema
~~~~~~~~~~~~~

**Version:** 1

Recurrent linear attention with optional decay, delta updates, and persistent state.

**Inputs**

- **query** (``T``): Query tensor with shape (batch, sequence, q_num_heads * key_head_size).
- **key** (``T``): Key tensor with shape (batch, sequence, key_heads * key_head_size).
- **value** (``T``): Value tensor with shape (batch, sequence, kv_num_heads * value_head_size).
- **past_state** (``T``): Optional state tensor with shape (batch, kv_num_heads, key_head_size, value_head_size).
- **decay** (``T``): Optional log-space decay tensor used by gated update rules.
- **beta** (``T``): Optional update-rate tensor used by delta update rules.

**Outputs**

- **output** (``T``): Output tensor with shape (batch, sequence, max(q_num_heads, kv_num_heads) * value_head_size).
- **present_state** (``T``): Updated state with shape (batch, kv_num_heads, key_head_size, value_head_size).

**Attributes**

- **chunk_size** (``INT``): CPU scheduling hint with no semantic effect. (optional, default: ``64``)
- **kv_num_heads** (``INT``): Number of value/state heads. (required)
- **q_num_heads** (``INT``): Number of query heads. (required)
- **scale** (``FLOAT``): Query readout scale; zero selects 1/sqrt(key_head_size). (optional, default: ``0``)
- **state_window** (``INT``): State window; the CPU implementation supports only zero. (optional, default: ``0``)
- **update_rule** (``STRING``): linear, gated, delta, or gated_delta. (optional, default: ``gated_delta``)

**Type constraints**

- **T**: Constrain the CPU implementation to FLOAT tensors. Allowed types: tensor(float).