SkipSimplifiedLayerNormalization (CPU)#

  • Domain: com.microsoft

  • Device: CPU

  • Kernel: onnx_light_cpu::SkipSimplifiedLayerNormalization

  • Supported types: FLOAT, FLOAT16, BFLOAT16

  • Opset bounds: since opset 1

Operator support#

Capability

Implementation

Shape inference

onnx_light_cpu::ComputeShapeSkipSimplifiedLayerNormalization()

Peak memory

onnx_light_cpu::ComputePeakMemorySkipSimplifiedLayerNormalization()

Fusion patterns

None

Gradient

Not provided

LightOpSchema#

Version: 1

Adds input, skip, and optional bias, then applies last-axis RMS normalization and gamma scaling. Supports rank-2 and rank-3 inputs with matching FLOAT, FLOAT16, or BFLOAT16 types. For rank-3 input, skip may also omit the batch dimension or have batch size one. Arithmetic uses FLOAT without narrowing the residual before normalization. Optional FLOAT statistics contain zero mean and the inverse RMS in slots 1 and 2. Optional output slot 3 contains the residual sum in the input type.

Inputs

  • input (T): Input tensor with shape (S, H) or (B, S, H), with 0 < H <= INT_MAX.

  • skip (T): Residual tensor matching input, or (S, H)/(1, S, H) for rank-3 input.

  • gamma (T): Rank-1 scale tensor with length H.

  • bias (T): Optional rank-1 bias tensor with length H.

Outputs

  • output (T): Normalized result with the input shape and type.

  • mean (U): Optional FLOAT zeros with input shape and final dimension one.

  • inv_std_var (U): Optional FLOAT inverse RMS with input shape and final dimension one.

  • input_skip_bias_sum (T): Optional residual sum with the input shape and type.

Attributes

  • epsilon (FLOAT): Finite nonnegative value added to the last-axis mean square. (optional, default: 1e-12)

Type constraints

  • T: Matching FLOAT, FLOAT16, or BFLOAT16 tensors. Allowed types: tensor(bfloat16), tensor(float), tensor(float16).

  • U: FLOAT saved-statistics tensors. Allowed types: tensor(float).