SimplifiedLayerNormalization

SimplifiedLayerNormalization - 1

Version

• name: SimplifiedLayerNormalization (GitHub)

• domain: main

• since_version: 1

• function:

• support_level: SupportType.EXPERIMENTAL

• shape inference: True

This version of the operator has been available since version 1.

Summary

Attributes

• axis - INT : The first normalization dimension: normalization will be performed along dimensions axis : rank(inputs).

• epsilon - FLOAT : The epsilon value to use to avoid division by zero.

• stash_type - INT : type used for stash mean/inv_std_var

Inputs

• X (heterogeneous) - T:

• scale (heterogeneous) - V:

Outputs

Between 1 and 2 outputs.

• Y (heterogeneous) - V:

• inv_std_var (optional, heterogeneous) - U:

Type Constraints

• T in ( tensor(bfloat16), tensor(double), tensor(float), tensor(float16) ): Constrain input X type to float tensors.

• U in ( tensor(double), tensor(float) ): Constrain mean and inv_std_var to be float tensors.

• V in ( tensor(bfloat16), tensor(double), tensor(float), tensor(float16) ): Constrain output Y and scale type to float tensors.

Examples