LayerNormalization#

LayerNormalization - 1#

Version

  • name: LayerNormalization (GitHub)

  • domain: main

  • since_version: 1

  • function:

  • support_level: SupportType.EXPERIMENTAL

  • shape inference: True

This version of the operator has been available since version 1.

Summary

Attributes

  • axis - INT : The first normalization dimension: normalization will be performed along dimensions axis : rank(inputs).

  • epsilon - FLOAT : The epsilon value to use to avoid division by zero.

  • stash_type - INT : type used for stash mean/inv_std_var

Inputs

Between 2 and 3 inputs.

  • X (heterogeneous) - T:

  • Scale (heterogeneous) - V:

  • B (optional, heterogeneous) - V:

Outputs

Between 1 and 3 outputs.

  • Y (heterogeneous) - V:

  • Mean (optional, heterogeneous) - U:

  • InvStdDev (optional, heterogeneous) - U:

Type Constraints

  • T in ( tensor(bfloat16), tensor(double), tensor(float), tensor(float16) ): Constrain input X type to float tensors.

  • U in ( tensor(double), tensor(float) ): Type of Mean and InvStdDev tensors.

  • V in ( tensor(bfloat16), tensor(double), tensor(float), tensor(float16) ): Constrain output Y, scale and bias type to float tensors.