BatchNormalization - version 9#

This page documents version 9 of operator BatchNormalization. See BatchNormalization for the latest version (since version 15).

  • Domain: ai.onnx

  • Since version: 9

Carries out batch normalization as described in the paper https://arxiv.org/abs/1502.03167. Depending on the mode it is being run, there are multiple cases for the number of outputs, which we list below:

Output case #1: Y, mean, var, saved_mean, saved_var (training mode) Output case #2: Y (test mode)

For previous (depreciated) non-spatial cases, implementers are suggested to flatten the input shape to (N x C*D1*D2 ..*Dn) before a BatchNormalization Op.

Inputs

  • X (T): Input data tensor from the previous operator; dimensions are in the form of (N x C x D1 x D2 … Dn), where N is the batch size, C is the number of channels. Statistics are computed for every channel of C over N and D1 to Dn dimensions. For image data, input dimensions become (N x C x H x W). The op also accepts single dimension input of size N in which case C is assumed to be 1

  • scale (T): Scale tensor of shape (C).

  • B (T): Bias tensor of shape (C).

  • mean (T): running (training) or estimated (testing) mean tensor of shape (C).

  • var (T): running (training) or estimated (testing) variance tensor of shape (C).

Outputs

  • Y (T): The output tensor of the same shape as X

  • mean (T): The running mean after the BatchNormalization operator.

  • var (T): The running variance after the BatchNormalization operator.

  • saved_mean (T): Saved mean used during training to speed up gradient computation.

  • saved_var (T): Saved variance used during training to speed up gradient computation.

Type Constraints

  • T: Constrain input and output types to float tensors. Allowed types: tensor(double), tensor(float), tensor(float16).

Differences with previous version (7)#

SchemaDiff: BatchNormalization (domain 'ai.onnx')

  • old version: 7

  • new version: 9

  • breaking: no

Documentation:

  • line similarity: 0.22 (+8/-6 lines)

--- BatchNormalization v7
+++ BatchNormalization v9
@@ -1,8 +1,10 @@

-    Carries out batch normalization as described in the paper
-    https://arxiv.org/abs/1502.03167. Depending on the mode it is being run,
-    there are multiple cases for the number of outputs, which we list below:
+Carries out batch normalization as described in the paper
+https://arxiv.org/abs/1502.03167. Depending on the mode it is being run,
+there are multiple cases for the number of outputs, which we list below:

-    Output case #1: Y, mean, var, saved_mean, saved_var (training mode)
-    Output case #2: Y (test mode)
-
+Output case #1: Y, mean, var, saved_mean, saved_var (training mode)
+Output case #2: Y (test mode)
+
+For previous (depreciated) non-spatial cases, implementers are suggested
+to flatten the input shape to (N x C*D1*D2 ..*Dn) before a BatchNormalization Op.