BatchNormalization - version 7#

This page documents version 7 of operator BatchNormalization. See BatchNormalization for the latest version (since version 15).

  • Domain: ai.onnx

  • Since version: 7

Carries out batch normalization as described in the paper https://arxiv.org/abs/1502.03167. Depending on the mode it is being run, there are multiple cases for the number of outputs, which we list below:

Output case #1: Y, mean, var, saved_mean, saved_var (training mode) Output case #2: Y (test mode)

Inputs

  • X (T): Input data tensor from the previous operator; dimensions for image case are (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data. For non image case, the dimensions are in the form of (N x C x D1 x D2 … Dn), where N is the batch size.

  • scale (T): If spatial is true, the dimension of scale is (C). If spatial is false, the dimensions of scale are (C x D1 x … x Dn)

  • B (T): If spatial is true, the dimension of bias is (C). If spatial is false, the dimensions of bias are (C x D1 x … x Dn)

  • mean (T): If spatial is true, the dimension of the running mean (training) or the estimated mean (testing) is (C). If spatial is false, the dimensions of the running mean (training) or the estimated mean (testing) are (C x D1 x … x Dn).

  • var (T): If spatial is true, the dimension of the running variance(training) or the estimated variance (testing) is (C). If spatial is false, the dimensions of the running variance(training) or the estimated variance (testing) are (C x D1 x … x Dn).

Outputs

  • Y (T): The output tensor of the same shape as X

  • mean (T): The running mean after the BatchNormalization operator.

  • var (T): The running variance after the BatchNormalization operator.

  • saved_mean (T): Saved mean used during training to speed up gradient computation.

  • saved_var (T): Saved variance used during training to speed up gradient computation.

Type Constraints

  • T: Constrain input and output types to float tensors. Allowed types: tensor(double), tensor(float), tensor(float16).

Differences with previous version (6)#

SchemaDiff: BatchNormalization (domain 'ai.onnx')

  • old version: 6

  • new version: 7

  • breaking: no

Documentation:

  • line similarity: 0.25 (+6/-6 lines)

--- BatchNormalization v6
+++ BatchNormalization v7
@@ -1,8 +1,8 @@

-Carries out batch normalization as described in the paper
-https://arxiv.org/abs/1502.03167. Depending on the mode it is being run,
-there are multiple cases for the number of outputs, which we list below:
+    Carries out batch normalization as described in the paper
+    https://arxiv.org/abs/1502.03167. Depending on the mode it is being run,
+    there are multiple cases for the number of outputs, which we list below:

-Output case #1: Y, mean, var, saved_mean, saved_var (training mode)
-Output case #2: Y (test mode)
-
+    Output case #1: Y, mean, var, saved_mean, saved_var (training mode)
+    Output case #2: Y (test mode)
+