BatchNormalization - 7 vs 9

Files changed (1) hide show
  1. BatchNormalization7 → BatchNormalization9 +15 -22
BatchNormalization7 → BatchNormalization9 RENAMED
@@ -1 +1 @@
1
1
  Carries out batch normalization as described in the paper
2
2
  https://arxiv.org/abs/1502.03167. Depending on the mode it is being run,
3
3
  there are multiple cases for the number of outputs, which we list below:
4
4
  Output case #1: Y, mean, var, saved_mean, saved_var (training mode)
5
5
  Output case #2: Y (test mode)
6
+
7
+ For previous (depreciated) non-spatial cases, implementors are suggested
8
+ to flatten the input shape to (N x C*D1*D2 ..*Dn) before a BatchNormalization Op.
6
- This operator has **optional** inputs/outputs. See ONNX <https://github.com/onnx/onnx/blob/master/docs/IR.md>_ for more details about the representation of optional arguments. An empty string may be used in the place of an actual argument's name to indicate a missing argument. Trailing optional arguments (those not followed by an argument that is present) may also be simply omitted.
9
+ This operator has **optional** inputs/outputs. See ONNX <https://github.com/onnx/onnx/blob/master/docs/IR.md>_ for more details about the representation of optional arguments. An empty string may be used in the place of an actual argument's name to indicate a missing argument. Trailing optional arguments (those not followed by an argument that is present) may also be simply omitted.
7
10
  **Attributes**
8
11
  * **epsilon**:
9
12
  The epsilon value to use to avoid division by zero.
10
13
  * **momentum**:
11
14
  Factor used in computing the running mean and variance.e.g.,
12
15
  running_mean = running_mean * momentum + mean * (1 - momentum).
13
- * **spatial**:
14
- If true, compute the mean and variance across per activation. If
15
- false, compute the mean and variance across per feature over each
16
- mini-batch.
17
16
  **Inputs**
18
17
  * **X** (heterogeneous) - **T**:
19
- Input data tensor from the previous operator; dimensions for image
18
+ Input data tensor from the previous operator; dimensions are in the
20
- case are (N x C x H x W), where N is the batch size, C is the number
19
+ form of (N x C x D1 x D2 ... Dn), where N is the batch size, C is
21
- of channels, and H and W are the height and the width of the data.
20
+ the number of channels. Statistics are computed for every channel of
22
- For non image case, the dimensions are in the form of (N x C x D1 x
21
+ C over N and D1 to Dn dimensions. For image data, input dimensions
22
+ become (N x C x H x W). The op also accepts single dimension input
23
- D2 ... Dn), where N is the batch size.
23
+ of size N in which case C is assumed to be 1
24
24
  * **scale** (heterogeneous) - **T**:
25
+ Scale tensor of shape (C).
25
- If spatial is true, the dimension of scale is (C). If spatial is
26
- false, the dimensions of scale are (C x D1 x ... x Dn)
27
26
  * **B** (heterogeneous) - **T**:
27
+ Bias tensor of shape (C).
28
- If spatial is true, the dimension of bias is (C). If spatial is
29
- false, the dimensions of bias are (C x D1 x ... x Dn)
30
28
  * **mean** (heterogeneous) - **T**:
29
+ running (training) or estimated (testing) mean tensor of shape (C).
31
- If spatial is true, the dimension of the running mean (training) or
32
- the estimated mean (testing) is (C). If spatial is false, the
33
- dimensions of the running mean (training) or the estimated mean
34
- (testing) are (C x D1 x ... x Dn).
35
30
  * **var** (heterogeneous) - **T**:
31
+ running (training) or estimated (testing) variance tensor of shape
32
+ (C).
36
- If spatial is true, the dimension of the running variance(training)
37
- or the estimated variance (testing) is (C). If spatial is false, the
38
- dimensions of the running variance(training) or the estimated
39
- variance (testing) are (C x D1 x ... x Dn).
40
33
  **Outputs**
41
34
  Between 1 and 5 outputs.
42
35
  * **Y** (heterogeneous) - **T**:
43
36
  The output tensor of the same shape as X
44
37
  * **mean** (optional, heterogeneous) - **T**:
45
38
  The running mean after the BatchNormalization operator.
46
39
  * **var** (optional, heterogeneous) - **T**:
47
40
  The running variance after the BatchNormalization operator.
48
41
  * **saved_mean** (optional, heterogeneous) - **T**:
49
42
  Saved mean used during training to speed up gradient computation.
50
43
  * **saved_var** (optional, heterogeneous) - **T**:
51
44
  Saved variance used during training to speed up gradient
52
45
  computation.
53
46
  **Type Constraints**
54
47
  * **T** in (
55
48
  tensor(double),
56
49
  tensor(float),
57
50
  tensor(float16)
58
51
  ):
59
52
  Constrain input and output types to float tensors.