Dropout - 6 vs 13#

Next section compares an older to a newer version of the same operator after both definition are converted into markdown text. Green means an addition to the newer version, red means a deletion. Anything else is unchanged.

Files changed (1) hide show
  1. Dropout6 → Dropout13 +13 -44
Dropout6 → Dropout13 RENAMED
@@ -1 +1 @@
1
+ Dropout takes one input data (Tensor<float>) and produces two Tensor outputs,
2
+ output (Tensor<float>) and mask (Tensor<bool>). Depending on whether it is in
3
+ test mode or not, the output Y will either be a random dropout, or a simple
4
+ copy of the input. Note that our implementation of Dropout does scaling in
5
+ the training phase, so during testing nothing needs to be done.
1
- Dropout takes an input floating-point tensor, an optional input ratio (floating-point scalar) and an optional input training_mode (boolean scalar). It produces two tensor outputs,
2
- output (floating-point tensor) and mask (optional Tensor<bool>). If training_mode is true then the output Y will be a random dropout;
3
- Note that this Dropout scales the masked input data by the following equation, so to convert the trained model into inference mode,
4
- the user can simply not pass training_mode input or set it to false.
5
- ::
6
-
7
- output = scale * data * mask,
8
-
9
- where
10
- ::
11
-
12
- scale = 1. / (1. - ratio).
13
-
14
- This operator has **optional** inputs/outputs. See ONNX <https://github.com/onnx/onnx/blob/master/docs/IR.md>_ for more details about the representation of optional arguments. An empty string may be used in the place of an actual argument's name to indicate a missing argument. Trailing optional arguments (those not followed by an argument that is present) may also be simply omitted.
15
6
  **Attributes**
16
- * **seed**:
7
+ * **is_test**:
17
- (Optional) Seed to the random generator, if not specified we will
8
+ (int, default 0) if nonzero, run dropout in test mode where the
9
+ output is simply Y = X.
18
- auto generate one.
10
+ * **ratio**:
11
+ (float, default 0.5) the ratio of random dropout
19
12
  **Inputs**
20
- Between 1 and 3 inputs.
21
-
22
13
  * **data** (heterogeneous) - **T**:
23
14
  The input data as Tensor.
24
- * **ratio** (optional, heterogeneous) - **T1**:
25
- The ratio of random dropout, with value in [0, 1). If this input was
26
- not set, or if it was set to 0, the output would be a simple copy of
27
- the input. If it's non-zero, output will be a random dropout of the
28
- scaled input, which is typically the case during training. It is an
29
- optional value, if not specified it will default to 0.5.
30
- * **training_mode** (optional, heterogeneous) - **T2**:
31
- If set to true then it indicates dropout is being used for training.
32
- It is an optional value hence unless specified explicitly, it is
33
- false. If it is false, ratio is ignored and the operation mimics
34
- inference mode where nothing will be dropped from the input data and
35
- if mask is requested as output it will contain all ones.
36
15
  **Outputs**
37
16
  Between 1 and 2 outputs.
38
17
  * **output** (heterogeneous) - **T**:
39
18
  The output.
40
- * **mask** (optional, heterogeneous) - **T2**:
19
+ * **mask** (optional, heterogeneous) - **T**:
41
- The output mask.
20
+ The output mask. If is_test is nonzero, this output is not filled.
42
21
  **Type Constraints**
43
22
  * **T** in (
44
- tensor(bfloat16),
45
23
  tensor(double),
46
24
  tensor(float),
47
25
  tensor(float16)
48
26
  ):
49
- Constrain input and output types to float tensors.
27
+ Constrain input and output types to float tensors.- * **T1** in (
50
- tensor(double),
51
- tensor(float),
52
- tensor(float16)
53
- ):
54
- Constrain input 'ratio' types to float tensors.
55
- * **T2** in (
56
- tensor(bool)
57
- ):
58
- Constrain output 'mask' types to boolean tensors.