Dropout - 10 vs 12

Files changed (1) hide show
  1. Dropout10 → Dropout12 +37 -9
Dropout10 → Dropout12 RENAMED
@@ -1 +1 @@
1
- Dropout takes one input floating tensor and produces two tensor outputs,
1
+ Dropout takes an input floating-point tensor, an optional input ratio (floating-point scalar) and an optional input training_mode (boolean scalar). It produces two tensor outputs,
2
- output (floating tensor) and mask (Tensor<bool>). Depending on whether it is
2
+ output (floating-point tensor) and mask (optional Tensor<bool>). If training_mode is true then the output Y will be a random dropout;
3
- in test mode or not, the output Y will either be a random dropout, or a simple
3
+ Note that this Dropout scales the masked input data by the following equation, so to convert the trained model into inference mode,
4
- copy of the input. Note that our implementation of Dropout does scaling in
5
- the training phase, so during testing nothing needs to be done.
4
+ the user can simply not pass training_mode input or set it to false.
5
+ ::
6
+
7
+ output = scale * data * mask,
8
+
9
+ where
10
+ ::
11
+
12
+ scale = 1. / (1. - ratio).
13
+
6
14
  This operator has **optional** inputs/outputs. See ONNX <https://github.com/onnx/onnx/blob/master/docs/IR.md>_ for more details about the representation of optional arguments. An empty string may be used in the place of an actual argument's name to indicate a missing argument. Trailing optional arguments (those not followed by an argument that is present) may also be simply omitted.
7
15
  **Attributes**
8
- * **ratio**:
16
+ * **seed**:
17
+ (Optional) Seed to the random generator, if not specified we will
9
- The ratio of random dropout
18
+ auto generate one.
10
19
  **Inputs**
20
+ Between 1 and 3 inputs.
21
+
11
22
  * **data** (heterogeneous) - **T**:
12
23
  The input data as Tensor.
24
+ * **ratio** (optional, heterogeneous) - **T1**:
25
+ The ratio of random dropout, with value in [0, 1). If this input was
26
+ not set, or if it was set to 0, the output would be a simple copy of
27
+ the input. If it's non-zero, output will be a random dropout of the
28
+ scaled input, which is typically the case during training. It is an
29
+ optional value, if not specified it will default to 0.5.
30
+ * **training_mode** (optional, heterogeneous) - **T2**:
31
+ If set to true then it indicates dropout is being used for training.
32
+ It is an optional value hence unless specified explicitly, it is
33
+ false. If it is false, ratio is ignored and the operation mimics
34
+ inference mode where nothing will be dropped from the input data and
35
+ if mask is requested as output it will contain all ones.
13
36
  **Outputs**
14
37
  Between 1 and 2 outputs.
15
38
  * **output** (heterogeneous) - **T**:
16
39
  The output.
17
- * **mask** (optional, heterogeneous) - **T1**:
40
+ * **mask** (optional, heterogeneous) - **T2**:
18
41
  The output mask.
19
42
  **Type Constraints**
20
43
  * **T** in (
21
44
  tensor(double),
22
45
  tensor(float),
23
46
  tensor(float16)
24
47
  ):
25
48
  Constrain input and output types to float tensors.
26
49
  * **T1** in (
50
+ tensor(double),
51
+ tensor(float),
52
+ tensor(float16)
53
+ ):
54
+ Constrain input 'ratio' types to float tensors.
55
+ * **T2** in (
27
56
  tensor(bool)
28
57
  ):
29
- Constrain output mask types to boolean tensors.+ Constrain output 'mask' types to boolean tensors.? + +