.. _op_ai_onnx_RNN-14:

RNN - version 14
================

This page documents version **14** of operator **RNN**. See :doc:`RNN` for the latest version (since version 22).

- **Domain**: ``ai.onnx``
- **Since version**: 14

Computes an one-layer simple RNN. This operator is usually supported
via some custom implementation such as CuDNN.

Notations:

* ``X`` - input tensor
* ``i`` - input gate
* ``t`` - time step (t-1 means previous time step)
* ``Wi`` - W parameter weight matrix for input gate
* ``Ri`` - R recurrence weight matrix for input gate
* ``Wbi`` - W parameter bias vector for input gate
* ``Rbi`` - R parameter bias vector for input gate
* ``WBi`` - W parameter weight matrix for backward input gate
* ``RBi`` - R recurrence weight matrix for backward input gate
* ``WBbi`` - WR bias vectors for backward input gate
* ``RBbi`` - RR bias vectors for backward input gate
* ``H`` - Hidden state
* ``num_directions`` - 2 if direction == bidirectional else 1

Activation functions:

* Relu(x)                - max(0, x)
* Tanh(x)                - (1 - e^{-2x})/(1 + e^{-2x})
* Sigmoid(x)             - 1/(1 + e^{-x})

NOTE: Below are optional

* Affine(x)              - alpha\*x + beta
* LeakyRelu(x)           - x if x >= 0 else alpha \* x
* ThresholdedRelu(x)     - x if x >= alpha else 0
* ScaledTanh(x)          - alpha\*Tanh(beta\*x)
* HardSigmoid(x)         - min(max(alpha\*x + beta, 0), 1)
* Elu(x)                 - x if x >= 0 else alpha\*(e^x - 1)
* Softsign(x)            - x/(1 + ``|x|``)
* Softplus(x)            - log(1 + e^x)

Equations (Default: f=Tanh):

* Ht = f(Xt\*(Wi^T) + Ht-1\*(Ri^T) + Wbi + Rbi)

**Inputs**

- **X** (*T*): The input sequences packed (and potentially padded) into one 3-D tensor with the shape of ``[seq_length, batch_size, input_size]``.
- **W** (*T*): The weight tensor for input gate. Concatenation of ``Wi`` and ``WBi`` (if bidirectional). The tensor has shape ``[num_directions, hidden_size, input_size]``.
- **R** (*T*): The recurrence weight tensor. Concatenation of ``Ri`` and ``RBi`` (if bidirectional). The tensor has shape ``[num_directions, hidden_size, hidden_size]``.
- **B** (*T*): The bias tensor for input gate. Concatenation of ``[Wbi, Rbi]`` and ``[WBbi, RBbi]`` (if bidirectional). The tensor has shape ``[num_directions, 2*hidden_size]``. Optional: If not specified - assumed to be 0.
- **sequence_lens** (*T1*): Optional tensor specifying lengths of the sequences in a batch. If not specified - assumed all sequences in the batch to have length ``seq_length``. It has shape ``[batch_size]``.
- **initial_h** (*T*): Optional initial value of the hidden. If not specified - assumed to be 0. It has shape ``[num_directions, batch_size, hidden_size]``.

**Outputs**

- **Y** (*T*): A tensor that concats all the intermediate output values of the hidden. It has shape ``[seq_length, num_directions, batch_size, hidden_size]``.
- **Y_h** (*T*): The last output value of the hidden. It has shape ``[num_directions, batch_size, hidden_size]``.

**Type Constraints**

- **T**: Constrain input and output types to float tensors.
  Allowed types: tensor(double), tensor(float), tensor(float16).
- **T1**: Constrain seq_lens to integer tensor.
  Allowed types: tensor(int32).

Differences with previous version (7)
-------------------------------------

**SchemaDiff**: ``RNN`` (domain ``'ai.onnx'``)

* old version: 7
* new version: 14
* breaking: no