.. _op_ai_onnx_preview_training_Adagrad:

Adagrad
=======

- **Domain**: ``ai.onnx.preview.training``
- **Since version**: 1

Compute one iteration of ADAGRAD, a stochastic gradient based optimization
algorithm. This operator can conduct the optimization of multiple tensor variables.

**Inputs**

- **R** (*T1*): The initial learning rate.
- **T** (*T2*): The update count of "X". It should be a scalar.
- **inputs** (*T3*): The current values of optimized tensors, followed by their respective gradients, followed by their respective accumulated squared gradients.For example, if two tensor "X_1" and "X_2" are optimized, The input list would be ["X_1", "X_2", gradient of "X_1", gradient of "X_2", accumulated squared gradient of "X_1", accumulated squared gradient of "X_2"].

**Outputs**

- **outputs** (*T3*): Updated values of optimized tensors, followed by their updated values of accumulated squared gradients. For example, if two tensor "X_1" and "X_2" are optimized, the output list would be [new value of "X_1," new value of "X_2" new accumulated squared gradient of "X_1", new accumulated squared gradient of "X_2"].

**Type Constraints**

- **T1**: Constrain input types to float scalars.
  Allowed types: tensor(double), tensor(float).
- **T2**: Constrain input types to 64-bit integer scalars.
  Allowed types: tensor(int64).
- **T3**: Constrain input and output types to float tensors.
  Allowed types: tensor(double), tensor(float).

Examples
--------

**test_adagrad**

.. code-block:: text

    Node:
      ai.onnx.preview.training.Adagrad(R, T, X, G, H) -> (X_new, H_new)
      Attributes:
        norm_coefficient = 0.0010000000474974513
        epsilon = 9.999999747378752e-06
        decay_factor = 0.10000000149011612

.. code-block:: text

    Inputs:
      R: shape=(), dtype=float32
        0.1
      T: shape=(), dtype=int64
        0
      X: shape=(1,), dtype=float32
        [1.]
      G: shape=(1,), dtype=float32
        [-1.]
      H: shape=(1,), dtype=float32
        [2.]

    Outputs:
      X_new: shape=(1,), dtype=float32
        [1.0576962]
      H_new: shape=(1,), dtype=float32
        [2.998001]

**test_adagrad_multiple**

.. code-block:: text

    Node:
      ai.onnx.preview.training.Adagrad(R, T, X1, X2, G1, G2, H1, H2) -> (X1_new, X2_new, H1_new, H2_new)
      Attributes:
        norm_coefficient = 0.0010000000474974513
        epsilon = 9.999999747378752e-06
        decay_factor = 0.10000000149011612

.. code-block:: text

    Inputs:
      R: shape=(), dtype=float32
        0.1
      T: shape=(), dtype=int64
        0
      X1: shape=(1,), dtype=float32
        [1.]
      X2: shape=(2,), dtype=float32
        [1., 2.]
      G1: shape=(1,), dtype=float32
        [-1.]
      G2: shape=(2,), dtype=float32
        [-1., -3.]
      H1: shape=(1,), dtype=float32
        [2.]
      H2: shape=(2,), dtype=float32
        [4., 1.]

    Outputs:
      X1_new: shape=(1,), dtype=float32
        [1.0576962]
      X2_new: shape=(2,), dtype=float32
        [1.0446854, 2.0948617]
      H1_new: shape=(1,), dtype=float32
        [2.998001]
      H2_new: shape=(2,), dtype=float32
        [4.998001, 9.988004]