.. _op_ai_onnx_RMSNormalization: RMSNormalization ================ - **Domain**: ``ai.onnx`` - **Since version**: 23 This is RMS normalization defined in ONNX as function as described in the paper https://arxiv.org/pdf/1910.07467. The overall computation can be split into two stages. The root mean squared norm is taken over the last D dimensions, where D is the dimension of normalized_shape. For example, if normalized_shape is (3, 5) (a 2-dimensional shape), the rms norm is computed over the last 2 dimensions of the input. The computation required by standardization can be described by the following equations. .. code-block:: XSquared = Mul(X, X) XSquaredMean = ReduceMean (XSquared) MeanSquareEpsilon = Add(XSquaredMean, epsilon) RMS = Sqrt(MeanSquareEpsilon) Normalized = Div(X, RMS) where ``normalized_axes`` is ``[axis, ..., rank of X - 1]``. The variables ``RMS`` stand for root mean square, Depending on ``stash_type`` attribute, the actual computation must happen in different floating-point precision. For example, if ``stash_type`` is 1, this operator casts all input variables to 32-bit float, perform the computation, and finally cast ``Normalized`` back to the original type of ``X``. The second stage then scales the outcome of the first stage using: .. code-block:: Y= Mul(Normalized, Scale) Let ``d[i]`` indicate the i-th dimension of ``X``. If ``X``'s shape is ``[d[0], ..., d[axis-1], d[axis], ..., d[rank-1]]``, the shape of ``RMS`` is ``[d[0], ..., d[axis-1], 1, ..., 1]``. ``Y`` and ``X`` have the same shape. This operator supports unidirectional broadcasting (``Scale`` should be unidirectional broadcastable to tensor ``X``); for more details please check `the doc `_. **Inputs** - **X** (*T*): The input tensor to be normalized. In general, the shape is (D1, D2, ... , Dn) for n-dimensional data, where the root mean squared norm is taken over the last D dimensions, D is determined by the axis attribute. - **scale** (*V*): Scale tensor. Scale tensor shape should be broadcastable to the normalized shape. **Outputs** - **Y** (*V*): Output data tensor. Same shape as X **Attributes** - **axis** (*int*): The first normalization dimension. If rank(X) is r, axis' allowed range is [-r, r). Negative value means counting dimensions from the back. - **epsilon** (*float*): The epsilon value to use to avoid division by zero. - **stash_type** (*int*): The floating-point precision used in stage one of the computation. **Type Constraints** - **T**: Constrain input X type to float tensors. Allowed types: tensor(bfloat16), tensor(double), tensor(float), tensor(float16). - **V**: Constrain output Y and scale type to float tensors. Allowed types: tensor(bfloat16), tensor(double), tensor(float), tensor(float16). Examples -------- **test_cc_rms_normalization_2d_axis0** .. code-block:: text Node: RMSNormalization(x, scale) -> (y) Attributes: axis = 0 .. code-block:: text Inputs: x: shape=(3, 4), dtype=float32 [[-0.5 , -0.45 , -0.4 , -0.35 ], [-0.3 , -0.25 , -0.19999999, -0.15 ], [-0.09999999, -0.04999998, 0. , 0.05000001]] scale: shape=(3, 4), dtype=float32 [[0.5 , 0.52 , 0.54 , 0.56 ], [0.58 , 0.6 , 0.62 , 0.64 ], [0.65999997, 0.68 , 0.7 , 0.72 ]] Outputs: y: shape=(3, 4), dtype=float32 [[-0.8815359 , -0.8251175 , -0.7616471 , -0.6911241 ], [-0.613549 , -0.52892154, -0.4372418 , -0.33850977], [-0.23272546, -0.11988883, 0. , 0.1269412 ]] **test_cc_rms_normalization_2d_axis1** .. code-block:: text Node: RMSNormalization(x, scale) -> (y) Attributes: axis = 1 .. code-block:: text Inputs: x: shape=(3, 4), dtype=float32 [[-0.5 , -0.45 , -0.4 , -0.35 ], [-0.3 , -0.25 , -0.19999999, -0.15 ], [-0.09999999, -0.04999998, 0. , 0.05000001]] scale: shape=(4,), dtype=float32 [0.5 , 0.52, 0.54, 0.56] Outputs: y: shape=(3, 4), dtype=float32 [[-0.583196 , -0.5458714 , -0.50388134, -0.45722565], [-0.6469365 , -0.56067824, -0.4657942 , -0.36228442], [-0.8154101 , -0.4240131 , 0. , 0.45662978]] **test_cc_rms_normalization_2d_axis_negative_1** .. code-block:: text Node: RMSNormalization(x, scale) -> (y) Attributes: axis = -1 .. code-block:: text Inputs: x: shape=(3, 4), dtype=float32 [[-0.5 , -0.45 , -0.4 , -0.35 ], [-0.3 , -0.25 , -0.19999999, -0.15 ], [-0.09999999, -0.04999998, 0. , 0.05000001]] scale: shape=(4,), dtype=float32 [0.5 , 0.52, 0.54, 0.56] Outputs: y: shape=(3, 4), dtype=float32 [[-0.583196 , -0.5458714 , -0.50388134, -0.45722565], [-0.6469365 , -0.56067824, -0.4657942 , -0.36228442], [-0.8154101 , -0.4240131 , 0. , 0.45662978]] **test_cc_rms_normalization_2d_axis_negative_2** .. code-block:: text Node: RMSNormalization(x, scale) -> (y) Attributes: axis = -2 .. code-block:: text Inputs: x: shape=(3, 4), dtype=float32 [[-0.5 , -0.45 , -0.4 , -0.35 ], [-0.3 , -0.25 , -0.19999999, -0.15 ], [-0.09999999, -0.04999998, 0. , 0.05000001]] scale: shape=(3, 4), dtype=float32 [[0.5 , 0.52 , 0.54 , 0.56 ], [0.58 , 0.6 , 0.62 , 0.64 ], [0.65999997, 0.68 , 0.7 , 0.72 ]] Outputs: y: shape=(3, 4), dtype=float32 [[-0.8815359 , -0.8251175 , -0.7616471 , -0.6911241 ], [-0.613549 , -0.52892154, -0.4372418 , -0.33850977], [-0.23272546, -0.11988883, 0. , 0.1269412 ]] **test_cc_rms_normalization_3d_axis0_epsilon** .. code-block:: text Node: RMSNormalization(x, scale) -> (y) Attributes: axis = 0 epsilon = 0.10000000149011612 .. code-block:: text Inputs: x: shape=(2, 3, 5), dtype=float32 [[[-0.5 , -0.45 , -0.4 , -0.35 , -0.3 ], [-0.25 , -0.19999999, -0.15 , -0.09999999, -0.04999998], [ 0. , 0.05000001, 0.10000002, 0.15000004, 0.19999999]], [[ 0.25 , 0.3 , 0.35000002, 0.40000004, 0.45 ], [ 0.5 , 0.5500001 , 0.6 , 0.65 , 0.70000005], [ 0.75 , 0.8000001 , 0.85 , 0.9 , 0.95000005]]] scale: shape=(2, 3, 5), dtype=float32 [[[0.5 , 0.52 , 0.54 , 0.56 , 0.58 ], [0.6 , 0.62 , 0.64 , 0.65999997, 0.68 ], [0.7 , 0.72 , 0.74 , 0.76 , 0.78 ]], [[0.79999995, 0.82 , 0.84000003, 0.86 , 0.88 ], [0.9 , 0.91999996, 0.94 , 0.96 , 0.98 ], [1. , 1.02 , 1.04 , 1.06 , 1.0799999 ]]] Outputs: y: shape=(2, 3, 5), dtype=float32 [[[-0.4300661 , -0.40254185, -0.3715771 , -0.33717182, -0.29932603], [-0.25803968, -0.21331279, -0.1651454 , -0.11353745, -0.05848897], [ 0. , 0.06192954, 0.1272996 , 0.19611017, 0.26836124]], [[ 0.34405288, 0.42318508, 0.50575775, 0.591771 , 0.6812247 ], [ 0.77411896, 0.87045383, 0.9702292 , 1.073445 , 1.1801014 ], [ 1.2901983 , 1.4037359 , 1.5207138 , 1.6411321 , 1.7649912 ]]] **test_cc_rms_normalization_3d_axis1_epsilon** .. code-block:: text Node: RMSNormalization(x, scale) -> (y) Attributes: axis = 1 epsilon = 0.10000000149011612 .. code-block:: text Inputs: x: shape=(2, 3, 5), dtype=float32 [[[-0.5 , -0.45 , -0.4 , -0.35 , -0.3 ], [-0.25 , -0.19999999, -0.15 , -0.09999999, -0.04999998], [ 0. , 0.05000001, 0.10000002, 0.15000004, 0.19999999]], [[ 0.25 , 0.3 , 0.35000002, 0.40000004, 0.45 ], [ 0.5 , 0.5500001 , 0.6 , 0.65 , 0.70000005], [ 0.75 , 0.8000001 , 0.85 , 0.9 , 0.95000005]]] scale: shape=(3, 5), dtype=float32 [[0.5 , 0.52 , 0.54 , 0.56 , 0.58 ], [0.6 , 0.62 , 0.64 , 0.65999997, 0.68 ], [0.7 , 0.72 , 0.74 , 0.76 , 0.78 ]] Outputs: y: shape=(2, 3, 5), dtype=float32 [[[-0.6078307 , -0.5689295 , -0.52516574, -0.47653928, -0.42305017], [-0.36469844, -0.30148402, -0.23340699, -0.16046728, -0.08266494], [ 0. , 0.08752765, 0.17991793, 0.27717087, 0.37928632]], [[ 0.17560984, 0.21916108, 0.26552212, 0.31469285, 0.36667332], [ 0.42146364, 0.4790637 , 0.5394735 , 0.60269296, 0.66872233], [ 0.73756135, 0.80921024, 0.8836688 , 0.960937 , 1.0410151 ]]] **test_cc_rms_normalization_3d_axis2_epsilon** .. code-block:: text Node: RMSNormalization(x, scale) -> (y) Attributes: axis = 2 epsilon = 0.10000000149011612 .. code-block:: text Inputs: x: shape=(2, 3, 5), dtype=float32 [[[-0.5 , -0.45 , -0.4 , -0.35 , -0.3 ], [-0.25 , -0.19999999, -0.15 , -0.09999999, -0.04999998], [ 0. , 0.05000001, 0.10000002, 0.15000004, 0.19999999]], [[ 0.25 , 0.3 , 0.35000002, 0.40000004, 0.45 ], [ 0.5 , 0.5500001 , 0.6 , 0.65 , 0.70000005], [ 0.75 , 0.8000001 , 0.85 , 0.9 , 0.95000005]]] scale: shape=(5,), dtype=float32 [0.5 , 0.52, 0.54, 0.56, 0.58] Outputs: y: shape=(2, 3, 5), dtype=float32 [[[-0.4856429 , -0.4545617 , -0.4195955 , -0.38074404, -0.33800748], [-0.35007 , -0.2912582 , -0.22684538, -0.15683134, -0.08121621], [ 0. , 0.07666983, 0.15923735, 0.24770254, 0.34206527]], [[ 0.2620712 , 0.32706484, 0.39625168, 0.4696316 , 0.5472046 ], [ 0.3666178 , 0.4194108 , 0.4751367 , 0.5337955 , 0.59538734], [ 0.4122373 , 0.4573086 , 0.5045785 , 0.55404687, 0.605714 ]]] **test_cc_rms_normalization_3d_axis_negative_1_epsilon** .. code-block:: text Node: RMSNormalization(x, scale) -> (y) Attributes: axis = -1 epsilon = 0.10000000149011612 .. code-block:: text Inputs: x: shape=(2, 3, 5), dtype=float32 [[[-0.5 , -0.45 , -0.4 , -0.35 , -0.3 ], [-0.25 , -0.19999999, -0.15 , -0.09999999, -0.04999998], [ 0. , 0.05000001, 0.10000002, 0.15000004, 0.19999999]], [[ 0.25 , 0.3 , 0.35000002, 0.40000004, 0.45 ], [ 0.5 , 0.5500001 , 0.6 , 0.65 , 0.70000005], [ 0.75 , 0.8000001 , 0.85 , 0.9 , 0.95000005]]] scale: shape=(5,), dtype=float32 [0.5 , 0.52, 0.54, 0.56, 0.58] Outputs: y: shape=(2, 3, 5), dtype=float32 [[[-0.4856429 , -0.4545617 , -0.4195955 , -0.38074404, -0.33800748], [-0.35007 , -0.2912582 , -0.22684538, -0.15683134, -0.08121621], [ 0. , 0.07666983, 0.15923735, 0.24770254, 0.34206527]], [[ 0.2620712 , 0.32706484, 0.39625168, 0.4696316 , 0.5472046 ], [ 0.3666178 , 0.4194108 , 0.4751367 , 0.5337955 , 0.59538734], [ 0.4122373 , 0.4573086 , 0.5045785 , 0.55404687, 0.605714 ]]] **test_cc_rms_normalization_3d_axis_negative_2_epsilon** .. code-block:: text Node: RMSNormalization(x, scale) -> (y) Attributes: axis = -2 epsilon = 0.10000000149011612 .. code-block:: text Inputs: x: shape=(2, 3, 5), dtype=float32 [[[-0.5 , -0.45 , -0.4 , -0.35 , -0.3 ], [-0.25 , -0.19999999, -0.15 , -0.09999999, -0.04999998], [ 0. , 0.05000001, 0.10000002, 0.15000004, 0.19999999]], [[ 0.25 , 0.3 , 0.35000002, 0.40000004, 0.45 ], [ 0.5 , 0.5500001 , 0.6 , 0.65 , 0.70000005], [ 0.75 , 0.8000001 , 0.85 , 0.9 , 0.95000005]]] scale: shape=(3, 5), dtype=float32 [[0.5 , 0.52 , 0.54 , 0.56 , 0.58 ], [0.6 , 0.62 , 0.64 , 0.65999997, 0.68 ], [0.7 , 0.72 , 0.74 , 0.76 , 0.78 ]] Outputs: y: shape=(2, 3, 5), dtype=float32 [[[-0.6078307 , -0.5689295 , -0.52516574, -0.47653928, -0.42305017], [-0.36469844, -0.30148402, -0.23340699, -0.16046728, -0.08266494], [ 0. , 0.08752765, 0.17991793, 0.27717087, 0.37928632]], [[ 0.17560984, 0.21916108, 0.26552212, 0.31469285, 0.36667332], [ 0.42146364, 0.4790637 , 0.5394735 , 0.60269296, 0.66872233], [ 0.73756135, 0.80921024, 0.8836688 , 0.960937 , 1.0410151 ]]] **test_cc_rms_normalization_3d_axis_negative_3_epsilon** .. code-block:: text Node: RMSNormalization(x, scale) -> (y) Attributes: axis = -3 epsilon = 0.10000000149011612 .. code-block:: text Inputs: x: shape=(2, 3, 5), dtype=float32 [[[-0.5 , -0.45 , -0.4 , -0.35 , -0.3 ], [-0.25 , -0.19999999, -0.15 , -0.09999999, -0.04999998], [ 0. , 0.05000001, 0.10000002, 0.15000004, 0.19999999]], [[ 0.25 , 0.3 , 0.35000002, 0.40000004, 0.45 ], [ 0.5 , 0.5500001 , 0.6 , 0.65 , 0.70000005], [ 0.75 , 0.8000001 , 0.85 , 0.9 , 0.95000005]]] scale: shape=(2, 3, 5), dtype=float32 [[[0.5 , 0.52 , 0.54 , 0.56 , 0.58 ], [0.6 , 0.62 , 0.64 , 0.65999997, 0.68 ], [0.7 , 0.72 , 0.74 , 0.76 , 0.78 ]], [[0.79999995, 0.82 , 0.84000003, 0.86 , 0.88 ], [0.9 , 0.91999996, 0.94 , 0.96 , 0.98 ], [1. , 1.02 , 1.04 , 1.06 , 1.0799999 ]]] Outputs: y: shape=(2, 3, 5), dtype=float32 [[[-0.4300661 , -0.40254185, -0.3715771 , -0.33717182, -0.29932603], [-0.25803968, -0.21331279, -0.1651454 , -0.11353745, -0.05848897], [ 0. , 0.06192954, 0.1272996 , 0.19611017, 0.26836124]], [[ 0.34405288, 0.42318508, 0.50575775, 0.591771 , 0.6812247 ], [ 0.77411896, 0.87045383, 0.9702292 , 1.073445 , 1.1801014 ], [ 1.2901983 , 1.4037359 , 1.5207138 , 1.6411321 , 1.7649912 ]]] **test_cc_rms_normalization_4d_axis0** .. code-block:: text Node: RMSNormalization(x, scale) -> (y) Attributes: axis = 0 .. code-block:: text Inputs: x: shape=(2, 3, 4, 5), dtype=float32 [[[[-0.5 , -0.45 , -0.4 , -0.35 , -0.3 ], [-0.25 , -0.19999999, -0.15 , -0.09999999, -0.04999998], [ 0. , 0.05000001, 0.10000002, 0.15000004, 0.19999999], [ 0.25 , 0.3 , 0.35000002, 0.40000004, 0.45 ]], [[ 0.5 , 0.5500001 , 0.6 , 0.65 , 0.70000005], [ 0.75 , 0.8000001 , 0.85 , 0.9 , 0.95000005], [ 1. , 1.0500001 , 1.1 , 1.15 , 1.2 ], [ 1.25 , 1.3000001 , 1.35 , 1.4 , 1.45 ]], [[ 1.5 , 1.55 , 1.6000001 , 1.6500001 , 1.7 ], [ 1.75 , 1.8 , 1.8500001 , 1.9000001 , 1.95 ], [ 2. , 2.05 , 2.1000001 , 2.15 , 2.2 ], [ 2.25 , 2.3 , 2.3500001 , 2.4 , 2.45 ]]], [[[ 2.5 , 2.55 , 2.6000001 , 2.65 , 2.7 ], [ 2.75 , 2.8 , 2.8500001 , 2.9 , 2.95 ], [ 3. , 3.05 , 3.1000001 , 3.15 , 3.2 ], [ 3.25 , 3.3 , 3.3500001 , 3.4 , 3.45 ]], [[ 3.5 , 3.5500002 , 3.6 , 3.65 , 3.7000003 ], [ 3.75 , 3.8000002 , 3.85 , 3.9 , 3.9500003 ], [ 4. , 4.05 , 4.1 , 4.15 , 4.2000003 ], [ 4.25 , 4.3 , 4.35 , 4.4 , 4.4500003 ]], [[ 4.5 , 4.55 , 4.6 , 4.65 , 4.7000003 ], [ 4.75 , 4.8 , 4.85 , 4.9 , 4.9500003 ], [ 5. , 5.05 , 5.1 , 5.15 , 5.2000003 ], [ 5.25 , 5.3 , 5.35 , 5.4 , 5.4500003 ]]]] scale: shape=(2, 3, 4, 5), dtype=float32 [[[[0.5 , 0.52 , 0.54 , 0.56 , 0.58 ], [0.6 , 0.62 , 0.64 , 0.65999997, 0.68 ], [0.7 , 0.72 , 0.74 , 0.76 , 0.78 ], [0.79999995, 0.82 , 0.84000003, 0.86 , 0.88 ]], [[0.9 , 0.91999996, 0.94 , 0.96 , 0.98 ], [1. , 1.02 , 1.04 , 1.06 , 1.0799999 ], [1.0999999 , 1.12 , 1.14 , 1.16 , 1.1800001 ], [1.2 , 1.22 , 1.24 , 1.26 , 1.28 ]], [[1.3 , 1.3199999 , 1.3399999 , 1.3599999 , 1.38 ], [1.4 , 1.42 , 1.44 , 1.46 , 1.48 ], [1.5 , 1.52 , 1.54 , 1.56 , 1.5799999 ], [1.6 , 1.62 , 1.64 , 1.66 , 1.68 ]]], [[[1.6999999 , 1.72 , 1.74 , 1.76 , 1.78 ], [1.8 , 1.8199999 , 1.8399999 , 1.86 , 1.88 ], [1.9 , 1.92 , 1.9399999 , 1.9599999 , 1.98 ], [2. , 2.02 , 2.04 , 2.06 , 2.08 ]], [[2.1 , 2.12 , 2.1399999 , 2.1599998 , 2.1799998 ], [2.1999998 , 2.2199998 , 2.24 , 2.26 , 2.28 ], [2.3 , 2.32 , 2.34 , 2.3600001 , 2.38 ], [2.4 , 2.42 , 2.44 , 2.46 , 2.48 ]], [[2.5 , 2.52 , 2.54 , 2.56 , 2.58 ], [2.6 , 2.62 , 2.6399999 , 2.6599998 , 2.68 ], [2.7 , 2.72 , 2.74 , 2.76 , 2.78 ], [2.8 , 2.82 , 2.84 , 2.86 , 2.8799999 ]]]] Outputs: y: shape=(2, 3, 4, 5), dtype=float32 [[[[-0.08275866, -0.0774621 , -0.07150348, -0.06488279, -0.05760003], [-0.0496552 , -0.04104829, -0.03177932, -0.02184828, -0.01125517], [ 0. , 0.01191725, 0.02449657, 0.03773796, 0.0516414 ], [ 0.06620692, 0.08143452, 0.09732419, 0.11387593, 0.13108972]], [[ 0.14896558, 0.16750354, 0.18670353, 0.20656559, 0.22708978], [ 0.24827597, 0.2701243 , 0.2926346 , 0.315807 , 0.33964154], [ 0.36413807, 0.38929677, 0.4151174 , 0.4416002 , 0.46874508], [ 0.49655196, 0.52502096, 0.554152 , 0.58394504, 0.61440027]], [[ 0.64551747, 0.6772968 , 0.70973825, 0.7428417 , 0.7766072 ], [ 0.8110348 , 0.8461245 , 0.88187635, 0.9182902 , 0.95536596], [ 0.99310386, 1.0315039 , 1.070566 , 1.1102902 , 1.1506764 ], [ 1.1917247 , 1.233435 , 1.2758076 , 1.3188419 , 1.3625386 ]]], [[[ 1.4068972 , 1.4519179 , 1.4976007 , 1.5439456 , 1.5909524 ], [ 1.6386214 , 1.6869524 , 1.7359456 , 1.7856009 , 1.8359181 ], [ 1.8868973 , 1.9385387 , 1.9908423 , 2.0438077 , 2.0974355 ], [ 2.151725 , 2.2066767 , 2.2622907 , 2.3185666 , 2.3755045 ]], [[ 2.4331043 , 2.4913666 , 2.5502906 , 2.6098769 , 2.6701252 ], [ 2.7310355 , 2.792608 , 2.8548427 , 2.9177392 , 2.9812982 ], [ 3.0455186 , 3.1104016 , 3.1759462 , 3.2421534 , 3.3090227 ], [ 3.3765533 , 3.4447465 , 3.5136018 , 3.583119 , 3.6532986 ]], [[ 3.7241397 , 3.795643 , 3.8678086 , 3.9406362 , 4.0141263 ], [ 4.088278 , 4.163091 , 4.2385674 , 4.3147054 , 4.3915057 ], [ 4.4689674 , 4.547092 , 4.625878 , 4.705326 , 4.7854366 ], [ 4.866209 , 4.9476438 , 5.02974 , 5.1124988 , 5.1959195 ]]]] **test_cc_rms_normalization_4d_axis1** .. code-block:: text Node: RMSNormalization(x, scale) -> (y) Attributes: axis = 1 .. code-block:: text Inputs: x: shape=(2, 3, 4, 5), dtype=float32 [[[[-0.5 , -0.45 , -0.4 , -0.35 , -0.3 ], [-0.25 , -0.19999999, -0.15 , -0.09999999, -0.04999998], [ 0. , 0.05000001, 0.10000002, 0.15000004, 0.19999999], [ 0.25 , 0.3 , 0.35000002, 0.40000004, 0.45 ]], [[ 0.5 , 0.5500001 , 0.6 , 0.65 , 0.70000005], [ 0.75 , 0.8000001 , 0.85 , 0.9 , 0.95000005], [ 1. , 1.0500001 , 1.1 , 1.15 , 1.2 ], [ 1.25 , 1.3000001 , 1.35 , 1.4 , 1.45 ]], [[ 1.5 , 1.55 , 1.6000001 , 1.6500001 , 1.7 ], [ 1.75 , 1.8 , 1.8500001 , 1.9000001 , 1.95 ], [ 2. , 2.05 , 2.1000001 , 2.15 , 2.2 ], [ 2.25 , 2.3 , 2.3500001 , 2.4 , 2.45 ]]], [[[ 2.5 , 2.55 , 2.6000001 , 2.65 , 2.7 ], [ 2.75 , 2.8 , 2.8500001 , 2.9 , 2.95 ], [ 3. , 3.05 , 3.1000001 , 3.15 , 3.2 ], [ 3.25 , 3.3 , 3.3500001 , 3.4 , 3.45 ]], [[ 3.5 , 3.5500002 , 3.6 , 3.65 , 3.7000003 ], [ 3.75 , 3.8000002 , 3.85 , 3.9 , 3.9500003 ], [ 4. , 4.05 , 4.1 , 4.15 , 4.2000003 ], [ 4.25 , 4.3 , 4.35 , 4.4 , 4.4500003 ]], [[ 4.5 , 4.55 , 4.6 , 4.65 , 4.7000003 ], [ 4.75 , 4.8 , 4.85 , 4.9 , 4.9500003 ], [ 5. , 5.05 , 5.1 , 5.15 , 5.2000003 ], [ 5.25 , 5.3 , 5.35 , 5.4 , 5.4500003 ]]]] scale: shape=(3, 4, 5), dtype=float32 [[[0.5 , 0.52 , 0.54 , 0.56 , 0.58 ], [0.6 , 0.62 , 0.64 , 0.65999997, 0.68 ], [0.7 , 0.72 , 0.74 , 0.76 , 0.78 ], [0.79999995, 0.82 , 0.84000003, 0.86 , 0.88 ]], [[0.9 , 0.91999996, 0.94 , 0.96 , 0.98 ], [1. , 1.02 , 1.04 , 1.06 , 1.0799999 ], [1.0999999 , 1.12 , 1.14 , 1.16 , 1.1800001 ], [1.2 , 1.22 , 1.24 , 1.26 , 1.28 ]], [[1.3 , 1.3199999 , 1.3399999 , 1.3599999 , 1.38 ], [1.4 , 1.42 , 1.44 , 1.46 , 1.48 ], [1.5 , 1.52 , 1.54 , 1.56 , 1.5799999 ], [1.6 , 1.62 , 1.64 , 1.66 , 1.68 ]]] Outputs: y: shape=(2, 3, 4, 5), dtype=float32 [[[[-0.19171718, -0.17944726, -0.16564365, -0.15030625, -0.13343516], [-0.11503031, -0.09509172, -0.0736194 , -0.05061333, -0.02607353], [ 0. , 0.02760728, 0.0567483 , 0.08742305, 0.11963151], [ 0.15337373, 0.18864971, 0.22545943, 0.2638029 , 0.30367997]], [[ 0.3450909 , 0.3880356 , 0.43251398, 0.47852606, 0.52607197], [ 0.57515156, 0.62576497, 0.67791194, 0.73159266, 0.7868073 ], [ 0.8435555 , 0.90183765, 0.9616534 , 1.0230029 , 1.0858862 ], [ 1.1503031 , 1.2162539 , 1.2837384 , 1.3527564 , 1.4233084 ]], [[ 1.495394 , 1.5690132 , 1.6441666 , 1.7208533 , 1.799074 ], [ 1.8788284 , 1.9601163 , 2.0429385 , 2.127294 , 2.2131832 ], [ 2.3006063 , 2.3895628 , 2.4800534 , 2.5720778 , 2.6656356 ], [ 2.7607274 , 2.8573527 , 2.955512 , 3.055205 , 3.1564317 ]]], [[[ 0.30725956, 0.3259409 , 0.34511396, 0.36477858, 0.38493478], [ 0.40558264, 0.42672208, 0.44835314, 0.47047585, 0.49309015], [ 0.5161961 , 0.5397936 , 0.56388277, 0.58846354, 0.6135359 ], [ 0.63909984, 0.6651555 , 0.6917028 , 0.7187416 , 0.746272 ]], [[ 0.7742941 , 0.80280775, 0.83181304, 0.86131 , 0.89129865], [ 0.9217787 , 0.95275044, 0.9842137 , 1.0161688 , 1.0486155 ], [ 1.0815536 , 1.1149836 , 1.1489049 , 1.183318 , 1.2182229 ], [ 1.2536191 , 1.289507 , 1.3258864 , 1.3627577 , 1.4001204 ]], [[ 1.4379747 , 1.4763207 , 1.5151582 , 1.5544875 , 1.5943084 ], [ 1.6346208 , 1.675425 , 1.7167206 , 1.7585081 , 1.8007869 ], [ 1.8435574 , 1.8868196 , 1.9305731 , 1.9748186 , 2.0195556 ], [ 2.0647843 , 2.1105046 , 2.156716 , 2.2034197 , 2.250615 ]]]] **test_cc_rms_normalization_4d_axis2** .. code-block:: text Node: RMSNormalization(x, scale) -> (y) Attributes: axis = 2 .. code-block:: text Inputs: x: shape=(2, 3, 4, 5), dtype=float32 [[[[-0.5 , -0.45 , -0.4 , -0.35 , -0.3 ], [-0.25 , -0.19999999, -0.15 , -0.09999999, -0.04999998], [ 0. , 0.05000001, 0.10000002, 0.15000004, 0.19999999], [ 0.25 , 0.3 , 0.35000002, 0.40000004, 0.45 ]], [[ 0.5 , 0.5500001 , 0.6 , 0.65 , 0.70000005], [ 0.75 , 0.8000001 , 0.85 , 0.9 , 0.95000005], [ 1. , 1.0500001 , 1.1 , 1.15 , 1.2 ], [ 1.25 , 1.3000001 , 1.35 , 1.4 , 1.45 ]], [[ 1.5 , 1.55 , 1.6000001 , 1.6500001 , 1.7 ], [ 1.75 , 1.8 , 1.8500001 , 1.9000001 , 1.95 ], [ 2. , 2.05 , 2.1000001 , 2.15 , 2.2 ], [ 2.25 , 2.3 , 2.3500001 , 2.4 , 2.45 ]]], [[[ 2.5 , 2.55 , 2.6000001 , 2.65 , 2.7 ], [ 2.75 , 2.8 , 2.8500001 , 2.9 , 2.95 ], [ 3. , 3.05 , 3.1000001 , 3.15 , 3.2 ], [ 3.25 , 3.3 , 3.3500001 , 3.4 , 3.45 ]], [[ 3.5 , 3.5500002 , 3.6 , 3.65 , 3.7000003 ], [ 3.75 , 3.8000002 , 3.85 , 3.9 , 3.9500003 ], [ 4. , 4.05 , 4.1 , 4.15 , 4.2000003 ], [ 4.25 , 4.3 , 4.35 , 4.4 , 4.4500003 ]], [[ 4.5 , 4.55 , 4.6 , 4.65 , 4.7000003 ], [ 4.75 , 4.8 , 4.85 , 4.9 , 4.9500003 ], [ 5. , 5.05 , 5.1 , 5.15 , 5.2000003 ], [ 5.25 , 5.3 , 5.35 , 5.4 , 5.4500003 ]]]] scale: shape=(4, 5), dtype=float32 [[0.5 , 0.52 , 0.54 , 0.56 , 0.58 ], [0.6 , 0.62 , 0.64 , 0.65999997, 0.68 ], [0.7 , 0.72 , 0.74 , 0.76 , 0.78 ], [0.79999995, 0.82 , 0.84000003, 0.86 , 0.88 ]] Outputs: y: shape=(2, 3, 4, 5), dtype=float32 [[[[-0.8638168 , -0.8085325 , -0.7463378 , -0.6772324 , -0.6012165 ], [-0.5182901 , -0.42845312, -0.33170566, -0.22804761, -0.11747905], [ 0. , 0.12438966, 0.25568986, 0.39390057, 0.5390216 ], [ 0.6910534 , 0.84999573, 1.0158486 , 1.1886121 , 1.3682858 ]], [[ 0.24588391, 0.28129122, 0.31866556, 0.35800695, 0.3993155 ], [ 0.44259104, 0.4878337 , 0.5350434 , 0.5842202 , 0.63536406], [ 0.68847495, 0.743553 , 0.8005981 , 0.85961014, 0.9205893 ], [ 0.9835356 , 1.048449 , 1.1153294 , 1.1841769 , 1.2549915 ]], [[ 0.3757636 , 0.40382057, 0.43287972, 0.46294078, 0.49400386], [ 0.52606905, 0.5591362 , 0.59320545, 0.6282767 , 0.66435003], [ 0.7014253 , 0.7395027 , 0.7785822 , 0.8186636 , 0.85974705], [ 0.9018325 , 0.9449201 , 0.9890098 , 1.0341014 , 1.0801951 ]]], [[[ 0.41820854, 0.4436356 , 0.4697319 , 0.4964972 , 0.5239316 ], [ 0.5520353 , 0.58080804, 0.61024994, 0.64036095, 0.6711411 ], [ 0.70259035, 0.7347087 , 0.76749635, 0.80095303, 0.8350788 ], [ 0.86987376, 0.9053378 , 0.94147116, 0.97827345, 1.0157449 ]], [[ 0.43909788, 0.46318555, 0.48777503, 0.5128663 , 0.5384595 ], [ 0.56455445, 0.59115124, 0.6182498 , 0.64585024, 0.6739526 ], [ 0.7025566 , 0.73166263, 0.7612703 , 0.79137987, 0.8219913 ], [ 0.8531045 , 0.88471955, 0.91683644, 0.94945514, 0.9825757 ]], [[ 0.45150366, 0.4747812 , 0.49846005, 0.5225402 , 0.5470218 ], [ 0.57190466, 0.5971889 , 0.6228744 , 0.64896125, 0.6754495 ], [ 0.702339 , 0.72963 , 0.75732213, 0.78541565, 0.8139106 ], [ 0.84280676, 0.87210447, 0.9018034 , 0.93190354, 0.96240515]]]] **test_cc_rms_normalization_4d_axis3** .. code-block:: text Node: RMSNormalization(x, scale) -> (y) Attributes: axis = 3 .. code-block:: text Inputs: x: shape=(2, 3, 4, 5), dtype=float32 [[[[-0.5 , -0.45 , -0.4 , -0.35 , -0.3 ], [-0.25 , -0.19999999, -0.15 , -0.09999999, -0.04999998], [ 0. , 0.05000001, 0.10000002, 0.15000004, 0.19999999], [ 0.25 , 0.3 , 0.35000002, 0.40000004, 0.45 ]], [[ 0.5 , 0.5500001 , 0.6 , 0.65 , 0.70000005], [ 0.75 , 0.8000001 , 0.85 , 0.9 , 0.95000005], [ 1. , 1.0500001 , 1.1 , 1.15 , 1.2 ], [ 1.25 , 1.3000001 , 1.35 , 1.4 , 1.45 ]], [[ 1.5 , 1.55 , 1.6000001 , 1.6500001 , 1.7 ], [ 1.75 , 1.8 , 1.8500001 , 1.9000001 , 1.95 ], [ 2. , 2.05 , 2.1000001 , 2.15 , 2.2 ], [ 2.25 , 2.3 , 2.3500001 , 2.4 , 2.45 ]]], [[[ 2.5 , 2.55 , 2.6000001 , 2.65 , 2.7 ], [ 2.75 , 2.8 , 2.8500001 , 2.9 , 2.95 ], [ 3. , 3.05 , 3.1000001 , 3.15 , 3.2 ], [ 3.25 , 3.3 , 3.3500001 , 3.4 , 3.45 ]], [[ 3.5 , 3.5500002 , 3.6 , 3.65 , 3.7000003 ], [ 3.75 , 3.8000002 , 3.85 , 3.9 , 3.9500003 ], [ 4. , 4.05 , 4.1 , 4.15 , 4.2000003 ], [ 4.25 , 4.3 , 4.35 , 4.4 , 4.4500003 ]], [[ 4.5 , 4.55 , 4.6 , 4.65 , 4.7000003 ], [ 4.75 , 4.8 , 4.85 , 4.9 , 4.9500003 ], [ 5. , 5.05 , 5.1 , 5.15 , 5.2000003 ], [ 5.25 , 5.3 , 5.35 , 5.4 , 5.4500003 ]]]] scale: shape=(5,), dtype=float32 [0.5 , 0.52, 0.54, 0.56, 0.58] Outputs: y: shape=(2, 3, 4, 5), dtype=float32 [[[[-0.6154388 , -0.5760507 , -0.5317392 , -0.48250404, -0.4283454 ], [-0.75364137, -0.6270296 , -0.48835963, -0.33763131, -0.17484474], [ 0. , 0.21221842, 0.44076136, 0.6856287 , 0.9468203 ], [ 0.3500563 , 0.43687028, 0.5292852 , 0.6273009 , 0.7309175 ]], [[ 0.41379726, 0.4733841 , 0.5362813 , 0.60248876, 0.6720068 ], [ 0.43965474, 0.48772368, 0.53813744, 0.59089595, 0.6459994 ], [ 0.45360735, 0.49533924, 0.53888553, 0.5842463 , 0.63142145], [ 0.46232793, 0.5000539 , 0.5392594 , 0.5799442 , 0.62210846]], [[ 0.46829197, 0.50325775, 0.5394724 , 0.57693577, 0.61564785], [ 0.47262716, 0.505576 , 0.53960526, 0.5747147 , 0.6109044 ], [ 0.47592017, 0.50733083, 0.53969353, 0.5730079 , 0.6072741 ], [ 0.47850642, 0.50870544, 0.5397553 , 0.5716557 , 0.6044068 ]]], [[[ 0.48059118, 0.5098111 , 0.53980005, 0.57055783, 0.60208464], [ 0.48230743, 0.5107197 , 0.5398336 , 0.569649 , 0.60016584], [ 0.4837449 , 0.51147956, 0.53985935, 0.56888396, 0.59855366], [ 0.48496643, 0.51212454, 0.53987956, 0.5682314 , 0.5971802 ]], [[ 0.48601717, 0.5126787 , 0.53989565, 0.5676681 , 0.5959959 ], [ 0.48693067, 0.51316 , 0.53990877, 0.5671769 , 0.5949644 ], [ 0.48773223, 0.51358205, 0.5399196 , 0.56674486, 0.59405786], [ 0.48844108, 0.51395494, 0.53992856, 0.56636184, 0.5932548 ]], [[ 0.48907256, 0.514287 , 0.5399361 , 0.56601995, 0.5925386 ], [ 0.4896386 , 0.51458436, 0.5399425 , 0.5657129 , 0.59189576], [ 0.4901489 , 0.5148524 , 0.53994805, 0.56543577, 0.5913157 ], [ 0.49061126, 0.51509506, 0.53995275, 0.5651842 , 0.59078944]]]] **test_cc_rms_normalization_4d_axis_negative_1** .. code-block:: text Node: RMSNormalization(x, scale) -> (y) Attributes: axis = -1 .. code-block:: text Inputs: x: shape=(2, 3, 4, 5), dtype=float32 [[[[-0.5 , -0.45 , -0.4 , -0.35 , -0.3 ], [-0.25 , -0.19999999, -0.15 , -0.09999999, -0.04999998], [ 0. , 0.05000001, 0.10000002, 0.15000004, 0.19999999], [ 0.25 , 0.3 , 0.35000002, 0.40000004, 0.45 ]], [[ 0.5 , 0.5500001 , 0.6 , 0.65 , 0.70000005], [ 0.75 , 0.8000001 , 0.85 , 0.9 , 0.95000005], [ 1. , 1.0500001 , 1.1 , 1.15 , 1.2 ], [ 1.25 , 1.3000001 , 1.35 , 1.4 , 1.45 ]], [[ 1.5 , 1.55 , 1.6000001 , 1.6500001 , 1.7 ], [ 1.75 , 1.8 , 1.8500001 , 1.9000001 , 1.95 ], [ 2. , 2.05 , 2.1000001 , 2.15 , 2.2 ], [ 2.25 , 2.3 , 2.3500001 , 2.4 , 2.45 ]]], [[[ 2.5 , 2.55 , 2.6000001 , 2.65 , 2.7 ], [ 2.75 , 2.8 , 2.8500001 , 2.9 , 2.95 ], [ 3. , 3.05 , 3.1000001 , 3.15 , 3.2 ], [ 3.25 , 3.3 , 3.3500001 , 3.4 , 3.45 ]], [[ 3.5 , 3.5500002 , 3.6 , 3.65 , 3.7000003 ], [ 3.75 , 3.8000002 , 3.85 , 3.9 , 3.9500003 ], [ 4. , 4.05 , 4.1 , 4.15 , 4.2000003 ], [ 4.25 , 4.3 , 4.35 , 4.4 , 4.4500003 ]], [[ 4.5 , 4.55 , 4.6 , 4.65 , 4.7000003 ], [ 4.75 , 4.8 , 4.85 , 4.9 , 4.9500003 ], [ 5. , 5.05 , 5.1 , 5.15 , 5.2000003 ], [ 5.25 , 5.3 , 5.35 , 5.4 , 5.4500003 ]]]] scale: shape=(5,), dtype=float32 [0.5 , 0.52, 0.54, 0.56, 0.58] Outputs: y: shape=(2, 3, 4, 5), dtype=float32 [[[[-0.6154388 , -0.5760507 , -0.5317392 , -0.48250404, -0.4283454 ], [-0.75364137, -0.6270296 , -0.48835963, -0.33763131, -0.17484474], [ 0. , 0.21221842, 0.44076136, 0.6856287 , 0.9468203 ], [ 0.3500563 , 0.43687028, 0.5292852 , 0.6273009 , 0.7309175 ]], [[ 0.41379726, 0.4733841 , 0.5362813 , 0.60248876, 0.6720068 ], [ 0.43965474, 0.48772368, 0.53813744, 0.59089595, 0.6459994 ], [ 0.45360735, 0.49533924, 0.53888553, 0.5842463 , 0.63142145], [ 0.46232793, 0.5000539 , 0.5392594 , 0.5799442 , 0.62210846]], [[ 0.46829197, 0.50325775, 0.5394724 , 0.57693577, 0.61564785], [ 0.47262716, 0.505576 , 0.53960526, 0.5747147 , 0.6109044 ], [ 0.47592017, 0.50733083, 0.53969353, 0.5730079 , 0.6072741 ], [ 0.47850642, 0.50870544, 0.5397553 , 0.5716557 , 0.6044068 ]]], [[[ 0.48059118, 0.5098111 , 0.53980005, 0.57055783, 0.60208464], [ 0.48230743, 0.5107197 , 0.5398336 , 0.569649 , 0.60016584], [ 0.4837449 , 0.51147956, 0.53985935, 0.56888396, 0.59855366], [ 0.48496643, 0.51212454, 0.53987956, 0.5682314 , 0.5971802 ]], [[ 0.48601717, 0.5126787 , 0.53989565, 0.5676681 , 0.5959959 ], [ 0.48693067, 0.51316 , 0.53990877, 0.5671769 , 0.5949644 ], [ 0.48773223, 0.51358205, 0.5399196 , 0.56674486, 0.59405786], [ 0.48844108, 0.51395494, 0.53992856, 0.56636184, 0.5932548 ]], [[ 0.48907256, 0.514287 , 0.5399361 , 0.56601995, 0.5925386 ], [ 0.4896386 , 0.51458436, 0.5399425 , 0.5657129 , 0.59189576], [ 0.4901489 , 0.5148524 , 0.53994805, 0.56543577, 0.5913157 ], [ 0.49061126, 0.51509506, 0.53995275, 0.5651842 , 0.59078944]]]] **test_cc_rms_normalization_4d_axis_negative_2** .. code-block:: text Node: RMSNormalization(x, scale) -> (y) Attributes: axis = -2 .. code-block:: text Inputs: x: shape=(2, 3, 4, 5), dtype=float32 [[[[-0.5 , -0.45 , -0.4 , -0.35 , -0.3 ], [-0.25 , -0.19999999, -0.15 , -0.09999999, -0.04999998], [ 0. , 0.05000001, 0.10000002, 0.15000004, 0.19999999], [ 0.25 , 0.3 , 0.35000002, 0.40000004, 0.45 ]], [[ 0.5 , 0.5500001 , 0.6 , 0.65 , 0.70000005], [ 0.75 , 0.8000001 , 0.85 , 0.9 , 0.95000005], [ 1. , 1.0500001 , 1.1 , 1.15 , 1.2 ], [ 1.25 , 1.3000001 , 1.35 , 1.4 , 1.45 ]], [[ 1.5 , 1.55 , 1.6000001 , 1.6500001 , 1.7 ], [ 1.75 , 1.8 , 1.8500001 , 1.9000001 , 1.95 ], [ 2. , 2.05 , 2.1000001 , 2.15 , 2.2 ], [ 2.25 , 2.3 , 2.3500001 , 2.4 , 2.45 ]]], [[[ 2.5 , 2.55 , 2.6000001 , 2.65 , 2.7 ], [ 2.75 , 2.8 , 2.8500001 , 2.9 , 2.95 ], [ 3. , 3.05 , 3.1000001 , 3.15 , 3.2 ], [ 3.25 , 3.3 , 3.3500001 , 3.4 , 3.45 ]], [[ 3.5 , 3.5500002 , 3.6 , 3.65 , 3.7000003 ], [ 3.75 , 3.8000002 , 3.85 , 3.9 , 3.9500003 ], [ 4. , 4.05 , 4.1 , 4.15 , 4.2000003 ], [ 4.25 , 4.3 , 4.35 , 4.4 , 4.4500003 ]], [[ 4.5 , 4.55 , 4.6 , 4.65 , 4.7000003 ], [ 4.75 , 4.8 , 4.85 , 4.9 , 4.9500003 ], [ 5. , 5.05 , 5.1 , 5.15 , 5.2000003 ], [ 5.25 , 5.3 , 5.35 , 5.4 , 5.4500003 ]]]] scale: shape=(4, 5), dtype=float32 [[0.5 , 0.52 , 0.54 , 0.56 , 0.58 ], [0.6 , 0.62 , 0.64 , 0.65999997, 0.68 ], [0.7 , 0.72 , 0.74 , 0.76 , 0.78 ], [0.79999995, 0.82 , 0.84000003, 0.86 , 0.88 ]] Outputs: y: shape=(2, 3, 4, 5), dtype=float32 [[[[-0.8638168 , -0.8085325 , -0.7463378 , -0.6772324 , -0.6012165 ], [-0.5182901 , -0.42845312, -0.33170566, -0.22804761, -0.11747905], [ 0. , 0.12438966, 0.25568986, 0.39390057, 0.5390216 ], [ 0.6910534 , 0.84999573, 1.0158486 , 1.1886121 , 1.3682858 ]], [[ 0.24588391, 0.28129122, 0.31866556, 0.35800695, 0.3993155 ], [ 0.44259104, 0.4878337 , 0.5350434 , 0.5842202 , 0.63536406], [ 0.68847495, 0.743553 , 0.8005981 , 0.85961014, 0.9205893 ], [ 0.9835356 , 1.048449 , 1.1153294 , 1.1841769 , 1.2549915 ]], [[ 0.3757636 , 0.40382057, 0.43287972, 0.46294078, 0.49400386], [ 0.52606905, 0.5591362 , 0.59320545, 0.6282767 , 0.66435003], [ 0.7014253 , 0.7395027 , 0.7785822 , 0.8186636 , 0.85974705], [ 0.9018325 , 0.9449201 , 0.9890098 , 1.0341014 , 1.0801951 ]]], [[[ 0.41820854, 0.4436356 , 0.4697319 , 0.4964972 , 0.5239316 ], [ 0.5520353 , 0.58080804, 0.61024994, 0.64036095, 0.6711411 ], [ 0.70259035, 0.7347087 , 0.76749635, 0.80095303, 0.8350788 ], [ 0.86987376, 0.9053378 , 0.94147116, 0.97827345, 1.0157449 ]], [[ 0.43909788, 0.46318555, 0.48777503, 0.5128663 , 0.5384595 ], [ 0.56455445, 0.59115124, 0.6182498 , 0.64585024, 0.6739526 ], [ 0.7025566 , 0.73166263, 0.7612703 , 0.79137987, 0.8219913 ], [ 0.8531045 , 0.88471955, 0.91683644, 0.94945514, 0.9825757 ]], [[ 0.45150366, 0.4747812 , 0.49846005, 0.5225402 , 0.5470218 ], [ 0.57190466, 0.5971889 , 0.6228744 , 0.64896125, 0.6754495 ], [ 0.702339 , 0.72963 , 0.75732213, 0.78541565, 0.8139106 ], [ 0.84280676, 0.87210447, 0.9018034 , 0.93190354, 0.96240515]]]] **test_cc_rms_normalization_4d_axis_negative_3** .. code-block:: text Node: RMSNormalization(x, scale) -> (y) Attributes: axis = -3 .. code-block:: text Inputs: x: shape=(2, 3, 4, 5), dtype=float32 [[[[-0.5 , -0.45 , -0.4 , -0.35 , -0.3 ], [-0.25 , -0.19999999, -0.15 , -0.09999999, -0.04999998], [ 0. , 0.05000001, 0.10000002, 0.15000004, 0.19999999], [ 0.25 , 0.3 , 0.35000002, 0.40000004, 0.45 ]], [[ 0.5 , 0.5500001 , 0.6 , 0.65 , 0.70000005], [ 0.75 , 0.8000001 , 0.85 , 0.9 , 0.95000005], [ 1. , 1.0500001 , 1.1 , 1.15 , 1.2 ], [ 1.25 , 1.3000001 , 1.35 , 1.4 , 1.45 ]], [[ 1.5 , 1.55 , 1.6000001 , 1.6500001 , 1.7 ], [ 1.75 , 1.8 , 1.8500001 , 1.9000001 , 1.95 ], [ 2. , 2.05 , 2.1000001 , 2.15 , 2.2 ], [ 2.25 , 2.3 , 2.3500001 , 2.4 , 2.45 ]]], [[[ 2.5 , 2.55 , 2.6000001 , 2.65 , 2.7 ], [ 2.75 , 2.8 , 2.8500001 , 2.9 , 2.95 ], [ 3. , 3.05 , 3.1000001 , 3.15 , 3.2 ], [ 3.25 , 3.3 , 3.3500001 , 3.4 , 3.45 ]], [[ 3.5 , 3.5500002 , 3.6 , 3.65 , 3.7000003 ], [ 3.75 , 3.8000002 , 3.85 , 3.9 , 3.9500003 ], [ 4. , 4.05 , 4.1 , 4.15 , 4.2000003 ], [ 4.25 , 4.3 , 4.35 , 4.4 , 4.4500003 ]], [[ 4.5 , 4.55 , 4.6 , 4.65 , 4.7000003 ], [ 4.75 , 4.8 , 4.85 , 4.9 , 4.9500003 ], [ 5. , 5.05 , 5.1 , 5.15 , 5.2000003 ], [ 5.25 , 5.3 , 5.35 , 5.4 , 5.4500003 ]]]] scale: shape=(3, 4, 5), dtype=float32 [[[0.5 , 0.52 , 0.54 , 0.56 , 0.58 ], [0.6 , 0.62 , 0.64 , 0.65999997, 0.68 ], [0.7 , 0.72 , 0.74 , 0.76 , 0.78 ], [0.79999995, 0.82 , 0.84000003, 0.86 , 0.88 ]], [[0.9 , 0.91999996, 0.94 , 0.96 , 0.98 ], [1. , 1.02 , 1.04 , 1.06 , 1.0799999 ], [1.0999999 , 1.12 , 1.14 , 1.16 , 1.1800001 ], [1.2 , 1.22 , 1.24 , 1.26 , 1.28 ]], [[1.3 , 1.3199999 , 1.3399999 , 1.3599999 , 1.38 ], [1.4 , 1.42 , 1.44 , 1.46 , 1.48 ], [1.5 , 1.52 , 1.54 , 1.56 , 1.5799999 ], [1.6 , 1.62 , 1.64 , 1.66 , 1.68 ]]] Outputs: y: shape=(2, 3, 4, 5), dtype=float32 [[[[-0.19171718, -0.17944726, -0.16564365, -0.15030625, -0.13343516], [-0.11503031, -0.09509172, -0.0736194 , -0.05061333, -0.02607353], [ 0. , 0.02760728, 0.0567483 , 0.08742305, 0.11963151], [ 0.15337373, 0.18864971, 0.22545943, 0.2638029 , 0.30367997]], [[ 0.3450909 , 0.3880356 , 0.43251398, 0.47852606, 0.52607197], [ 0.57515156, 0.62576497, 0.67791194, 0.73159266, 0.7868073 ], [ 0.8435555 , 0.90183765, 0.9616534 , 1.0230029 , 1.0858862 ], [ 1.1503031 , 1.2162539 , 1.2837384 , 1.3527564 , 1.4233084 ]], [[ 1.495394 , 1.5690132 , 1.6441666 , 1.7208533 , 1.799074 ], [ 1.8788284 , 1.9601163 , 2.0429385 , 2.127294 , 2.2131832 ], [ 2.3006063 , 2.3895628 , 2.4800534 , 2.5720778 , 2.6656356 ], [ 2.7607274 , 2.8573527 , 2.955512 , 3.055205 , 3.1564317 ]]], [[[ 0.30725956, 0.3259409 , 0.34511396, 0.36477858, 0.38493478], [ 0.40558264, 0.42672208, 0.44835314, 0.47047585, 0.49309015], [ 0.5161961 , 0.5397936 , 0.56388277, 0.58846354, 0.6135359 ], [ 0.63909984, 0.6651555 , 0.6917028 , 0.7187416 , 0.746272 ]], [[ 0.7742941 , 0.80280775, 0.83181304, 0.86131 , 0.89129865], [ 0.9217787 , 0.95275044, 0.9842137 , 1.0161688 , 1.0486155 ], [ 1.0815536 , 1.1149836 , 1.1489049 , 1.183318 , 1.2182229 ], [ 1.2536191 , 1.289507 , 1.3258864 , 1.3627577 , 1.4001204 ]], [[ 1.4379747 , 1.4763207 , 1.5151582 , 1.5544875 , 1.5943084 ], [ 1.6346208 , 1.675425 , 1.7167206 , 1.7585081 , 1.8007869 ], [ 1.8435574 , 1.8868196 , 1.9305731 , 1.9748186 , 2.0195556 ], [ 2.0647843 , 2.1105046 , 2.156716 , 2.2034197 , 2.250615 ]]]] **test_cc_rms_normalization_4d_axis_negative_4** .. code-block:: text Node: RMSNormalization(x, scale) -> (y) Attributes: axis = -4 .. code-block:: text Inputs: x: shape=(2, 3, 4, 5), dtype=float32 [[[[-0.5 , -0.45 , -0.4 , -0.35 , -0.3 ], [-0.25 , -0.19999999, -0.15 , -0.09999999, -0.04999998], [ 0. , 0.05000001, 0.10000002, 0.15000004, 0.19999999], [ 0.25 , 0.3 , 0.35000002, 0.40000004, 0.45 ]], [[ 0.5 , 0.5500001 , 0.6 , 0.65 , 0.70000005], [ 0.75 , 0.8000001 , 0.85 , 0.9 , 0.95000005], [ 1. , 1.0500001 , 1.1 , 1.15 , 1.2 ], [ 1.25 , 1.3000001 , 1.35 , 1.4 , 1.45 ]], [[ 1.5 , 1.55 , 1.6000001 , 1.6500001 , 1.7 ], [ 1.75 , 1.8 , 1.8500001 , 1.9000001 , 1.95 ], [ 2. , 2.05 , 2.1000001 , 2.15 , 2.2 ], [ 2.25 , 2.3 , 2.3500001 , 2.4 , 2.45 ]]], [[[ 2.5 , 2.55 , 2.6000001 , 2.65 , 2.7 ], [ 2.75 , 2.8 , 2.8500001 , 2.9 , 2.95 ], [ 3. , 3.05 , 3.1000001 , 3.15 , 3.2 ], [ 3.25 , 3.3 , 3.3500001 , 3.4 , 3.45 ]], [[ 3.5 , 3.5500002 , 3.6 , 3.65 , 3.7000003 ], [ 3.75 , 3.8000002 , 3.85 , 3.9 , 3.9500003 ], [ 4. , 4.05 , 4.1 , 4.15 , 4.2000003 ], [ 4.25 , 4.3 , 4.35 , 4.4 , 4.4500003 ]], [[ 4.5 , 4.55 , 4.6 , 4.65 , 4.7000003 ], [ 4.75 , 4.8 , 4.85 , 4.9 , 4.9500003 ], [ 5. , 5.05 , 5.1 , 5.15 , 5.2000003 ], [ 5.25 , 5.3 , 5.35 , 5.4 , 5.4500003 ]]]] scale: shape=(2, 3, 4, 5), dtype=float32 [[[[0.5 , 0.52 , 0.54 , 0.56 , 0.58 ], [0.6 , 0.62 , 0.64 , 0.65999997, 0.68 ], [0.7 , 0.72 , 0.74 , 0.76 , 0.78 ], [0.79999995, 0.82 , 0.84000003, 0.86 , 0.88 ]], [[0.9 , 0.91999996, 0.94 , 0.96 , 0.98 ], [1. , 1.02 , 1.04 , 1.06 , 1.0799999 ], [1.0999999 , 1.12 , 1.14 , 1.16 , 1.1800001 ], [1.2 , 1.22 , 1.24 , 1.26 , 1.28 ]], [[1.3 , 1.3199999 , 1.3399999 , 1.3599999 , 1.38 ], [1.4 , 1.42 , 1.44 , 1.46 , 1.48 ], [1.5 , 1.52 , 1.54 , 1.56 , 1.5799999 ], [1.6 , 1.62 , 1.64 , 1.66 , 1.68 ]]], [[[1.6999999 , 1.72 , 1.74 , 1.76 , 1.78 ], [1.8 , 1.8199999 , 1.8399999 , 1.86 , 1.88 ], [1.9 , 1.92 , 1.9399999 , 1.9599999 , 1.98 ], [2. , 2.02 , 2.04 , 2.06 , 2.08 ]], [[2.1 , 2.12 , 2.1399999 , 2.1599998 , 2.1799998 ], [2.1999998 , 2.2199998 , 2.24 , 2.26 , 2.28 ], [2.3 , 2.32 , 2.34 , 2.3600001 , 2.38 ], [2.4 , 2.42 , 2.44 , 2.46 , 2.48 ]], [[2.5 , 2.52 , 2.54 , 2.56 , 2.58 ], [2.6 , 2.62 , 2.6399999 , 2.6599998 , 2.68 ], [2.7 , 2.72 , 2.74 , 2.76 , 2.78 ], [2.8 , 2.82 , 2.84 , 2.86 , 2.8799999 ]]]] Outputs: y: shape=(2, 3, 4, 5), dtype=float32 [[[[-0.08275866, -0.0774621 , -0.07150348, -0.06488279, -0.05760003], [-0.0496552 , -0.04104829, -0.03177932, -0.02184828, -0.01125517], [ 0. , 0.01191725, 0.02449657, 0.03773796, 0.0516414 ], [ 0.06620692, 0.08143452, 0.09732419, 0.11387593, 0.13108972]], [[ 0.14896558, 0.16750354, 0.18670353, 0.20656559, 0.22708978], [ 0.24827597, 0.2701243 , 0.2926346 , 0.315807 , 0.33964154], [ 0.36413807, 0.38929677, 0.4151174 , 0.4416002 , 0.46874508], [ 0.49655196, 0.52502096, 0.554152 , 0.58394504, 0.61440027]], [[ 0.64551747, 0.6772968 , 0.70973825, 0.7428417 , 0.7766072 ], [ 0.8110348 , 0.8461245 , 0.88187635, 0.9182902 , 0.95536596], [ 0.99310386, 1.0315039 , 1.070566 , 1.1102902 , 1.1506764 ], [ 1.1917247 , 1.233435 , 1.2758076 , 1.3188419 , 1.3625386 ]]], [[[ 1.4068972 , 1.4519179 , 1.4976007 , 1.5439456 , 1.5909524 ], [ 1.6386214 , 1.6869524 , 1.7359456 , 1.7856009 , 1.8359181 ], [ 1.8868973 , 1.9385387 , 1.9908423 , 2.0438077 , 2.0974355 ], [ 2.151725 , 2.2066767 , 2.2622907 , 2.3185666 , 2.3755045 ]], [[ 2.4331043 , 2.4913666 , 2.5502906 , 2.6098769 , 2.6701252 ], [ 2.7310355 , 2.792608 , 2.8548427 , 2.9177392 , 2.9812982 ], [ 3.0455186 , 3.1104016 , 3.1759462 , 3.2421534 , 3.3090227 ], [ 3.3765533 , 3.4447465 , 3.5136018 , 3.583119 , 3.6532986 ]], [[ 3.7241397 , 3.795643 , 3.8678086 , 3.9406362 , 4.0141263 ], [ 4.088278 , 4.163091 , 4.2385674 , 4.3147054 , 4.3915057 ], [ 4.4689674 , 4.547092 , 4.625878 , 4.705326 , 4.7854366 ], [ 4.866209 , 4.9476438 , 5.02974 , 5.1124988 , 5.1959195 ]]]] **test_cc_rms_normalization_default_axis** .. code-block:: text Node: RMSNormalization(x, scale) -> (y) .. code-block:: text Inputs: x: shape=(2, 3, 4, 5), dtype=float32 [[[[-0.5 , -0.45 , -0.4 , -0.35 , -0.3 ], [-0.25 , -0.19999999, -0.15 , -0.09999999, -0.04999998], [ 0. , 0.05000001, 0.10000002, 0.15000004, 0.19999999], [ 0.25 , 0.3 , 0.35000002, 0.40000004, 0.45 ]], [[ 0.5 , 0.5500001 , 0.6 , 0.65 , 0.70000005], [ 0.75 , 0.8000001 , 0.85 , 0.9 , 0.95000005], [ 1. , 1.0500001 , 1.1 , 1.15 , 1.2 ], [ 1.25 , 1.3000001 , 1.35 , 1.4 , 1.45 ]], [[ 1.5 , 1.55 , 1.6000001 , 1.6500001 , 1.7 ], [ 1.75 , 1.8 , 1.8500001 , 1.9000001 , 1.95 ], [ 2. , 2.05 , 2.1000001 , 2.15 , 2.2 ], [ 2.25 , 2.3 , 2.3500001 , 2.4 , 2.45 ]]], [[[ 2.5 , 2.55 , 2.6000001 , 2.65 , 2.7 ], [ 2.75 , 2.8 , 2.8500001 , 2.9 , 2.95 ], [ 3. , 3.05 , 3.1000001 , 3.15 , 3.2 ], [ 3.25 , 3.3 , 3.3500001 , 3.4 , 3.45 ]], [[ 3.5 , 3.5500002 , 3.6 , 3.65 , 3.7000003 ], [ 3.75 , 3.8000002 , 3.85 , 3.9 , 3.9500003 ], [ 4. , 4.05 , 4.1 , 4.15 , 4.2000003 ], [ 4.25 , 4.3 , 4.35 , 4.4 , 4.4500003 ]], [[ 4.5 , 4.55 , 4.6 , 4.65 , 4.7000003 ], [ 4.75 , 4.8 , 4.85 , 4.9 , 4.9500003 ], [ 5. , 5.05 , 5.1 , 5.15 , 5.2000003 ], [ 5.25 , 5.3 , 5.35 , 5.4 , 5.4500003 ]]]] scale: shape=(5,), dtype=float32 [0.5 , 0.52, 0.54, 0.56, 0.58] Outputs: y: shape=(2, 3, 4, 5), dtype=float32 [[[[-0.6154388 , -0.5760507 , -0.5317392 , -0.48250404, -0.4283454 ], [-0.75364137, -0.6270296 , -0.48835963, -0.33763131, -0.17484474], [ 0. , 0.21221842, 0.44076136, 0.6856287 , 0.9468203 ], [ 0.3500563 , 0.43687028, 0.5292852 , 0.6273009 , 0.7309175 ]], [[ 0.41379726, 0.4733841 , 0.5362813 , 0.60248876, 0.6720068 ], [ 0.43965474, 0.48772368, 0.53813744, 0.59089595, 0.6459994 ], [ 0.45360735, 0.49533924, 0.53888553, 0.5842463 , 0.63142145], [ 0.46232793, 0.5000539 , 0.5392594 , 0.5799442 , 0.62210846]], [[ 0.46829197, 0.50325775, 0.5394724 , 0.57693577, 0.61564785], [ 0.47262716, 0.505576 , 0.53960526, 0.5747147 , 0.6109044 ], [ 0.47592017, 0.50733083, 0.53969353, 0.5730079 , 0.6072741 ], [ 0.47850642, 0.50870544, 0.5397553 , 0.5716557 , 0.6044068 ]]], [[[ 0.48059118, 0.5098111 , 0.53980005, 0.57055783, 0.60208464], [ 0.48230743, 0.5107197 , 0.5398336 , 0.569649 , 0.60016584], [ 0.4837449 , 0.51147956, 0.53985935, 0.56888396, 0.59855366], [ 0.48496643, 0.51212454, 0.53987956, 0.5682314 , 0.5971802 ]], [[ 0.48601717, 0.5126787 , 0.53989565, 0.5676681 , 0.5959959 ], [ 0.48693067, 0.51316 , 0.53990877, 0.5671769 , 0.5949644 ], [ 0.48773223, 0.51358205, 0.5399196 , 0.56674486, 0.59405786], [ 0.48844108, 0.51395494, 0.53992856, 0.56636184, 0.5932548 ]], [[ 0.48907256, 0.514287 , 0.5399361 , 0.56601995, 0.5925386 ], [ 0.4896386 , 0.51458436, 0.5399425 , 0.5657129 , 0.59189576], [ 0.4901489 , 0.5148524 , 0.53994805, 0.56543577, 0.5913157 ], [ 0.49061126, 0.51509506, 0.53995275, 0.5651842 , 0.59078944]]]] **test_cc_shape_inference_tiny_llm** .. code-block:: text Node: RMSNormalization(hidden, input_layernorm.weight) -> (normed1)