Scan - version 9#

This page documents version 9 of operator Scan. See Scan for the latest version (since version 11).

  • Domain: ai.onnx

  • Since version: 9

Scan can be used to iterate over one or more scan_input tensors, constructing zero or more scan_output tensors. It combines ideas from general recurrences, functional programming constructs such as scan, fold, map, and zip, and is intended to enable generalizations of RNN-like constructs for sequence-to-sequence processing. Other tensors (referred to as state_variables here) can be used to carry a state when iterating from one element to another (similar to hidden-state in RNNs, also referred to as loop-carried dependences in the context of loops). Many common usages involve a single scan_input tensor (where functionality similar to scan, fold and map can be obtained). When more than one scan_input is used, a behavior similar to zip is obtained.

The attribute body must be a graph, specifying the computation to be performed in every iteration. It takes as input the current values of the state_variables and the current iterated element of the scan_inputs. It must return the (updated) values of the state_variables and zero or more scan_output_element tensors. The values of the scan_output_element tensors are concatenated over all the iterations to produce the scan_output values of the scan construct (similar to the concatenated intermediate hidden-state values of RNN-like constructs). All the output tensors (state_variables as well as scan_output_element tensors) are required to have the same shape in each iteration of the loop (a restriction imposed to enable efficient memory allocation).

The operation supports batching, and the batch-axis is required to be 0. When multiple scan_input tensors are used, they must all have the same batch-size, and they must all have the same maximum-sequence-length (the dimensionality of the sequence axis or scan axis).

Inputs

  • initial_state_and_scan_inputs (V): Initial values of the loop’s N state variables followed by M scan_inputs

Outputs

  • final_state_and_scan_outputs (V): Final values of the loop’s N state variables followed by K scan_outputs

Attributes

  • body (graph): The graph run each iteration. It has N+M inputs: (loop state variables…, scan_input_elts…). It has N+K outputs: (loop state variables…, scan_output_elts…). Each scan_output is created by concatenating the value of the specified scan_output_elt value at the end of each iteration of the loop. It is an error if the dimensions of these values change across loop iterations.

  • num_scan_inputs (int): An attribute specifying the number of scan_inputs M.

  • scan_input_axes (int[]): An optional list of M flags. The i-th element of the list specifies the axis to be scanned (the sequence axis) for the i-th scan_input. If omitted, 0 will be used as the scan axis for every scan_input. Negative value for an axis means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(input).

  • scan_input_directions (int[]): An optional list of M flags. The i-th element of the list specifies the direction to be scanned for the i-th scan_input tensor: 0 indicates forward direction and 1 indicates reverse direction. If omitted, all scan_input tensors will be scanned in the forward direction.

  • scan_output_axes (int[]): An optional list of K flags. The i-th element of the list specifies the axis for the i-th scan_output. The scan outputs are accumulated along the specified axis. If omitted, 0 will be used as the scan axis for every scan_output. Negative value for an axis means counting dimensions from the back. Accepted range is [-r, r-1].

  • scan_output_directions (int[]): An optional list of K flags, one for each scan_output. The i-th element of the list specifies whether the i-th scan_output should be constructed by appending or prepending a new value in each iteration: 0 indicates appending and 1 indicates prepending. If omitted, all scan_output tensors will be produced by appending a value in each iteration.

Type Constraints

  • V: All Tensor types Allowed types: tensor(bool), tensor(complex128), tensor(complex64), tensor(double), tensor(float), tensor(float16), tensor(int16), tensor(int32), tensor(int64), tensor(int8), tensor(string), tensor(uint16), tensor(uint32), tensor(uint64), tensor(uint8).

Examples#

test_cc_scan9_input_axis_negative

Node:
  Scan(initial, x) -> (y, z)
  Attributes:
    body = <subgraph>
    num_scan_inputs = 1
    scan_input_axes = [-1]

Inputs:
  initial: shape=(2,), dtype=float32
    [0., 0.]
  x: shape=(2, 3), dtype=float32
    [[1., 2., 3.],
     [4., 5., 6.]]

Outputs:
  y: shape=(2,), dtype=float32
    [ 6., 15.]
  z: shape=(3, 2), dtype=float32
    [[ 1.,  4.],
     [ 3.,  9.],
     [ 6., 15.]]

test_cc_scan9_input_reverse

Node:
  Scan(initial, x) -> (y, z)
  Attributes:
    body = <subgraph>
    num_scan_inputs = 1
    scan_input_directions = [1]

Inputs:
  initial: shape=(), dtype=float32
    0.
  x: shape=(5,), dtype=float32
    [1., 2., 3., 4., 5.]

Outputs:
  y: shape=(), dtype=float32
    15.
  z: shape=(5,), dtype=float32
    [ 5.,  9., 12., 14., 15.]

test_cc_scan9_output_axis1

Node:
  Scan(initial, x) -> (y, z)
  Attributes:
    body = <subgraph>
    num_scan_inputs = 1
    scan_output_axes = [1]

Inputs:
  initial: shape=(2,), dtype=float32
    [0., 0.]
  x: shape=(3, 2), dtype=float32
    [[1., 2.],
     [3., 4.],
     [5., 6.]]

Outputs:
  y: shape=(2,), dtype=float32
    [ 9., 12.]
  z: shape=(2, 3), dtype=float32
    [[ 1.,  4.,  9.],
     [ 2.,  6., 12.]]

test_cc_scan9_output_reverse

Node:
  Scan(initial, x) -> (y, z)
  Attributes:
    body = <subgraph>
    num_scan_inputs = 1
    scan_output_directions = [1]

Inputs:
  initial: shape=(), dtype=float32
    0.
  x: shape=(5,), dtype=float32
    [1., 2., 3., 4., 5.]

Outputs:
  y: shape=(), dtype=float32
    15.
  z: shape=(5,), dtype=float32
    [15., 10.,  6.,  3.,  1.]

test_scan9_multi_state

Node:
  Scan(initial_sum, initial_prod, x) -> (y_sum, y_prod, z)
  Attributes:
    body = <subgraph>
    num_scan_inputs = 1

Inputs:
  initial_sum: shape=(2,), dtype=float32
    [0., 0.]
  initial_prod: shape=(2,), dtype=float32
    [1., 1.]
  x: shape=(3, 2), dtype=float32
    [[1., 2.],
     [3., 4.],
     [5., 6.]]

Outputs:
  y_sum: shape=(2,), dtype=float32
    [ 9., 12.]
  y_prod: shape=(2,), dtype=float32
    [15., 48.]
  z: shape=(3, 2), dtype=float32
    [[ 1.,  2.],
     [ 4.,  6.],
     [ 9., 12.]]

test_scan9_scalar

Node:
  Scan(initial, x) -> (y, z)
  Attributes:
    body = <subgraph>
    num_scan_inputs = 1

Inputs:
  initial: shape=(), dtype=float32
    0.
  x: shape=(5,), dtype=float32
    [1., 2., 3., 4., 5.]

Outputs:
  y: shape=(), dtype=float32
    15.
  z: shape=(5,), dtype=float32
    [ 1.,  3.,  6., 10., 15.]

test_scan9_sum

Node:
  Scan(initial, x) -> (y, z)
  Attributes:
    body = <subgraph>
    num_scan_inputs = 1

Inputs:
  initial: shape=(2,), dtype=float32
    [0., 0.]
  x: shape=(3, 2), dtype=float32
    [[1., 2.],
     [3., 4.],
     [5., 6.]]

Outputs:
  y: shape=(2,), dtype=float32
    [ 9., 12.]
  z: shape=(3, 2), dtype=float32
    [[ 1.,  2.],
     [ 4.,  6.],
     [ 9., 12.]]

Differences with previous version (8)#

SchemaDiff: Scan (domain 'ai.onnx')

  • old version: 8

  • new version: 9

  • breaking: yes

Breaking reasons:

  • input ‘sequence_lens’ (removed): was at position 0

  • input ‘initial_state_and_scan_inputs’ (changed): position changed 1 -> 0

  • attribute ‘directions’ (removed): type=INTS; required=False

  • type constraint ‘I’ (removed): entire constraint removed

Inputs:

  • [BREAKING] removed ‘sequence_lens’: was at position 0

  • [BREAKING] changed ‘initial_state_and_scan_inputs’: position changed 1 -> 0

Attributes:

  • [BREAKING] removed ‘directions’: type=INTS; required=False

  • added ‘scan_input_directions’: type=INTS; required=False; default=UNDEFINED

  • added ‘scan_output_directions’: type=INTS; required=False; default=UNDEFINED

  • added ‘scan_input_axes’: type=INTS; required=False; default=UNDEFINED

  • added ‘scan_output_axes’: type=INTS; required=False; default=UNDEFINED

Type constraints:

  • [BREAKING] removed ‘I’: entire constraint removed

Documentation:

  • line similarity: 0.95 (+1/-1 lines)

--- Scan v8
+++ Scan v9
@@ -19,4 +19,4 @@
 well as scan_output_element tensors) are required to have the same shape in each iteration
 of the loop (a restriction imposed to enable efficient memory allocation).

-Note that the iterated element passed to the body subgraph does not have a sequence axis. It will have a rank one less than the rank of the corresponding scan_input.
+The operation supports batching, and the batch-axis is required to be 0. When multiple scan_input tensors are used, they must all have the same batch-size, and they must all have the same maximum-sequence-length (the dimensionality of the sequence axis or scan axis).