GridSample#

  • Domain: ai.onnx

  • Since version: 22

Given an input X and a flow-field grid, computes the output Y using X values and pixel locations from the grid. For spatial input X with shape (N, C, H, W), the grid will have shape (N, H_out, W_out, 2), the output Y will have shape (N, C, H_out, W_out). For volumetric input X with shape (N, C, D, H, W), the grid will have shape (N, D_out, H_out, W_out, 3), the output Y will have shape (N, C, D_out, H_out, W_out). More generally, for an input X of rank r+2 with shape (N, C, d1, d2, …, dr), the grid will have shape (N, D1_out, D2_out, …, Dr_out, r), the output Y will have shape (N, C, D1_out, D2_out, …, Dr_out).

The tensor X contains values at centers of square pixels (voxels, etc) locations such as (n, c, d1_in, d2_in, …, dr_in). The (n, d1_out, d2_out, …, dr_out, :) values from the tensor grid are the normalized positions for interpolating the values at the (n, c, d1_out, d2_out, …, dr_out) locations from the output tensor Y using a specified interpolation method (the mode) and a padding mode (for grid positions falling outside the 2-dimensional image).

For example, the values in grid[n, h_out, w_out, :] are size-2 vectors specifying normalized positions in the 2-dimensional space of X. They are used to interpolate output values of Y[n, c, h_out, w_out].

The GridSample operator is often used in doing grid generator and sampler in the Spatial Transformer Networks. See also in torch.nn.functional.grid_sample.

Inputs

  • X (T1): Input tensor of rank r+2 that has shape (N, C, D1, D2, …, Dr), where N is the batch size, C is the number of channels, D1, D2, …, Dr are the spatial dimensions.

  • grid (T2): Input offset of shape (N, D1_out, D2_out, …, Dr_out, r), where D1_out, D2_out, …, Dr_out are the spatial dimensions of the grid and output, and r is the number of spatial dimensions. Grid specifies the sampling locations normalized by the input spatial dimensions. Therefore, it should have most values in the range of [-1, 1]. If the grid has values outside the range of [-1, 1], the corresponding outputs will be handled as defined by padding_mode. Following computer vision convention, the coordinates in the length-r location vector are listed from the innermost tensor dimension to the outermost, the opposite of regular tensor indexing.

Outputs

  • Y (T1): Output tensor of rank r+2 that has shape (N, C, D1_out, D2_out, …, Dr_out) of the sampled values. For integer input types, intermediate values are computed as floating point and cast to integer at the end.

Attributes

  • align_corners (int): If align_corners=1, the extrema (-1 and 1) are considered as referring to the center points of the input’s corner pixels (voxels, etc.). If align_corners=0, they are instead considered as referring to the corner points of the input’s corner pixels (voxels, etc.), making the sampling more resolution agnostic.

  • mode (string): Three interpolation modes: linear (default), nearest and cubic. The “linear” mode includes linear and N-linear interpolation modes depending on the number of spatial dimensions of the input tensor (i.e. linear for 1 spatial dimension, bilinear for 2 spatial dimensions, etc.). The “cubic” mode also includes N-cubic interpolation modes following the same rules. The “nearest” mode rounds to the nearest even index when the sampling point falls halfway between two indices.

  • padding_mode (string): Support padding modes for outside grid values: zeros``(default), ``border, reflection. zeros: use 0 for out-of-bound grid locations, border: use border values for out-of-bound grid locations, reflection: use values at locations reflected by the border for out-of-bound grid locations. If index 0 represents the margin pixel, the reflected value at index -1 will be the same as the value at index 1. For location far away from the border, it will keep being reflected until becoming in bound. If pixel location x = -3.5 reflects by border -1 and becomes x’ = 1.5, then reflects by border 1 and becomes x’’ = 0.5.

Type Constraints

  • T1: Constrain input X and output Y types to all tensor types. Allowed types: tensor(bfloat16), tensor(bool), tensor(complex128), tensor(complex64), tensor(double), tensor(float), tensor(float16), tensor(int16), tensor(int32), tensor(int64), tensor(int8), tensor(string), tensor(uint16), tensor(uint32), tensor(uint64), tensor(uint8).

  • T2: Constrain grid types to float tensors. Allowed types: tensor(bfloat16), tensor(double), tensor(float), tensor(float16).

Differences with previous version (20)#

SchemaDiff: GridSample (domain 'ai.onnx')

  • old version: 20

  • new version: 22

  • breaking: no

Type constraints:

  • changed ‘T1’: added types: [‘tensor(bfloat16)’]

  • changed ‘T2’: added types: [‘tensor(bfloat16)’]

Version History#