DelayedInitializer

• Domain: ai.rt

• Since version: 1

Defers materialization of a tensor stored in an external weights file.

In onnx-light, `load_device`` must be either ``"cpu"`` or ``"file"` and `runtime_device`` must be ``"cpu"``. When ``load_device`` is ``"cpu"`, the kernel loads the tensor bytes during kernel initialization and returns a CPU copy at execution time. When `load_device`` is ``"file"`, initialization does not touch the file and execution loads the tensor bytes directly from `filename`` at byte ``offset`.

The static output shape comes from the required `shape` attribute and the element type comes from the required `dtype` attribute.

Outputs

• output (T): Tensor produced by the delayed initializer.

Attributes

• dtype (int): Element type of the output tensor, encoded as a TensorProto::DataType value.

• filename (string): Filename containing the serialized tensor payload.

• load_device (string): Device where the initializer is first loaded.

• offset (int): Byte offset of the tensor payload within filename.

• runtime_device (string): Device where the initializer is moved at runtime.

• shape (int[]): Shape of the output tensor.

Type Constraints

• T: Constrain output to tensor types backed by raw byte storage. Allowed types: tensor(bool), tensor(complex128), tensor(complex64), tensor(double), tensor(float), tensor(float16), tensor(int16), tensor(int32), tensor(int64), tensor(int8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(uint8).

Examples

test_cc_delayedinitializer_cpu

Node:
  ai.rt.DelayedInitializer() -> (y)
  Attributes:
    shape = [3]
    dtype = 1
    load_device = "cpu"
    runtime_device = "cpu"
    filename = "/tmp/onnx_light_backend_delayedinitializer_cpu.bin"
    offset = 0

Inputs:

Outputs:
  y: shape=(3,), dtype=float32
    [3., 4., 5.]

test_cc_delayedinitializer_file

Node:
  ai.rt.DelayedInitializer() -> (y)
  Attributes:
    shape = [2]
    dtype = 1
    load_device = "file"
    runtime_device = "cpu"
    filename = "/tmp/onnx_light_backend_delayedinitializer_file.bin"
    offset = 8

Inputs:

Outputs:
  y: shape=(2,), dtype=float32
    [ 1.5, -2. ]