DelayedInitializer

• Domain: ai.rt

• Since version: 1

Defers materialization of a tensor stored in an external weights file.

In onnx-light, load_device must be either "cpu" or "file" and runtime_device must be "cpu". When load_device is "cpu", the kernel loads the tensor bytes during kernel initialization and returns a CPU copy at execution time. When load_device is "file", initialization does not touch the file and execution loads the tensor bytes directly from filename at byte offset.

The static output shape comes from the required shape attribute and the element type comes from the required dtype attribute.

Outputs

• output (T): Tensor produced by the delayed initializer.

Attributes

• dtype (int): Element type of the output tensor, encoded as a TensorProto::DataType value.

• filename (string): Filename containing the serialized tensor payload.

• load_device (string): Device where the initializer is first loaded.

• offset (int): Byte offset of the tensor payload within filename.

• runtime_device (string): Device where the initializer is moved at runtime.

• shape (int[]): Shape of the output tensor.

Type Constraints

• T: Constrain output to tensor types backed by raw byte storage. Allowed types: tensor(bool), tensor(complex128), tensor(complex64), tensor(double), tensor(float), tensor(float16), tensor(int16), tensor(int32), tensor(int64), tensor(int8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(uint8).

Examples

test_cc_delayedinitializer_cpu

Node:
  ai.rt.DelayedInitializer() -> (y)
  Attributes:
    shape = [3]
    dtype = 1
    load_device = "cpu"
    runtime_device = "cpu"
    filename = "/tmp/onnx_light_backend_delayedinitializer_cpu.bin"
    offset = 0

Inputs:

Outputs:
  y: shape=(3,), dtype=float32
    [3., 4., 5.]

test_cc_delayedinitializer_file

Node:
  ai.rt.DelayedInitializer() -> (y)
  Attributes:
    shape = [2]
    dtype = 1
    load_device = "file"
    runtime_device = "cpu"
    filename = "/tmp/onnx_light_backend_delayedinitializer_file.bin"
    offset = 8

Inputs:

Outputs:
  y: shape=(2,), dtype=float32
    [ 1.5, -2. ]