onnx_light.onnx.external_data_helper#

Compatibility shim for onnx.external_data_helper.

Re-exports the helpers from onnx_light.onnx_lib.external_data_helper and adds ExternalDataInfo which ir-py uses to inspect external-data metadata on a TensorProto.

class onnx_light.onnx.external_data_helper.ExternalDataInfo(tensor: TensorProto)#

Parses external-data metadata from a TensorProto.

location#: Relative path to the external data file.

offset#: Byte offset into the file, or None.

length#: Number of bytes to read, or None.

checksum#: Optional checksum string, or None.

basepath#: Optional base path override, or None.

onnx_light.onnx.external_data_helper.convert_model_to_external_data(model: ModelProto, all_tensors_to_one_file: bool = True, location: str | None = None, size_threshold: int = 1024, convert_attribute: bool = False) → None#

Marks every initializer tensor with raw data above size_threshold as external data. The actual bytes are not written; they remain in raw_data and are flushed to disk by the next call to onnx_light.onnx.save().

Parameters:

model – model to modify in place.
all_tensors_to_one_file – when True (default), every qualifying tensor points at the same external file (location or a generated <uuid>.data name). When False, each tensor is given its own file named after the tensor.
location – relative path of the external data file. Must be relative to the model file. Ignored when all_tensors_to_one_file=False.
size_threshold – only tensors whose raw_data size is greater than or equal to size_threshold bytes are moved to external storage. Set to 0 to externalize every tensor with raw data.
convert_attribute – when True, also externalize tensors stored inside node attributes (AttributeProto.t and AttributeProto.tensors).

Raises:

ValueError – if location is an absolute path.
FileExistsError – if location already exists on disk.

onnx_light.onnx.external_data_helper.load_external_data_for_model(model: ModelProto, base_dir: str) → None#

Loads external tensor bytes into model in place.

For every tensor whose data lives in an external file, this function reads the bytes from base_dir into the tensor’s raw_data and resets data_location to DEFAULT, dropping the external_data entries.

Parameters:

model – model whose external tensors are loaded in place.
base_dir – directory that contains the external data files referenced by the tensors.

onnx_light.onnx.external_data_helper.load_external_data_for_tensor(tensor: TensorProto, base_dir: str) → None#

Loads data from an external file into tensor.raw_data.

Validates that the external data path does not escape base_dir via path traversal or symlink indirection before reading the file. Also validates that offset and length metadata values are non-negative and within the actual file size (GHSA-3jf9-582g-jjmq).

Parameters:

tensor – a TensorProto object whose external_data field describes the file.
base_dir – directory that contains the external data file.

Raises:

ValueError – If the location escapes the base directory, or if offset / length are negative or exceed the file size.

onnx_light.onnx.external_data_helper.set_external_data(tensor: TensorProto, location: str, offset: int | None = None, length: int | None = None, checksum: str | None = None, basepath: str | None = None) → None#

Marks tensor as EXTERNAL and records the given external-data entries.

The tensor must currently carry inline raw_data; the bytes are left untouched so they can be written out by a subsequent save call.

onnx_light.onnx.external_data_helper.uses_external_data(tensor: TensorProto) → bool#: Returns True if the tensor stores its bytes in an external file.