Replacing onnxruntime’s protobuf usage with onnx-light#

onnxruntime (ORT) is written against the protobuf implementation of onnx. Besides the message types (ModelProto, TensorProto, …), ORT calls the protobuf message API (ParseFromArray, SerializeToArray, ByteSizeLong, …) and the protobuf I/O streams (google::protobuf::io::ZeroCopyInputStream, FileInputStream, FileOutputStream, …).

onnx-light is protobuf-free: its message classes do not expose these methods. This page documents, method by method, how to replace every protobuf / onnx C++ construct that onnxruntime relies on with its onnx-light equivalent. It is the reference used by the onnxruntime_USE_ONNX_LIGHT build option.

Detecting the backend at compile time#

When onnxruntime is configured with -Donnxruntime_USE_ONNX_LIGHT=ON the build defines the preprocessor macro ORT_USE_ONNX_LIGHT for every translation unit (add_compile_definitions(ORT_USE_ONNX_LIGHT)). Code that must behave differently between the two backends branches on that macro:

#if defined(ORT_USE_ONNX_LIGHT)
  // onnx-light native API
#else
  // protobuf API
#endif

The onnx-light drop-in header <onnx/onnx_pb.h> also defines ONNX_LIGHT_NAMESPACE (the upstream onnx package does not), which can be used as a header-only fallback detector.

The message (de)serialization API#

onnx-light message classes expose the following native API (declared by the SERIALIZATION_METHOD() macro in onnx_light/onnx_proto/stream_class.h):

onnx-light method

Meaning

bool ParseFromString(const std::string&)

Parse the message from a byte buffer (returns true; throws on error).

bool ParseFromIstream(std::istream*)

Parse the message from a std::istream (reads to EOF).

bool SerializeToString(std::string&) const

Serialize the message into a std::string (returns true; throws on error).

SerializeSizeResult SerializeSize() const

Compute the serialized-size split (proto / small / big data).

size_t ByteSizeLong() const

Total serialized size in bytes (SerializeSize().size()); no serialization.

Every protobuf message method used by onnxruntime maps onto that native API:

protobuf (onnx) call

onnx-light replacement

bool p.ParseFromArray(data, size)

p.ParseFromString(std::string(reinterpret_cast<const char*>(data), size)); // returns bool

bool p.ParseFromString(str)

p.ParseFromString(str); // returns bool, throws on error

bool p.ParseFromZeroCopyStream(&IstreamInputStream)

p.ParseFromIstream(&istream)

bool p.ParseFromZeroCopyStream(&FileInputStream) (fd)

read the descriptor into a std::string then p.ParseFromString(...)

bool p.SerializeToString(&str)

p.SerializeToString(str); // returns bool

std::string p.SerializeAsString()

std::string s; p.SerializeToString(s); return s;

size_t p.ByteSizeLong()

p.ByteSizeLong(); // native, calls SerializeSize() – no serialization

bool p.SerializeToArray(data, size)

std::string s; p.SerializeToString(s); memcpy(data, s.data(), s.size());

bool p.SerializeToOstream(&os)

std::string s; p.SerializeToString(s); os.write(s.data(), s.size());

bool p.SerializeToFileDescriptor(fd)

std::string s; p.SerializeToString(s); write(fd, s.data(), s.size());

FileOutputStream o(fd); p.SerializeToZeroCopyStream(&o) && o.Flush()

std::string s; p.SerializeToString(s); write(fd, s.data(), s.size());

Note

onnx-light has no 2 GB message-size limit and lets exceptions propagate instead of returning false (see the design notes). The wrappers above return true on success so onnxruntime’s bool call sites keep working unchanged.

A ready-made onnxruntime helper#

To keep the two backends in a single place, onnxruntime ships a header-only adapter, include/onnxruntime/core/graph/onnx_proto_serialize.h, exposing templated free functions in namespace onnxruntime::proto_io. Each function compiles to the protobuf call or the onnx-light call depending on ORT_USE_ONNX_LIGHT:

Helper

Replaces

proto_io::ParseFromArray(p, data, size)

p.ParseFromArray(data, size)

proto_io::ParseFromString(p, str)

p.ParseFromString(str) (bool context)

proto_io::ParseFromIStream(p, istream)

ParseFromZeroCopyStream(&IstreamInputStream)

proto_io::ParseFromFileDescriptor(p, fd)

ParseFromZeroCopyStream(&FileInputStream)

proto_io::SerializeToString(p, str)

p.SerializeToString(&str)

proto_io::SerializeAsString(p)

p.SerializeAsString()

proto_io::ByteSize(p)

p.ByteSizeLong()

proto_io::SerializeToArray(p, data, size)

p.SerializeToArray(data, size)

proto_io::SerializeToOStream(p, os)

p.SerializeToOstream(&os)

proto_io::SerializeToFileDescriptor(p, fd)

p.SerializeToFileDescriptor(fd)

proto_io::SaveToFileDescriptor(p, fd)

FileOutputStream(fd) + SerializeToZeroCopyStream + Flush

 
#include <google/protobuf/io/zero_copy_stream_impl.h>

google::protobuf::io::IstreamInputStream zci(&istream);
bool ok = model_proto.ParseFromZeroCopyStream(&zci) && istream.eof();

auto n = model_proto.ByteSizeLong();
model_proto.SerializeToArray(buffer, n);

#include "core/graph/onnx_proto_serialize.h"

bool ok = onnxruntime::proto_io::ParseFromIStream(model_proto, istream);

auto n = onnxruntime::proto_io::ByteSize(model_proto);
onnxruntime::proto_io::SerializeToArray(model_proto, buffer, n);

The google::protobuf symbols still referenced by onnxruntime#

A few onnxruntime files reference the google::protobuf namespace for non-serialization purposes. onnx-light provides drop-in equivalents in onnx_light/onnx_proto/google_protobuf_compat.h (pulled in automatically by the <onnx/onnx_pb.h> compatibility header):

protobuf symbol

onnx-light shim

google::protobuf::RepeatedField<T>

alias of onnx_light::utils::RepeatedField<T>

google::protobuf::RepeatedPtrField<T>

alias of onnx_light::utils::RepeatedProtoField<T>

google::protobuf::RepeatedFieldBackInsertIterator<T>

alias of onnx_light::utils::RepeatedFieldBackInsertIterator<T>

google::protobuf::RepeatedFieldBackInserter(&f)

forwards to onnx_light::utils::RepeatedFieldBackInserter

google::protobuf::ShutdownProtobufLibrary()

no-op (onnx-light has no global protobuf state)

google::protobuf::io::ArrayInputStream

alias of onnx_light::utils::StringStream

google::protobuf::io::CodedInputStream

alias of onnx_light::utils::CodedInputStream

google::protobuf::io::StringOutputStream

alias of onnx_light::utils::StdStringWriteStream

google::protobuf::io::FileOutputStream

alias of onnx_light::utils::FdWriteStream

google::protobuf::io::IstreamInputStream

alias of onnx_light::utils::IstreamStream

google::protobuf::io::OstreamOutputStream

alias of onnx_light::utils::OstreamWriteStream

Every class in google_protobuf_compat.h is now a pure using alias to a concrete onnx-light class. The ZeroCopy stream interface (Next / BackUp / ByteCount and, for output streams, Flush) is implemented directly on the onnx-light BinaryStream / BinaryWriteStream hierarchy in stream.h / stream.cc, so the compat header carries no implementation of its own.

Version guards#

Blocks guarded by GOOGLE_PROTOBUF_VERSION must also exclude the onnx-light build, because the macro is undefined there (and would evaluate to 0). The protobuf-arena helpers RepeatedPtrField::ClearedCount / ReleaseCleared have no onnx-light counterpart and are simply skipped (onnx-light stores repeated fields in std::vector and frees them on Clear()):

#if !defined(ORT_USE_ONNX_LIGHT) && GOOGLE_PROTOBUF_VERSION < 5026000
  // protobuf-only cleared-object reclamation
#endif

onnxruntime files converted#

The core library (always built) has been converted to route through onnxruntime::proto_io or to guard protobuf-only code:

  • core/graph/model.cc — istream / file-descriptor load and save, ByteSizeLong, SerializeToArray, ParseFromArray.

  • core/graph/graph.cc — ClearedCount / ReleaseCleared guards.

  • core/session/inference_session.cc — ParseFromArray, ByteSizeLong.

  • core/session/provider_bridge_ort.cc — ModelProto / FunctionProto SerializeToString / SerializeToOstream / ParseFromString / SerializeAsString bridge methods.

  • core/framework/graph_partitioner.cc — ByteSizeLong, SerializeToArray, SerializeToOstream, file-descriptor save.

  • core/framework/debug_node_inputs_outputs_utils.cc — SerializeToFileDescriptor, SerializeAsString.

  • core/framework/allocation_planner.cc — ByteSizeLong.

The same substitution (include core/graph/onnx_proto_serialize.h and call the matching proto_io helper) applies to the optional components that are only built on demand and therefore not covered by the default build:

  • execution providers — providers/tensorrt, providers/nv_tensorrt_rtx, providers/openvino, providers/cann, providers/migraphx, providers/vitisai (SerializeToString / ParseFromString / SerializeAsString used in a bool / return context).

  • training — orttraining/... (ParseFromArray / SerializeToZeroCopyStream).

  • tests — test/onnx, test/framework, test/fuzzing (model load / save helpers).

See also#