graph_builder.h#
Incremental builder for ONNX graphs, models and functions.
:cpp:class:core::builder::GraphBuilder accumulates graph inputs, initializers, nodes and outputs one call at a time and keeps the associated compute metadata (shapes, in-place reuse, value tags and per-node peak memory) up to date through an owned :cpp:class:core::compute::ComputeContext.
The builder does not use a :cpp:class:GraphProto as its working container. Inputs, outputs, nodes, initializers and the nested local functions / subgraphs (each of which is itself a :cpp:class:GraphBuilder) are kept in plain vectors, in declaration order; each entry carries its own name so no side map is needed. A proto is only materialised on demand by :cpp:func:BuildGraph and the finalizers.
A builder starts empty. Every value name it hands out (graph inputs, initializers and node outputs) is recorded so a name can never be reused. Each :cpp:func:GraphBuilder::MakeNode call resolves the operator opset (falling back to the latest known one when the domain has no explicit opset), validates the node against the matching :cpp:class:core::schema::LightOpSchema when one is available, assigns output names when the caller left them empty and runs incremental shape inference for the new node.
:cpp:func:GraphBuilder::ToModel, :cpp:func:GraphBuilder::ToGraph and :cpp:func:GraphBuilder::ToFunction finalize the accumulated graph: they run the whole-graph compute analyses and write the inferred shapes, the in-place / release-after metadata, the value tags and the peak-memory estimates into the produced proto.
-
namespace onnx_light
Alias that makes onnx-light headers compatible with code that references
ONNX_LIGHT_NAMESPACE(the macro used in the standard onnx package).Set to
ONNX_LIGHT_NAMESPACEso both names resolve to the same namespace.Symbol-visibility attribute for the public onnx-light C++ API.
Maps the upstream compatibility macro to onnx-light’s explicit proto ABI annotation. This keeps declarations from vendored ONNX headers visible when
lib_onnx_protouses hidden visibility by default.Namespace alias so that ONNX C++ code (and consumers such as onnxruntime) that refers to the literal
onnxnamespace — rather than theONNX_NAMESPACEmacro — resolves to the onnx-light namespace. The standard onnx package lives innamespace onnx; onnx-light usesonnx_light(via ONNX_LIGHT_NAMESPACE), so this alias keeps onnx-light a true drop-in. It is only introduced when the onnx-light namespace differs fromonnx.-
namespace core
-
namespace builder#
Enums
-
enum class ProtoKind#
Selects which ONNX proto :cpp:func:
GraphBuilder::ToOnnxproduces.Values:
-
enumerator kModel#
Produce a :cpp:class:
ModelProto(the default).
-
enumerator kGraph#
Produce a bare :cpp:class:
GraphProto.
-
enumerator kFunction#
Produce a :cpp:class:
FunctionProto.
-
enumerator kModel#
-
class BuilderError : public std::runtime_error#
- #include <graph_builder.h>
Thrown when :cpp:class:
GraphBuilderis used incorrectly, for example when a name is reused or the opset version of a domain cannot be resolved.Public Functions
-
inline explicit BuilderError(const std::string &message, std::source_location location = std::source_location::current())#
Constructs an error with the message and its call-site source location.
-
~BuilderError() override#
-
inline explicit BuilderError(const std::string &message, std::source_location location = std::source_location::current())#
-
struct ConstantFoldingOptions#
- #include <graph_builder.h>
Options controlling :cpp:func:
GraphBuilder::ConstantFold.Constant folding evaluates every node whose outputs are known before inference (initializers and, transitively, the outputs of deterministic nodes fed only by constants) and replaces the node with the resulting initializers. The evaluation uses the process-wide runtime kernel registry (:cpp:func:
core::runtime::KernelDispatchTable), which must be populated by a kernel library (onnx_kernels) for folding to happen.Public Members
-
bool enabled = true#
Master switch: when
false:cpp:func:ConstantFoldis a no-op and returns0without touching the graph.
-
int64_t max_element_count = -1#
Skips folding a node when any of its outputs would hold strictly more than
max_element_countelements. A negative value (the default) means no limit, so results of any size are folded.
-
std::set<std::pair<std::string, std::string>> excluded_ops#
(domain, op_type)pairs that must never be folded. An empty domain matches every domain and an empty op_type matches every operator, so an empty-empty pair disables folding for every node. The domain is matched after normalisation (""and"ai.onnx"compare equal). Astd::setis used so exclusion lookups stay logarithmic instead of scanning a vector.
-
bool fold_weights = true#
Controls whether nodes whose results are tagged
"weight"(or untagged) are folded. Shape-tagged results are always foldable and can be folded at any point in an optimization pipeline; weight results are usually better folded only at the end (after other passes have run), so this switch lets a caller fold shapes early and defer weight folding to a final pass. Whenfalseonly shape-tagged results are folded.
-
bool raise_on_missing_weight_kernel = false#
When
truea node whose outputs are tagged"weight"(or untagged) but for which no runtime kernel is registered raises a :cpp:class:BuilderErrorinstead of being left untouched. Nodes whose outputs are tagged"shape"always raise when their kernel is missing, regardless of this flag.
-
bool enabled = true#
-
class GraphBuilder#
- #include <graph_builder.h>
Incrementally builds an ONNX graph, model or function while keeping the associated compute metadata up to date.
The builder owns a :cpp:class:
core::compute::ComputeContext; every node added through :cpp:func:MakeNodeis immediately fed to incremental shape inference so the shape of any produced value can be queried mid-build with :cpp:func:GetShape.Public Types
-
using SchemaLookupFn = std::function<std::vector<LightOpSchema>(const std::string &op_type)>#
Signature of the optional callback used to resolve the versioned schema history of an operator. Given an
op_typeit returns every :cpp:class:core::schema::LightOpSchemaregistered for that operator (across every domain); an empty vector means the operator is unknown.onnx_coreowns :cpp:class:core::schema::LightOpSchemabut not the built-in operator schemas (those live in theonnx_oplibrary, which depends ononnx_core), so the provider is injected by the caller (see the Python bindings, which wire the built-in ONNX schemas).
Public Functions
-
explicit GraphBuilder(std::string name = "graph", SchemaLookupFn schema_lookup = {})#
Constructs an empty builder.
- Parameters:
name – Name given to the produced graph / function.
schema_lookup – Optional schema provider used to validate nodes and to resolve the “latest opset” of a domain.
-
explicit GraphBuilder(const ModelProto &model, SchemaLookupFn schema_lookup = {})#
Constructs a builder from an existing model by replaying every graph and function node through :cpp:func:
MakeNode.Graph-valued node attributes are converted into nested subgraph builders, and the owning node stores a
<attr_name>_refSTRING (or STRINGS) attribute carrying the nested builder name(s). :cpp:func:BuildGraph/ :cpp:func:ToGraph/ :cpp:func:ToModelmaterialize those references back into GRAPH / GRAPHS attributes.
-
~GraphBuilder()#
-
GraphBuilder(GraphBuilder&&) noexcept#
-
GraphBuilder &operator=(GraphBuilder&&) noexcept#
-
GraphBuilder(const GraphBuilder&) = delete#
-
GraphBuilder &operator=(const GraphBuilder&) = delete#
-
void SetOpsetVersion(const std::string &domain, int version)#
Records the opset version to use for
domain(an empty string denotes the default ONNX domain). Explicitly setting an opset prevents the builder from deriving it from operator schemas.
-
int OpsetVersion(const std::string &domain) const#
Returns the opset version recorded for
domainor :cpp:var:core::shapes::kUnknownOpsetVersionwhen none is set.
-
inline const std::unordered_map<std::string, int> &OpsetVersions() const noexcept#
Read-only access to the recorded
domain -> opset versionmap.
-
bool HasName(const std::string &name) const noexcept#
Returns
truewhennamehas already been handed out.
-
const std::string &ReserveName(const std::string &name)#
Records
nameas used and returns it. Throws :cpp:class:BuilderErrorwhen the name is empty or already used.
-
std::string UniqueName(const std::string &prefix = "n")#
Returns a fresh, unused name starting with
prefixand records it.
-
const std::string &MakeInitializer(const TensorProto &tensor)#
Appends
tensoras a graph initializer. The tensor may carry external data (data_location == EXTERNAL). Returns the initializer name.
-
const std::string &MakeExternalInitializer(const std::string &name, TensorType dtype, const std::vector<int64_t> &dims, const std::string &location, int64_t offset, int64_t length)#
Builds and appends an initializer whose data lives in an external file.
- Parameters:
name – Initializer name.
dtype – Element type.
dims – Tensor shape.
location – Path of the external data file (relative to the model).
offset – Byte offset of the data inside the file.
length – Number of bytes of the data inside the file.
- Returns:
The initializer name.
-
inline const utils::RepeatedProtoField<TensorProto> &Initializers() const noexcept#
Read-only access to the graph initializers, in declaration order.
-
const std::string &MakeInput(const ValueInfoProto &value_info)#
Declares a graph input from a ready-made :cpp:class:
ValueInfoProtoand returns its name.
-
const std::string &MakeInput(const std::string &name, const SymTensor &type)#
Declares a graph input described by
typeand returns its name.
-
const std::string &MakeInput(const std::string &name, TensorType dtype, const SymShape &shape)#
Declares a graph input with element type
dtypeand shapeshape.
-
void MakeOutput(const ValueInfoProto &value_info)#
Declares a graph output from a ready-made :cpp:class:
ValueInfoProto.
-
void MakeOutput(const std::string &name, const SymTensor &type)#
Declares
name(which must already exist) as a graph output described bytype.
-
void MakeOutput(const std::string &name, TensorType dtype, const SymShape &shape)#
Declares
nameas a graph output with element typedtypeand shapeshape.
-
void MakeOutput(const std::string &name)#
Declares
nameas a graph output without a declared type; the inferred type is filled in by :cpp:func:ToModel/ :cpp:func:ToGraph.
-
inline const utils::RepeatedProtoField<ValueInfoProto> &Inputs() const noexcept#
Read-only access to the declared graph inputs (in declaration order).
-
inline const utils::RepeatedProtoField<ValueInfoProto> &Outputs() const noexcept#
Read-only access to the declared graph outputs (in declaration order).
-
std::vector<std::string> MakeNode(const std::string &op_type, const std::vector<std::string> &inputs, const std::vector<std::string> &outputs = {}, const std::string &domain = "", const std::string &name = "", const utils::RepeatedProtoField<AttributeProto> &attributes = utils::RepeatedProtoField<AttributeProto>())#
Appends a node to the graph.
The opset version of
domainis resolved (defaulting to the latest known one when unset), the node is validated against the matching :cpp:class:core::schema::LightOpSchemawhen a schema provider is available, missing output names are generated, the node is appended and incremental shape inference is run for it.- Parameters:
op_type – Operator type (e.g.
"Add").inputs – Input value names.
outputs – Output value names; empty entries (or a shorter list than the operator produces) are auto-generated.
domain – Operator domain (empty for the default ONNX domain).
name – Optional node name.
attributes – Node attributes.
- Returns:
The (possibly generated) output names of the node.
-
inline const utils::RepeatedProtoField<NodeProto> &Nodes() const noexcept#
Read-only access to the accumulated nodes (in insertion order).
-
std::size_t RemoveUnusedNodes()#
Removes dead-end (unused) nodes from the builder.
A node is kept only when at least one of its outputs is (transitively) needed to compute a declared graph output. The analysis is recursive: removing a node can turn the nodes that only fed it into dead ends as well, and the pruning descends into nested subgraphs and local functions to remove their own unused nodes. Values a subgraph consumes from the enclosing scope are treated as inputs of the owning control-flow node, so the producers a subgraph body relies on are kept alive.
- Returns:
The total number of nodes removed, including those pruned from nested subgraphs and local functions.
-
std::size_t RemoveDuplicateInitializers()#
Removes duplicated initializers from the builder.
Initializers that carry byte-for-byte identical content (same element type, shape and data, whether stored inline or as external data) are collapsed onto a single copy: the first occurrence is kept and every later duplicate is dropped. The content is compared field by field — element type and shape first, then the payload — without copying or serializing the tensors. All references to a dropped initializer — in this builder’s node inputs and in the node inputs of nested subgraphs, which capture values from the enclosing scope — are rewritten to the surviving initializer name. An initializer that also happens to be a declared graph output keeps its own name and is never dropped.
The deduplication spans the enclosing graph and its subgraphs: because a subgraph body sees the initializers of the enclosing scope, a subgraph initializer that duplicates one visible from an enclosing graph is dropped and its references rewritten to that enclosing initializer. Local functions have an isolated scope and are deduplicated on their own.
- Returns:
The total number of initializers removed, including those pruned from nested subgraphs and local functions.
-
std::size_t RemoveIdentityNodes()#
Removes :onnx:
Identitynodes from the builder.Every default-domain
Identitynode simply forwards its single input to its single output. Such a node is dropped and every reference to its output — in this builder’s node inputs and in the node inputs of nested subgraphs, which capture values from the enclosing scope — is rewritten to the node’s input. Chains of identities are collapsed transitively, so a value that flowed through several identities ends up pointing at the original producer in a single pass.An
Identitywhose output is a declared graph output is kept, because the graph must still produce a value under that name. Nodes with an empty input or output name are left untouched.The removal is recursive: it descends into nested subgraphs and local functions to remove their own identities as well.
- Returns:
The total number of
Identitynodes removed, including those pruned from nested subgraphs and local functions.
-
std::size_t RemoveDuplicateNodes()#
Removes duplicated nodes (common subexpressions) from the builder.
Two nodes are duplicates when they share the same operator type, domain, inputs and attributes and therefore compute the same value. The first occurrence is kept and every later duplicate is dropped; each reference to a dropped node’s output — in this builder’s node inputs and in the node inputs of nested subgraphs, which capture values from the enclosing scope — is rewritten to the surviving node’s matching output. Inputs are resolved against earlier-dropped duplicates while nodes are scanned in insertion (topological) order, so a whole duplicated branch collapses in a single pass: once a node’s producers point at the survivors, the node itself becomes a duplicate of the corresponding surviving node.
A node whose output is a declared graph output is kept, because the graph must still produce a value under that name (it can still act as the survivor for a later duplicate). Nodes referencing control-flow subgraphs carry a per-node unique subgraph name and are never merged.
The removal is recursive: it descends into nested subgraphs and local functions to collapse their own duplicates as well.
- Returns:
The total number of duplicated nodes removed, including those pruned from nested subgraphs and local functions.
-
std::size_t MoveShapeAndSizeNodes()#
Moves every :onnx:
Shapeand :onnx:Sizenode right after the node that produces the tensor it reads.A default-domain
ShapeorSizenode only inspects the metadata (shape / element count) of its single input, so it can run as soon as that input exists. Emitting it immediately after its producer — instead of wherever it happened to be inserted — lets the producer’s output be released as early as possible, which improves the peak-memory analysis run at finalisation. This pass is called automatically before the builder is exported into an ONNX proto.Each such node is relocated to directly follow the node producing its input, preserving the relative order of several
Shape/Sizenodes that share the same producer. A node whose input is a graph input or an initializer (i.e. it has no producing node) is left in place. Because the producer always precedes its consumer in the insertion (topological) order, a node is only ever moved earlier, so the result stays topologically valid.The pass is recursive: it descends into nested subgraphs and local functions to reorder their own
Shape/Sizenodes as well.- Returns:
The total number of
Shape/Sizenodes moved, including those moved in nested subgraphs and local functions.
-
std::size_t InlineLocalFunctions(const std::vector<std::pair<std::string, std::string>> &include = {}, const std::vector<std::pair<std::string, std::string>> &exclude = {})#
Inlines calls to local functions into the calling graph.
A node calls a local function when its operator type and domain match a local function declared on this builder. Every such call is replaced, in place, by a renamed copy of the function body: the function formal inputs are rewired to the call inputs, the formal outputs to the call outputs, and every other body value (node outputs, and any function initializers or control-flow subgraphs) is copied under a fresh, unused name so it can never collide with a value of the calling graph. Body-node attributes that reference a function attribute (
ref_attr_name) are resolved against the attributes carried by the call node, falling back to the operator default when the call leaves the attribute unset.The expansion runs to a fixed point, so a function that itself calls another local function is fully inlined in a single pass, and it descends into nested subgraphs, whose bodies may call the enclosing local functions too. Local function definitions that are no longer referenced once every call site has been inlined are dropped, so a fully inlined model carries no leftover function.
The set of functions to inline is selected by
(domain, name)pairs. A pair matches a local function when both its domain and name match; an empty domain matches every domain (all functions sharing the name) and an empty name matches every name (all functions in the domain), so an empty-empty pair matches every function. When bothincludeandexcludeare empty (the default) every local function is inlined. Whenincludeis non-empty only the functions matched by one of its pairs are inlined. Whenexcludeis non-empty every local function except those matched by one of its pairs is inlined. Passing a non-emptyincludetogether with a non-emptyexcludethrows aBuilderError.- Parameters:
include –
(domain, name)pairs of the only functions to inline; empty for all.exclude –
(domain, name)pairs of the functions to leave untouched.
- Returns:
The total number of call nodes that were inlined.
-
std::size_t ConstantFold(const ConstantFoldingOptions &options = {})#
Folds constant subgraphs into initializers.
A node is constant when every value it reads is constant (a graph initializer or, transitively, the output of an earlier constant node) and its operator is deterministic. Every such node is evaluated once through the process-wide runtime kernel registry (:cpp:func:
core::runtime::KernelDispatchTable, populated byonnx_kernels) and replaced by initializers carrying its computed outputs; the freshly materialized constants let the nodes that only fed it fold in the same pass.A node’s outputs are classified by their inferred value tag. Outputs tagged
"shape"(shape-carrying values, e.g. the output of :onnx:Shapeor a :onnx:Concatof shapes) must be foldable: when no kernel is registered for such a node a :cpp:class:BuilderErroris thrown. Every other constant node ("weight"or untagged) is folded only when a kernel is available; otherwise it is left untouched, unless :cpp:member:ConstantFoldingOptions::raise_on_missing_weight_kernelasks for a :cpp:class:BuilderErrorinstead.Folding is skipped for a node when it is listed in :cpp:member:
ConstantFoldingOptions::excluded_ops, when any of its outputs would exceed :cpp:member:ConstantFoldingOptions::max_element_countelements, or when it carries a control-flow subgraph. A node whose output is a declared graph output is still folded: the computed constant is materialized as an initializer carrying that name, which remains a valid graph output. Setting :cpp:member:ConstantFoldingOptions::enabledtofalseturns the whole pass into a no-op.The pass is recursive: it descends into nested subgraphs and local functions to fold their own constants as well. Values a subgraph captures from the enclosing scope are not seeded as constants there, so a subgraph node reading such a capture is left untouched.
- Parameters:
options – Folding options (enable flag, size threshold, op blacklist, strictness for missing weight kernels).
- Returns:
The total number of nodes folded away, including those folded in nested subgraphs and local functions.
-
GraphBuilder &MakeLocalFunction(const std::string &name, const std::string &domain = "")#
Creates and returns a nested builder for a local function named
name. The nested builder is appended to this builder’s local function list; local functions are emitted into the produced :cpp:class:ModelProto. Throws whennameis already used.
-
inline bool HasLocalFunction(const std::string &name) const#
Returns
truewhen a local function namednameexists.
-
inline GraphBuilder &LocalFunction(const std::string &name)#
Returns the nested local-function builder named
name. Throws when it does not exist.
-
inline const GraphBuilder &LocalFunction(const std::string &name) const#
-
inline const std::vector<std::unique_ptr<GraphBuilder>> &LocalFunctions() const noexcept#
Read-only access to the local function list, in declaration order.
-
GraphBuilder &MakeSubgraph(const std::string &name)#
Creates and returns a nested builder for a subgraph named
name(used as the body of a control-flow node such as :onnx:If, :onnx:Loopor :onnx:Scan). The nested builder is appended to this builder’s subgraph list. Throws whennameis already used.
-
inline bool HasSubgraph(const std::string &name) const#
Returns
truewhen a subgraph namednameexists.
-
inline GraphBuilder &Subgraph(const std::string &name)#
Returns the nested subgraph builder named
name. Throws when it does not exist.
-
inline const GraphBuilder &Subgraph(const std::string &name) const#
-
inline const std::vector<std::unique_ptr<GraphBuilder>> &Subgraphs() const noexcept#
Read-only access to the subgraph list, in declaration order.
-
bool HasShape(const std::string &name) const#
Returns
truewhen the shape ofnamehas been inferred.
-
const SymTensor &GetShape(const std::string &name) const#
Returns the inferred descriptor of
name. Throws when it is unknown.
-
GraphProto BuildGraph() const#
Assembles (without finalising) the accumulated inputs, initializers, nodes and outputs into a :cpp:class:
GraphProto.
-
inline const ComputeContext &Compute() const noexcept#
Read-only access to the owned :cpp:class:
ComputeContext.
-
inline const ShapesContext &Shapes() const noexcept#
Read-only access to the :cpp:class:
ShapesContextholding the inferred descriptors computed so far.
-
std::string ToString() const#
Returns a comprehensive, human-readable description of the current content of the builder: its name, resolved opsets, inputs, initializers, nodes, outputs and nested local functions / subgraphs.
-
inline void set_device(Device device) noexcept#
Logical device used for the peak-memory analysis run by the finalizers.
-
inline Device device() const noexcept#
-
GraphProto ToGraph()#
Returns the finalized :cpp:class:
GraphProto.
-
ModelProto ToModel(int64_t ir_version = 0)#
Returns the finalized graph wrapped in a :cpp:class:
ModelProto.- Parameters:
ir_version – IR version to write;
0selects the library default.
-
FunctionProto ToFunction(const std::string &domain = "")#
Returns the finalized nodes wrapped in a :cpp:class:
FunctionProto.- Parameters:
domain – Function domain.
Private Types
-
using InitializerContentIndex = std::unordered_map<int64_t, std::vector<const TensorProto*>>#
Private Functions
-
std::size_t RemoveIdentityNodesImpl(bool recursive, std::unordered_map<std::string, std::string> *applied_renames = nullptr)#
-
std::size_t RemoveDuplicateNodesImpl(bool recursive, std::unordered_map<std::string, std::string> *applied_renames = nullptr)#
-
std::size_t ConstantFoldNodes(const ConstantFoldingOptions &options, const std::unordered_set<std::string> &included_outputs)#
-
std::size_t ConstantFoldImpl(const ConstantFoldingOptions &options, const std::unordered_set<std::string> *included_outputs)#
-
const std::vector<LightOpSchema> &DomainSchemas(const std::string &op_type, const std::string &normalised_domain)#
-
std::vector<GraphBuilder*> ReferencedSubgraphs(const NodeProto &node) const#
-
std::size_t InlineFunctionCalls(const std::vector<GraphBuilder*> &functions)#
-
void AppendInlinedBody(GraphBuilder &function, const NodeProto &call, utils::RepeatedProtoField<NodeProto> &out)#
-
std::size_t DeduplicateInitializers(const InitializerContentIndex &inherited, bool recursive, std::unordered_map<std::string, std::string> *applied_renames = nullptr)#
-
void ImportGraph(const GraphProto &graph)#
-
void ImportFunction(const FunctionProto &function)#
-
utils::RepeatedProtoField<AttributeProto> ImportAttributes(const NodeProto &node, const std::unordered_set<std::string> &excluded_inherited_names = {})#
-
void Finalize(GraphProto &graph)#
Private Members
-
SchemaLookupFn schema_lookup_#
-
std::unordered_map<std::string, std::unordered_map<std::string, std::vector<LightOpSchema>>> schema_table_#
-
ComputeContext compute_#
-
utils::RepeatedProtoField<ValueInfoProto> inputs_#
-
utils::RepeatedProtoField<ValueInfoProto> outputs_#
-
utils::RepeatedProtoField<NodeProto> nodes_#
-
utils::RepeatedProtoField<TensorProto> initializers_#
-
std::vector<std::unique_ptr<GraphBuilder>> local_functions_#
-
std::vector<std::unique_ptr<GraphBuilder>> subgraphs_#
-
Device device_ = Device::kUndefined#
Private Static Functions
-
static GraphBuilder *FindCalledFunction(const std::vector<GraphBuilder*> &functions, const NodeProto &node)#
-
static GraphBuilder *FindNamedBuilder(const std::vector<std::unique_ptr<GraphBuilder>> &builders, const std::string &name)#
-
static GraphBuilder &NamedBuilderOrThrow(const std::vector<std::unique_ptr<GraphBuilder>> &builders, const std::string &name, const char *kind)#
Friends
- friend class GraphGraph
-
using SchemaLookupFn = std::function<std::vector<LightOpSchema>(const std::string &op_type)>#
-
enum class ProtoKind#
-
namespace builder#
-
namespace core