parser.h#

ONNX text-format parser declarations, including onnx_light::ParserBase (cursor-based tokenizer) and onnx_light::OnnxParser (builds protobuf structures from text). Also defines utility maps onnx_light::PrimitiveTypeNameMap, onnx_light::AttributeTypeNameMap, and onnx_light::KeyWordMap.

Declares the ONNX text-format parser and related helpers.

This header exposes ParserBase (cursor-based tokenizer) and OnnxParser (builds protobuf structures from text) for parsing ONNX models, graphs, functions, nodes, and types from their textual representation. It also defines utility maps (PrimitiveTypeNameMap, AttributeTypeNameMap, KeyWordMap) and convenience type aliases used throughout the parser.

Note

The ONNX text syntax is experimental and may change.

Defines

CHECK_PARSER_STATUS(status)#

namespace onnx_light

Alias that makes onnx-light headers compatible with code that references ONNX_LIGHT_NAMESPACE (the macro used in the standard onnx package).

Set to ONNX_LIGHT_NAMESPACE so both names resolve to the same namespace.

Symbol-visibility attribute for the public onnx-light C++ API.

Defined as empty because onnx-light does not require explicit __declspec(dllexport) or __attribute__((visibility("default"))) annotations — visibility is controlled at the shared-library level. The macro is provided so that vendored ONNX headers that decorate their declarations with ONNX_API compile without modification.

Namespace alias so that ONNX C++ code (and consumers such as onnxruntime) that refers to the literal onnx namespace — rather than the ONNX_NAMESPACE macro — resolves to the onnx-light namespace. The standard onnx package lives in namespace onnx; onnx-light uses onnx_light (via ONNX_LIGHT_NAMESPACE), so this alias keeps onnx-light a true drop-in. It is only introduced when the onnx-light namespace differs from onnx.

Typedefs

using IdList = std::vector<std::string>#: List of identifiers used in parser productions.

using NodeList = utils::RepeatedProtoField<NodeProto>#: List of parsed node definitions.

using AttrList = utils::RepeatedProtoField<AttributeProto>#: List of parsed node attributes.

using ValueInfoList = utils::RepeatedProtoField<ValueInfoProto>#: List of value-info records used for graph/function signatures.

using TensorList = utils::RepeatedProtoField<TensorProto>#: List of tensor literals and initializers.

using OpsetIdList = utils::RepeatedProtoField<OperatorSetIdProto>#: List of opset imports.

using StringStringList = utils::RepeatedProtoField<StringStringEntryProto>#: List of key/value metadata entries.

Functions

float LocaleIndependentStof(const std::string &s)#

double LocaleIndependentStod(const std::string &s)#

class AttributeTypeNameMap : public onnx_light::StringIntMap<AttributeTypeNameMap>#

#include <parser.h>

Maps ONNX attribute type name strings to AttributeProto::AttributeType values.

Covers scalar types (“float”, “int”, “string”, “tensor”, “graph”, “sparse_tensor”, “type_proto”) and their list counterparts (“floats”, “ints”, “strings”, “tensors”, “graphs”, “sparse_tensors”, “type_protos”).

Public Functions

inline AttributeTypeNameMap()#

class KeyWordMap#

#include <parser.h>

Singleton map from keyword identifier strings to KeyWord enum values.

Recognizes ONNX model-level keywords such as “ir_version”, “opset_import”, “domain”, “seq”, “map”, “optional”, “sparse_tensor”, and “overload”.

Public Types

enum class KeyWord : std::uint8_t#

Enumeration of all reserved ONNX text-format keywords.

Values:

enumerator NONE#

enumerator IR_VERSION#

enumerator OPSET_IMPORT#

enumerator PRODUCER_NAME#

enumerator PRODUCER_VERSION#

enumerator DOMAIN_KW#

enumerator MODEL_VERSION#

enumerator DOC_STRING#

enumerator METADATA_PROPS#

enumerator SEQ_TYPE#

enumerator MAP_TYPE#

enumerator OPTIONAL_TYPE#

enumerator SPARSE_TENSOR_TYPE#

enumerator OVERLOAD_KW#

Public Functions

inline KeyWordMap()#: Constructs the keyword map and populates all reserved identifier entries.

Public Static Functions

static const std::unordered_map<std::string, KeyWord> &Instance()#: Returns the singleton string-to-keyword map instance.

static inline KeyWord Lookup(const std::string &id)#: Looks up a keyword by identifier string; returns KeyWord::NONE if not found.

static const std::string &ToString(KeyWord kw)#: Converts a KeyWord enum value to its canonical string representation.

Private Members

std::unordered_map<std::string, KeyWord> map_#

class OnnxParser : public onnx_light::ParserBase #

#include <parser.h>

High-level ONNX text-format parser that builds protobuf structures.

Inherits from ParserBase and provides overloaded Parse() methods for every major ONNX protobuf type (ModelProto, GraphProto, FunctionProto, NodeProto, TensorProto, TypeProto, AttributeProto, etc.).

The most convenient entry point is the static Parse() helper:

ModelProto model;
auto status = OnnxParser::Parse(model, text_buffer);

Public Functions

Common::Status Parse(TensorShapeProto &shape)#: Parses a TensorShapeProto.

Common::Status Parse(TypeProto &typeProto)#: Parses a TypeProto.

Common::Status Parse(StringStringList &stringStringList)#: Parses a metadata key/value list.

Common::Status Parse(TensorProto &tensorProto)#: Parses a TensorProto.

Common::Status Parse(AttributeProto &attr)#: Parses an AttributeProto.

Common::Status Parse(AttributeProto &attr, std::string &name)#: Parses a named AttributeProto.

Common::Status Parse(AttrList &attrlist)#: Parses a comma-separated attribute list.

Common::Status Parse(NodeProto &node)#: Parses a NodeProto.

Common::Status Parse(NodeList &nodelist)#: Parses a list of nodes.

Common::Status Parse(GraphProto &graph)#: Parses a GraphProto.

Common::Status Parse(FunctionProto &fn)#: Parses a FunctionProto.

Common::Status Parse(ModelProto &model)#: Parses a ModelProto.

Common::Status Parse(Literal &result)#: Allows callers to parse literals directly and pass std::string_view without qualifying ParserBase explicitly.

inline Common::Status Parse(int64_t &val)#: Allows callers to parse literals directly and pass std::string_view without qualifying ParserBase explicitly.

inline Common::Status Parse(uint64_t &val)#: Allows callers to parse literals directly and pass std::string_view without qualifying ParserBase explicitly.

inline Common::Status Parse(float &val)#: Allows callers to parse literals directly and pass std::string_view without qualifying ParserBase explicitly.

inline Common::Status Parse(double &val)#: Allows callers to parse literals directly and pass std::string_view without qualifying ParserBase explicitly.

inline Common::Status Parse(std::string &val)#: Allows callers to parse literals directly and pass std::string_view without qualifying ParserBase explicitly.

inline Common::Status Parse(KeyWordMap::KeyWord &keyword)#: Allows callers to parse literals directly and pass std::string_view without qualifying ParserBase explicitly.

inline explicit ParserBase(std::string_view input)#: Creates a parser from a contiguous character buffer.

inline explicit ParserBase(const std::string &str)#: Creates a parser from a string buffer.

inline explicit ParserBase(const char *cstr)#: Creates a parser from a null-terminated string.

Public Static Functions

template<typename T> static inline Common::Status Parse(T &parsedData, std::string_view input)#: Parses an ONNX text snippet into the requested protobuf type.

Private Functions

Common::Status Parse(std::string name, GraphProto &graph)#

Common::Status Parse(IdList &idlist)#

Common::Status Parse(char open, IdList &idlist, char close)#

Common::Status Parse(IdList &idlist, AttrList &attrlist)#

Common::Status Parse(char open, IdList &idlist, AttrList &attrlist, char close)#

Common::Status ParseSingleAttributeValue(AttributeProto &attr, AttributeProto::AttributeType expected)#

Common::Status Parse(ValueInfoProto &valueinfo)#

Common::Status ParseGraphInputOutput(ValueInfoList &vilist)#

Common::Status ParseFunctionInputOutput(IdList &idlist, ValueInfoList &vilist)#

Common::Status Parse(char open, ValueInfoList &vilist, char close)#

Common::Status ParseInput(ValueInfoList &inputs, TensorList &initializers)#

Common::Status ParseValueInfo(ValueInfoList &value_infos, TensorList &initializers)#

Common::Status Parse(TensorProto &tensorProto, const TypeProto &tensorTypeProto)#

Common::Status Parse(OpsetIdList &opsets)#

bool NextIsType()#

bool NextIsIdentifier()#

class ParserBase#

#include <parser.h>

Cursor-based tokenizer that drives parsing of ONNX text format.

Maintains a read cursor over a character buffer and provides low-level helpers for skipping whitespace, matching characters, reading literals (integer, float, string), and parsing identifiers. OnnxParser inherits from this class to build higher-level protobuf parsing on top.

Subclassed by onnx_light::OnnxParser

Public Types

enum class LiteralType : std::uint8_t#

Classifies the kind of a parsed literal token.

Values:

enumerator UNDEFINED#

enumerator INT_LITERAL#

enumerator FLOAT_LITERAL#

enumerator STRING_LITERAL#

Public Functions

inline explicit ParserBase(std::string_view input)#: Creates a parser from a contiguous character buffer.

inline explicit ParserBase(const std::string &str)#: Creates a parser from a string buffer.

inline explicit ParserBase(const char *cstr)#: Creates a parser from a null-terminated string.

inline void SavePos()#: Saves the current parser cursor position.

inline void RestorePos()#: Restores the parser cursor to the most recently saved position.

inline std::string GetCurrentPos()#: Returns the current parser position as (line, column).

inline std::string GetErrorContext()#: Returns a line of source context around the current cursor position for error messages.

template<typename ...Args> inline Common::Status ParseError(const Args&... args)#: Generates a parse error with current position and local context.

inline void SkipWhiteSpace()#: Advances the cursor past whitespace characters and #-prefixed line comments.

inline int NextChar(bool skipspace = true)#

Returns the next character without consuming it; returns 0 at end of input.

Parameters:: skipspace – When true (default), skips whitespace before peeking.

inline bool Matches(char ch, bool skipspace = true)#

Consumes ch if it is the next character and returns true; otherwise returns false.

Parameters:: skipspace – When true (default), skips whitespace before matching.

inline Common::Status Match(char ch, bool skipspace = true)#

Consumes ch or returns a parse error if it is not the next character.

Parameters:: skipspace – When true (default), skips whitespace before matching.

inline bool EndOfInput()#: Returns true when all remaining input (after skipping whitespace) has been consumed.

Common::Status Parse(Literal &result)#: Parses the next scalar literal token.

inline Common::Status Parse(int64_t &val)#: Parses a required integer literal into int64_t. Returns a parse error if the next token is not an integer literal.

inline Common::Status Parse(uint64_t &val)#: Parses a required integer literal into uint64_t. Returns a parse error if the next token is not an integer literal.

inline Common::Status Parse(float &val)#: Parses an integer or floating-point literal as float.

inline Common::Status Parse(double &val)#: Parses an integer or floating-point literal as double.

inline Common::Status Parse(std::string &val)#: Parses a double-quoted string literal; returns a parse error if a string is not found.

inline std::string ParseOptionalIdentifier()#: Parses an optional identifier (including keywords); returns an empty string if none found.

inline Common::Status ParseIdentifier(std::string &id)#: Parses an identifier and fails if none is found.

inline Common::Status ParseQuotableIdentifier(std::string &id)#: Parses a quoted identifier or an unquoted identifier.

inline Common::Status ParseOptionalQuotableIdentifier(std::string &id)#: Parses an optional identifier that may be enclosed in double quotes. Sets id to the identifier text (possibly empty) and returns OK.

inline Common::Status ParseOptionalQuotableIdentifier(std::string &id, bool &id_found)#

Parses an optional quotable identifier and sets id_found to indicate whether one was found.

An empty string followed by a comma is treated as a valid but empty identifier to support operand-list patterns such as Op(,x) (two operands, first empty) versus Op() (no operands). A trailing comma after a non-empty identifier is silently accepted. Using "" as an explicit empty identifier is preferred over relying on this behavior.

inline std::string PeekIdentifier()#: Returns the next identifier without advancing the cursor.

inline Common::Status Parse(KeyWordMap::KeyWord &keyword)#: Parses a keyword token.

Protected Functions

inline bool AtEnd() const#: Returns true when the cursor has consumed all input.

inline char Cur() const#: Returns the character at the cursor; only valid when !AtEnd().

bool NextIsValidFloatString()#: Returns true if the characters at the current position form a valid floating-point literal.

Protected Attributes

std::string_view input_#

size_t pos_ = 0#

size_t saved_pos_ = 0#

Protected Static Functions

static inline bool IsSpace(char c)#: ctype wrapper that passes the character as unsigned char (UB otherwise for bytes > 127).

static inline bool IsAlpha(char c)#

static inline bool IsAlnum(char c)#

static inline bool IsDigit(char c)#

struct Literal#

#include <parser.h>

Holds the raw text and classification of a parsed literal token.

Public Members

LiteralType type = {LiteralType::UNDEFINED}#

std::string value#

class PrimitiveTypeNameMap : public onnx_light::StringIntMap<PrimitiveTypeNameMap>#

#include <parser.h>

Maps ONNX primitive type name strings to TensorProto::DataType values.

Supports all scalar element types such as “float”, “int64”, “bfloat16”, and the low-precision types “float8e4m3fn”, “uint4”, “float4e2m1”, etc.

Public Functions

inline PrimitiveTypeNameMap()#

Public Static Functions

static inline bool IsTypeName(const std::string &dtype)#: Returns true if the given string is a recognized ONNX primitive type name.

template<typename Map> class StringIntMap#

#include <parser.h>

CRTP singleton base that maps string names to integer codes.

Subclasses populate map_ in their constructor and gain static Instance(), Lookup(), and ToString() helpers automatically.

Template Parameters:: Map – Concrete subclass (CRTP pattern).

Public Static Functions

static inline const std::unordered_map<std::string, int32_t> &Instance()#: Returns the singleton name-to-value map instance.

static inline int32_t Lookup(const std::string &dtype)#: Finds a value by textual name.

static inline const std::string &ToString(int32_t dtype)#: Returns a textual name for an enum/integer value, or “undefined”.

Protected Attributes

std::unordered_map<std::string, int32_t> map_#