parser.h#

ONNX text-format parser declarations, including onnx::ParserBase (cursor-based tokenizer) and onnx::OnnxParser (builds protobuf structures from text). Also defines utility maps onnx::PrimitiveTypeNameMap, onnx::AttributeTypeNameMap, and onnx::KeyWordMap.

Declares the ONNX text-format parser and related helpers.

This header exposes ParserBase (cursor-based tokenizer) and OnnxParser (builds protobuf structures from text) for parsing ONNX models, graphs, functions, nodes, and types from their textual representation. It also defines utility maps (PrimitiveTypeNameMap, AttributeTypeNameMap, KeyWordMap) and convenience type aliases used throughout the parser.

Note

The ONNX text syntax is experimental and may change.

Defines

CHECK_PARSER_STATUS(status)#

namespace ONNX_LIGHT_NAMESPACE

Typedefs

using IdList = std::vector<std::string>#: List of identifiers used in parser productions.

using NodeList = utils::RepeatedProtoField<NodeProto>#: List of parsed node definitions.

using AttrList = utils::RepeatedProtoField<AttributeProto>#: List of parsed node attributes.

using ValueInfoList = utils::RepeatedProtoField<ValueInfoProto>#: List of value-info records used for graph/function signatures.

using TensorList = utils::RepeatedProtoField<TensorProto>#: List of tensor literals and initializers.

using OpsetIdList = utils::RepeatedProtoField<OperatorSetIdProto>#: List of opset imports.

using StringStringList = utils::RepeatedProtoField<StringStringEntryProto>#: List of key/value metadata entries.

class AttributeTypeNameMap : public ONNX_LIGHT_NAMESPACE::StringIntMap<AttributeTypeNameMap>#

#include <parser.h>

Maps ONNX attribute type name strings to AttributeProto::AttributeType values.

Covers scalar types (“float”, “int”, “string”, “tensor”, “graph”, “sparse_tensor”, “type_proto”) and their list counterparts (“floats”, “ints”, “strings”, “tensors”, “graphs”, “sparse_tensors”, “type_protos”).

Public Functions

inline AttributeTypeNameMap()#

class KeyWordMap#

#include <parser.h>

Singleton map from keyword identifier strings to KeyWord enum values.

Recognizes ONNX model-level keywords such as “ir_version”, “opset_import”, “domain”, “seq”, “map”, “optional”, “sparse_tensor”, and “overload”.

Public Types

enum class KeyWord : std::uint8_t#

Enumeration of all reserved ONNX text-format keywords.

Values:

enumerator NONE#

enumerator IR_VERSION#

enumerator OPSET_IMPORT#

enumerator PRODUCER_NAME#

enumerator PRODUCER_VERSION#

enumerator DOMAIN_KW#

enumerator MODEL_VERSION#

enumerator DOC_STRING#

enumerator METADATA_PROPS#

enumerator SEQ_TYPE#

enumerator MAP_TYPE#

enumerator OPTIONAL_TYPE#

enumerator SPARSE_TENSOR_TYPE#

enumerator OVERLOAD_KW#

Public Functions

inline KeyWordMap()#: Constructs the keyword map and populates all reserved identifier entries.

Public Static Functions

static const std::unordered_map<std::string, KeyWord> &Instance()#: Returns the singleton string-to-keyword map instance.

static inline KeyWord Lookup(const std::string &id)#: Looks up a keyword by identifier string; returns KeyWord::NONE if not found.

static const std::string &ToString(KeyWord kw)#: Converts a KeyWord enum value to its canonical string representation.

Private Members

std::unordered_map<std::string, KeyWord> map_#

class OnnxParser : public ONNX_LIGHT_NAMESPACE::ParserBase #

#include <parser.h>

High-level ONNX text-format parser that builds protobuf structures.

Inherits from ParserBase and provides overloaded Parse() methods for every major ONNX protobuf type (ModelProto, GraphProto, FunctionProto, NodeProto, TensorProto, TypeProto, AttributeProto, etc.).

The most convenient entry point is the static Parse() helper:

ModelProto model;
auto status = OnnxParser::Parse(model, text_buffer);

Public Functions

inline explicit OnnxParser(const char *cstr)#: Creates an ONNX text parser from a null-terminated buffer.

Common::Status Parse(TensorShapeProto &shape)#: Parses a TensorShapeProto.

Common::Status Parse(TypeProto &typeProto)#: Parses a TypeProto.

Common::Status Parse(StringStringList &stringStringList)#: Parses a metadata key/value list.

Common::Status Parse(TensorProto &tensorProto)#: Parses a TensorProto.

Common::Status Parse(AttributeProto &attr)#: Parses an AttributeProto.

Common::Status Parse(AttributeProto &attr, std::string &name)#: Parses a named AttributeProto.

Common::Status Parse(AttrList &attrlist)#: Parses a comma-separated attribute list.

Common::Status Parse(NodeProto &node)#: Parses a NodeProto.

Common::Status Parse(NodeList &nodelist)#: Parses a list of nodes.

Common::Status Parse(GraphProto &graph)#: Parses a GraphProto.

Common::Status Parse(FunctionProto &fn)#: Parses a FunctionProto.

Common::Status Parse(ModelProto &model)#: Parses a ModelProto.

Public Static Functions

template<typename T> static inline Common::Status Parse(T &parsedData, const char *input)#: Parses an ONNX text snippet into the requested protobuf type.

Private Functions

Common::Status Parse(std::string name, GraphProto &graph)#

Common::Status Parse(IdList &idlist)#

Common::Status Parse(char open, IdList &idlist, char close)#

Common::Status Parse(IdList &idlist, AttrList &attrlist)#

Common::Status Parse(char open, IdList &idlist, AttrList &attrlist, char close)#

Common::Status ParseSingleAttributeValue(AttributeProto &attr, AttributeProto::AttributeType expected)#

Common::Status Parse(ValueInfoProto &valueinfo)#

Common::Status ParseGraphInputOutput(ValueInfoList &vilist)#

Common::Status ParseFunctionInputOutput(IdList &idlist, ValueInfoList &vilist)#

Common::Status Parse(char open, ValueInfoList &vilist, char close)#

Common::Status ParseInput(ValueInfoList &inputs, TensorList &initializers)#

Common::Status ParseValueInfo(ValueInfoList &value_infos, TensorList &initializers)#

Common::Status Parse(TensorProto &tensorProto, const TypeProto &tensorTypeProto)#

Common::Status Parse(OpsetIdList &opsets)#

bool NextIsType()#

bool NextIsIdentifier()#

class ParserBase#

#include <parser.h>

Cursor-based tokenizer that drives parsing of ONNX text format.

Maintains a read cursor over a character buffer and provides low-level helpers for skipping whitespace, matching characters, reading literals (integer, float, string), and parsing identifiers. OnnxParser inherits from this class to build higher-level protobuf parsing on top.

Subclassed by ONNX_LIGHT_NAMESPACE::OnnxParser

Public Types

enum class LiteralType : std::uint8_t#

Classifies the kind of a parsed literal token.

Values:

enumerator UNDEFINED#

enumerator INT_LITERAL#

enumerator FLOAT_LITERAL#

enumerator STRING_LITERAL#

Public Functions

inline explicit ParserBase(const std::string &str)#: Creates a parser from a string buffer. The underlying buffer must remain valid and unchanged during the parser lifetime.

inline explicit ParserBase(const char *cstr)#: Creates a parser from a null-terminated string. The referenced character buffer must outlive the parser instance.

inline void SavePos()#: Saves the current parser cursor position.

inline void RestorePos()#: Restores the parser cursor to the most recently saved position.

inline std::string GetCurrentPos()#: Returns the current parser position as (line, column).

inline std::string GetErrorContext()#: Returns a line of source context around the current cursor position for error messages.

template<typename ...Args> inline Common::Status ParseError(const Args&... args)#: Generates a parse error with current position and local context.

inline void SkipWhiteSpace()#: Advances the cursor past whitespace characters and #-prefixed line comments.

inline int NextChar(bool skipspace = true)#

Returns the next character without consuming it; returns 0 at end of input.

Parameters:: skipspace – When true (default), skips whitespace before peeking.

inline bool Matches(char ch, bool skipspace = true)#

Consumes ch if it is the next character and returns true; otherwise returns false.

Parameters:: skipspace – When true (default), skips whitespace before matching.

inline Common::Status Match(char ch, bool skipspace = true)#

Consumes ch or returns a parse error if it is not the next character.

Parameters:: skipspace – When true (default), skips whitespace before matching.

inline bool EndOfInput()#: Returns true when all remaining input (after skipping whitespace) has been consumed.

Common::Status Parse(Literal &result)#: Parses the next scalar literal token.

inline Common::Status Parse(int64_t &val)#: Parses a required integer literal into int64_t. Returns a parse error if the next token is not an integer literal.

inline Common::Status Parse(uint64_t &val)#: Parses a required integer literal into uint64_t. Returns a parse error if the next token is not an integer literal.

inline Common::Status Parse(float &val)#: Parses an integer or floating-point literal as float.

inline Common::Status Parse(double &val)#: Parses an integer or floating-point literal as double.

inline Common::Status Parse(std::string &val)#: Parses a double-quoted string literal; returns a parse error if a string is not found.

inline std::string ParseOptionalIdentifier()#: Parses an optional identifier (including keywords); returns an empty string if none found.

inline Common::Status ParseIdentifier(std::string &id)#: Parses an identifier and fails if none is found.

inline Common::Status ParseQuotableIdentifier(std::string &id)#: Parses a quoted identifier or an unquoted identifier.

inline Common::Status ParseOptionalQuotableIdentifier(std::string &id)#: Parses an optional identifier that may be enclosed in double quotes. Sets id to the identifier text (possibly empty) and returns OK.

inline Common::Status ParseOptionalQuotableIdentifier(std::string &id, bool &id_found)#

Parses an optional quotable identifier and sets id_found to indicate whether one was found.

An empty string followed by a comma is treated as a valid but empty identifier to support operand-list patterns such as Op(,x) (two operands, first empty) versus Op() (no operands). A trailing comma after a non-empty identifier is silently accepted. Using "" as an explicit empty identifier is preferred over relying on this behavior.

inline std::string PeekIdentifier()#: Returns the next identifier without advancing the cursor.

inline Common::Status Parse(KeyWordMap::KeyWord &keyword)#: Parses a keyword token.

Protected Functions

bool NextIsValidFloatString()#: Returns true if the characters at the current position form a valid floating-point literal.

Protected Attributes

const char *start_#: Pointer to the beginning of the input buffer.

const char *next_#: Pointer to the current read position within the input buffer.

const char *end_#: Pointer to one past the last character of the input buffer.

const char *saved_pos_#: Cursor snapshot written by SavePos() and restored by RestorePos().

struct Literal#

#include <parser.h>

Holds the raw text and classification of a parsed literal token.

Public Members

LiteralType type = {LiteralType::UNDEFINED}#

std::string value#

class PrimitiveTypeNameMap : public ONNX_LIGHT_NAMESPACE::StringIntMap<PrimitiveTypeNameMap>#

#include <parser.h>

Maps ONNX primitive type name strings to TensorProto::DataType values.

Supports all scalar element types such as “float”, “int64”, “bfloat16”, and the low-precision types “float8e4m3fn”, “uint4”, “float4e2m1”, etc.

Public Functions

inline PrimitiveTypeNameMap()#

Public Static Functions

static inline bool IsTypeName(const std::string &dtype)#: Returns true if the given string is a recognized ONNX primitive type name.

template<typename Map> class StringIntMap#

#include <parser.h>

CRTP singleton base that maps string names to integer codes.

Subclasses populate map_ in their constructor and gain static Instance(), Lookup(), and ToString() helpers automatically.

Template Parameters:: Map – Concrete subclass (CRTP pattern).

Subclassed by ONNX_LIGHT_NAMESPACE::AttributeTypeNameMap, ONNX_LIGHT_NAMESPACE::PrimitiveTypeNameMap

Public Static Functions

static inline const std::unordered_map<std::string, int32_t> &Instance()#: Returns the singleton name-to-value map instance.

static inline int32_t Lookup(const std::string &dtype)#: Finds a value by textual name.

static inline const std::string &ToString(int32_t dtype)#: Returns a textual name for an enum/integer value, or “undefined”.

Protected Attributes

std::unordered_map<std::string, int32_t> map_#