test_case.h#

Defines

ONNX_LIGHT_BACKEND_TEST_LOCAL#

namespace onnx_light

Alias that makes onnx-light headers compatible with code that references ONNX_LIGHT_NAMESPACE (the macro used in the standard onnx package).

Set to ONNX_LIGHT_NAMESPACE so both names resolve to the same namespace.

Symbol-visibility attribute for the public onnx-light C++ API.

Maps the upstream compatibility macro to onnx-light’s explicit proto ABI annotation. This keeps declarations from vendored ONNX headers visible when lib_onnx_proto uses hidden visibility by default.

Namespace alias so that ONNX C++ code (and consumers such as onnxruntime) that refers to the literal onnx namespace — rather than the ONNX_NAMESPACE macro — resolves to the onnx-light namespace. The standard onnx package lives in namespace onnx; onnx-light uses onnx_light (via ONNX_LIGHT_NAMESPACE), so this alias keeps onnx-light a true drop-in. It is only introduced when the onnx-light namespace differs from onnx.

namespace core

namespace backend_test

Typedefs

using RegisterCasesFn = void (*)(std::vector<TestCase>&)#: Function pointer registering one or more :ref:TestCase entries into the caller-supplied registry. Used by Collect*TestCases dispatch tables.

using OpRegisterMap = std::unordered_map<std::string_view, RegisterCasesFn>#: Per-category dispatch table: maps an op_type to the function that registers its test cases. Built once per Collect*TestCases as a function-local static const so lookup is amortised O(1).

using RegisterCasesModeFn = void (*)(std::vector<TestCase>&, TestMode)#: Mode-aware variant of :ref:RegisterCasesFn. In addition to the output registry it receives a :ref:TestMode selecting standard (TEST) or benchmark-sized (BENCHMARK) case generation.

using OpRegisterModeMap = std::unordered_map<std::string_view, RegisterCasesModeFn>#: Mode-aware variant of :ref:OpRegisterMap whose values are :ref:RegisterCasesModeFn.

Enums

enum class TestMode#

Selects how a Register*Cases / Collect* helper generates its cases.

TEST (the default) produces the standard correctness cases with small, fixed inputs. The generated cases are byte-for-byte unchanged from before this mode existed.
BENCHMARK produces cases whose inputs are enlarged so a single kernel evaluation processes enough elements to run long enough (~0.1 s) to be timed reliably. The exact sizes are hand-tuned per operator.

Values:

enumerator TEST#

enumerator BENCHMARK#

Functions

void InitModel(ModelProto &model, int64_t ir_version, const std::vector<OpsetId> &opset_imports, const std::string &producer_name = "backend-test")#: Initializes model with ir_version, producer_name and the given opset_imports (default ai.onnx domain when an entry’s domain is empty). Mirrors the boilerplate that opens every manually-built backend test case model so callers don’t have to repeat it.

void AppendValueInfo(ValueInfoProto &vi, const std::string &name, int32_t elem_type, const std::vector<int64_t> &shape)#: Fills vi with a tensor-typed ValueInfo (name, elem_type and the concrete dimension values from shape). Mirrors the boilerplate every manually-built graph repeats when declaring graph inputs / value_info / outputs for which a literal shape is already known (e.g. the gallery shapes used by the shape-inference cases). For Tensor-backed metadata see the FillValueInfo(const Tensor&, ValueInfoProto&) overload in simple_tensor.h.

void AppendValueInfo(ValueInfoProto &vi, const std::string &name, int32_t elem_type, const std::vector<DimSpec> &dims)#: Overload of :ref:AppendValueInfo accepting a mix of concrete (DimSpec(int64_t)), symbolic (DimSpec("name")) and unannotated (DimSpec()) dimensions. Used by the shape-inference cases to declare symbolic batch/seq/d_model/nnz dims without repeating the TypeProto::Tensor::add_shape() + add_dim() boilerplate.

void AppendValueInfo(ValueInfoProto &vi, const std::string &name, TensorProto::DataType elem_type, const std::vector<DimSpec> &dims)#: Overload of :ref:AppendValueInfo accepting a mix of concrete (DimSpec(int64_t)), symbolic (DimSpec("name")) and unannotated (DimSpec()) dimensions. Used by the shape-inference cases to declare symbolic batch/seq/d_model/nnz dims without repeating the TypeProto::Tensor::add_shape() + add_dim() boilerplate.

TypeSpec TensorTypeSpec(int32_t elem_type)#: Returns a TypeSpec describing a Tensor of elem_type with no declared shape (used e.g. for map value types).

TypeSpec TensorTypeSpec(int32_t elem_type, std::vector<int64_t> shape)#: Returns a TypeSpec describing a Tensor of elem_type whose declared shape has the given concrete dimension values (an empty shape declares a rank-0 / scalar shape).

TypeSpec SequenceTypeSpec(TypeSpec elem)#: Returns a TypeSpec describing a Sequence whose elements have type elem.

TypeSpec MapTypeSpec(int32_t key_type, TypeSpec value)#: Returns a TypeSpec describing a Map from key_type keys to value values.

void AppendValueInfo(ValueInfoProto &vi, const std::string &name, const TypeSpec &spec)#: Fills vi with name and the type described by spec.

void AppendDataSet(TestCase &tc, Tensors inputs, Tensors outputs)#: Appends a new DataSet to tc.data_sets() populated with the given inputs and outputs. Saves the DataSet ds; ds.inputs.push_back(...); ds.outputs.push_back(...); tc.data_sets().emplace_back(std::move(ds)); boilerplate that every manually-built TestCase otherwise repeats.

void DispatchRegisterByOpType(std::vector<TestCase> &registry, const std::string &op_type, const OpRegisterMap &entries)#: Invokes the Register*Cases functions declared in entries. When op_type is empty, every entry is invoked (order is unspecified). Otherwise, only the entry whose key matches op_type (case-sensitive) is invoked; if no entry matches, no registration occurs. Used by per-category Collect*TestCases helpers to dispatch via a hash map instead of an explicit if chain or linear scan.

void DispatchRegisterByOpType(std::vector<TestCase> &registry, const std::string &op_type, const OpRegisterModeMap &entries, TestMode mode)#: Mode-aware overload of :ref:DispatchRegisterByOpType. Forwards mode to each invoked :ref:RegisterCasesModeFn so a category can generate either the standard (TestMode::TEST) or benchmark-sized (TestMode::BENCHMARK) cases.

std::vector<TestCase> CollectTestCases(const std::string &op_type = "", bool include_big = false, TestMode mode = TestMode::TEST)#

Collects all C++-implemented backend test node cases. Each call is deterministic and independent: the result owns its ModelProtos and Tensor data.

Iterates the collector functions registered via :func:RegisterTestCasesCollector (typically every Collect*TestCases category in lib_onnx_backend_test).

Parameters:

op_type – Optional operator type filter. When non-empty, only test cases whose top-level graph contains a node with this op_type are returned.
include_big – When false (the default), test cases whose name contains the substring "_big_" are excluded from the result. Pass true to include them; the big models are intentionally opt-in because they carry large weight tensors that make exhaustive test loops slow.
mode – When :cpp:enumerator:TestMode::BENCHMARK, categories that support it emit benchmark-sized cases (large inputs) instead of the standard correctness cases. Defaults to :cpp:enumerator:TestMode::TEST.

Returns:

A fresh registry of test cases (Abs, Add equal-shape, Add scalar broadcast).

std::vector<TestCase> CollectTestCasesByName(const std::string &name_regex, bool include_big = false, TestMode mode = TestMode::TEST)#

Collects C++-implemented backend test node cases whose :attr:TestCase::name matches a regular expression. Uses std::regex_search semantics (substring match by default; anchor with ^...$ to require a full match).

Parameters:

name_regex – ECMAScript regular expression matched against each test case name. An empty string matches every case (equivalent to :func:CollectTestCases).
include_big – When false (the default), test cases whose name contains "_big_" are excluded before the regex filter is applied. Pass true to include them.
mode – Forwarded to :func:CollectTestCases; selects standard or benchmark-sized case generation.

Throws:

std::regex_error – if name_regex is not a valid ECMAScript regular expression.

Returns:

The subset of cases whose name matches name_regex, in the same registration order as :func:CollectTestCases.

std::vector<TestCase> GetTestCaseByName(const std::string &name, bool include_big = false, TestMode mode = TestMode::TEST)#

Returns the single C++-implemented backend test case whose :attr:TestCase::name equals name exactly, or an empty optional if no such case exists.

Unlike :func:CollectTestCasesByName this performs a plain string comparison instead of compiling a std::regex and avoids copying all non-matching cases. It is therefore the most efficient way to retrieve a single known case by name.

Parameters:

name – The exact test case name to look up (e.g. "test_cc_abs").
include_big – When false (the default), a case whose name contains "_big_" is excluded even if it matches name.
mode – Forwarded to :func:CollectTestCases; selects standard or benchmark-sized case generation.

Returns:

A vector containing at most one :class:TestCase. An empty vector signals that no case with the requested name was found.

struct BuiltCase#

#include <test_case.h>

Product of a lazily-built :ref:TestCase: the single-node ModelProto together with its input/output data sets. A TestCase stores a builder returning this so that constructing the (potentially large) model and running the kernel that computes the expected outputs is deferred until a consumer actually needs them. Collecting a large family of cases (in particular the BENCHMARK cases whose inputs contain millions of elements) therefore stays cheap.

Public Members

ModelProto model#

std::vector<DataSet> data_sets#

struct DataSet#

#include <test_case.h>

A single (inputs, expected outputs) data set associated with a TestCase.

Public Members

Tensors inputs#

Tensors outputs#

std::vector<Map> maps#: Map-typed inputs keyed by the graph input name.

struct DimSpec#

#include <test_case.h>

Describes one tensor dimension entry used to build a ValueInfoProto.

DimSpec(int64_t v) (v >= 0): concrete dim_value.
DimSpec("name") / DimSpec(std::string): symbolic dim_param.
DimSpec(): unannotated dim (neither dim_value nor dim_param).

Public Functions

DimSpec() = default#

inline DimSpec(int v)#

inline DimSpec(int64_t v)#

inline DimSpec(const char *p)#

inline DimSpec(std::string p)#

Public Members

int64_t value = -1#

std::string param#

struct TestCase#

#include <test_case.h>

A backend test case mirroring onnx_light.backend.test.case.base.TestCase. It bundles a single-node ModelProto together with the expected input/ output data sets a runtime must reproduce.

The model is not stored directly. Every case built through :func:Expect (both the correctness TEST cases and the BENCHMARK cases) is lazy: it carries a build closure that produces the :ref:BuiltCase — the ModelProto and its data_sets — on first access via :func:model / :func:data_sets / :func:Materialize. A handful of manually-assembled cases (control-flow, sequence, …) are instead eager: they populate the model cache with :func:emplace_model / :func:set_model and append their data sets directly, so build is left unset and :func:Materialize is a no-op. Every case records declared_input_element_counts / declared_output_element_counts so its sizing can be checked without running the (potentially expensive) builder.

The string-typed fields (name, model_name, kind, tag) are declared const and must therefore be supplied at construction time. tag is an optional, free-form label used to group families of cases (e.g. "empty_shape", "nan_inf", "inference"); it defaults to the empty string for the ordinary node cases in the default ai.onnx domain. For test cases whose underlying node belongs to a non-default operator domain (e.g. "ai.onnx.ml", "ai.onnx.preview.training"), :func:Expect defaults the tag to the node’s domain string when the caller does not provide an explicit one.

Public Functions

inline TestCase()#

inline explicit TestCase(std::string name_, std::string model_name_ = "", std::string kind_ = "node", std::string tag_ = "", double atol_ = 1e-7, double rtol_ = 1e-3)#

inline TestCase(TestCase &&other) noexcept#

TestCase(const TestCase&) = delete#

TestCase &operator=(const TestCase&) = delete#

TestCase &operator=(TestCase&&) = delete#

inline ModelProto &emplace_model()#: Creates (if needed) and returns the mutable model cache. Used by eager case builders that populate the ModelProto in place. Clears any previously-built cache.

inline void set_model(ModelProto model)#: Stores an already-built model into the cache.

inline bool materialized() const#: Returns whether the case has already been materialized (its model cache exists). Introspection helper that does not trigger materialization.

inline bool is_lazy() const#: Returns whether the case is lazy (carries a build closure). Does not trigger materialization.

inline void Materialize()#: Runs the build closure once (if the case is lazy and not yet built), materializing the model cache and data_sets. No-op for eager cases and for already-materialized cases.

inline ModelProto &model()#: Lazily builds (once) and returns the model.

inline const ModelProto &model() const#: Const overload. Materializes the case (model and data sets) on first access via the same builder, so data_sets is consistent in const contexts as well.

inline std::vector<DataSet> &data_sets()#: Lazily builds (once) and returns the mutable data sets. Eager producers also use this to append their data sets (build is unset, so materialization is a no-op).

inline const std::vector<DataSet> &data_sets() const#: Const overload. Materializes the case on first access.

Public Members

const std::string name#

const std::string model_name#

const std::string kind#

const std::string tag#

double rtol = 1e-3#

double atol = 1e-7#

std::function<BuiltCase()> build#: Optional builder producing the model + data sets on demand. When set the case is lazy: data_sets starts empty and the model is unbuilt until :func:Materialize / :func:model / :func:data_sets runs the builder once.

std::vector<int64_t> declared_input_element_counts#: Declared element count of each input/output, recorded without materializing tensor data. Used to validate the sizing of a case (in particular the large benchmark cases) without running build.

std::vector<int64_t> declared_output_element_counts#

Private Functions

inline void EnsureMaterialized() const#

Private Members

mutable std::vector<DataSet> data_sets_#: Data sets. Empty until the build closure has run (for lazy cases) or until an eager producer appends them directly via :func:data_sets.

mutable std::unique_ptr<ModelProto> model_#

struct TypeSpec#

#include <test_case.h>

Describes an ONNX value type for a graph value-info, supporting the container kinds the backend test cases need: a plain Tensor, a Sequence of an element type, or a Map from a key type to a value type. Built via the factory helpers :func:TensorTypeSpec, :func:SequenceTypeSpec and :func:MapTypeSpec and consumed by :func:AppendValueInfo / :func:Expect to emit value-infos whose declared schema type differs from the materialized Tensor representation (e.g. sequence- or map-valued outputs).

Public Types

enum class Kind#

Values:

enumerator kTensor#

enumerator kSequence#

enumerator kMap#

Public Members

Kind kind = Kind::kTensor #

int32_t elem_type = 0#: For kTensor: the tensor element type. For kMap: the key type.

bool has_shape = false#: For kTensor only: whether a (possibly empty) shape is declared.

std::vector<int64_t> shape#: For kTensor only: the concrete dimension values of the shape.

std::vector<TypeSpec> children#: Nested element type. For kSequence the single sequence element type, for kMap the single map value type; empty for kTensor.