shape_text.h#

Shape-inference functions for ONNX operators in the text family.

namespace onnx_light

Alias that makes onnx-light headers compatible with code that references ONNX_LIGHT_NAMESPACE (the macro used in the standard onnx package).

Set to ONNX_LIGHT_NAMESPACE so both names resolve to the same namespace.

Symbol-visibility attribute for the public onnx-light C++ API.

Defined as empty because onnx-light does not require explicit __declspec(dllexport) or __attribute__((visibility("default"))) annotations — visibility is controlled at the shared-library level. The macro is provided so that vendored ONNX headers that decorate their declarations with ONNX_API compile without modification.

Namespace alias so that ONNX C++ code (and consumers such as onnxruntime) that refers to the literal onnx namespace — rather than the ONNX_NAMESPACE macro — resolves to the onnx-light namespace. The standard onnx package lives in namespace onnx; onnx-light uses onnx_light (via ONNX_LIGHT_NAMESPACE), so this alias keeps onnx-light a true drop-in. It is only introduced when the onnx-light namespace differs from onnx.

namespace onnx_optim
namespace shapes
namespace text#

Functions

void ComputeShapeStringConcat(ShapesContext &ctx, const NodeProto &node, const char *a, const char *b)#

Computes the output :cpp:class:OptimTensor of a StringConcat node and stores it in ctx.

StringConcat concatenates two string tensors element-wise with numpy-style multidirectional broadcasting (since opset 20 in the ai.onnx domain). The output dtype is always :cpp:enumerator:TensorType::kString and the output shape is the broadcast of the two input shapes.

Parameters:

  • ctx – In/out context. Must already contain entries for both a and b; on return it also contains an entry for node.output(0).

  • node – The StringConcat NodeProto whose output should be described. node.op_type() must be "StringConcat" and node must declare at least one output.

  • a – Name of the first input value to read from ctx.

  • b – Name of the second input value to read from ctx.

Throws:

  • std::invalid_argument – if node.op_type() is not "StringConcat", if node has no output, or if the two input shapes are not broadcast-compatible.

  • std::out_of_range – if either a or b is missing from ctx.

void ComputeShapeStringSplit(ShapesContext &ctx, const NodeProto &node, const char *a)#

Computes the output :cpp:class:OptimTensor descriptors of a StringSplit node and stores them in ctx.

StringSplit preserves the input rank for its Z output (substring counts, tensor(int64)) and appends one symbolic final dimension to the input rank for its Y output (padded substrings, tensor(string)).

Parameters:

  • ctx – In/out context. Must already contain an entry for a; on return it also contains entries for node.output(0) and node.output(1).

  • node – The StringSplit NodeProto whose outputs should be described. node.op_type() must be "StringSplit" and node must declare at least two outputs.

  • a – Name of the input value to read from ctx.

Throws:

  • std::invalid_argument – if node.op_type() is not "StringSplit", if node declares fewer than two outputs, or if either output name is empty.

  • std::out_of_range – if a is missing from ctx.

void ComputeShapeStringNormalizer(ShapesContext &ctx, const NodeProto &node, const char *a)#

Computes the output :cpp:class:OptimTensor of a StringNormalizer node and stores it in ctx.

StringNormalizer only accepts [C]- or [1, C]-shaped tensor(string) inputs (since opset 10 in the ai.onnx domain). The output dtype is always :cpp:enumerator:TensorType::kString and has the same rank as the input. The last dimension of the output is symbolic ("StringNormalizer(<input>)") because it depends on how many entries the stopwords attribute drops at runtime.

Parameters:

  • ctx – In/out context. Must already contain an entry for a; on return it also contains an entry for node.output(0).

  • node – The StringNormalizer NodeProto whose output should be described. node.op_type() must be "StringNormalizer" and node must declare at least one output.

  • a – Name of the input value to read from ctx.

Throws:

  • std::invalid_argument – if node.op_type() is not "StringNormalizer", if node has no output, or if the input shape has an unsupported rank.

  • std::out_of_range – if a is missing from ctx.

void ComputeShapeRegexFullMatch(ShapesContext &ctx, const NodeProto &node, const char *a)#

Computes the output :cpp:class:OptimTensor of a RegexFullMatch node and stores it in ctx.

RegexFullMatch (since opset 20 in the ai.onnx domain) performs an element-wise full-match regex test on a tensor(string) input and produces a tensor(bool) output of the same shape.

Parameters:

  • ctx – In/out context. Must already contain an entry for a; on return it also contains an entry for node.output(0).

  • node – The RegexFullMatch NodeProto whose output should be described. node.op_type() must be "RegexFullMatch" and node must declare at least one output.

  • a – Name of the input value to read from ctx.

Throws:

  • std::invalid_argument – if node.op_type() is not "RegexFullMatch" or if node has no output.

  • std::out_of_range – if a is missing from ctx.

void ComputeShapeTfIdfVectorizer(ShapesContext &ctx, const NodeProto &node, const char *a)#

Computes the output :cpp:class:OptimTensor of a TfIdfVectorizer node and stores it in ctx.

TfIdfVectorizer (since opset 9 in the ai.onnx domain) extracts n-grams from a [C]- or [N, C]-shaped integer / string input and produces a tensor(float) whose last dimension is max(ngram_indexes) + 1. The output preserves the input batch dimension when the input has rank 2.

Parameters:

  • ctx – In/out context. Must already contain an entry for a; on return it also contains an entry for node.output(0).

  • node – The TfIdfVectorizer NodeProto whose output should be described. node.op_type() must be "TfIdfVectorizer" and node must declare at least one output. The ngram_indexes attribute must be a non-empty list of non-negative int64s.

  • a – Name of the input value to read from ctx.

Throws:

  • std::invalid_argument – if node.op_type() is not "TfIdfVectorizer", if node has no output, if ngram_indexes is missing/invalid, or if the input shape has an unsupported rank.

  • std::out_of_range – if a is missing from ctx.