Shape-inference coverage (onnx_optim)

This page reports, for every backend test case tagged "inference" (see onnx_light.onnx.backend.TestCase.tag), the outcome of running the onnx_optim shape-inference pipeline against the expected intermediate and output shapes recorded by the test author.

For every collected case, the report:

1. renders the original model — including the expected value_info and output shapes — as a compact text listing via pretty_onnx();

2. clones the model and strips its ``graph.value_info`` so shape inference cannot just reuse the recorded intermediate shapes;

3. runs onnx_light.onnx_optim.shape_inference.infer_shapes_model() on the stripped clone;

4. emits a side-by-side table contrasting the expected and the computed shape of every input, intermediate and output value, with the match status colored yes / no.

When shape inference itself raises, the error message is shown instead of the comparison table.

---

Summary

Overall pass rate across all "inference"-tagged backend test cases:

Scenario	Passed	Total	Pass rate
onnx_optim shape inference	28	28	100.0%

---

Per-case details

test_cc_shape_inference_add_concat_reshape

opset: domain='' version=18
graph: name='test_cc_shape_inference_add_concat_reshape'
input: float[batch,seq,d_model] X
input: float[batch,seq,d_model] Y
init: int64[3] reshape_shape
0: Add(X, Y) -> added
1: Concat(added, X) -> concat_out
2: Reshape(concat_out, reshape_shape) -> Z_pre_abs
3: Abs(Z_pre_abs) -> Z
output: float[batch,seq,2*d_model] Z

Name	Role	Expected	Computed	Match
X	input	FLOAT[batch, seq, d_model]	FLOAT[batch, seq, d_model]	yes
Y	input	FLOAT[batch, seq, d_model]	FLOAT[batch, seq, d_model]	yes
Z_pre_abs	value_info	FLOAT[batch, seq, 2*d_model]	FLOAT[batch, seq, 2*d_model]	yes
added	value_info	FLOAT[batch, seq, d_model]	FLOAT[batch, seq, d_model]	yes
concat_out	value_info	FLOAT[batch, seq, 2*d_model]	FLOAT[batch, seq, 2*d_model]	yes
Z	output	FLOAT[batch, seq, 2*d_model]	FLOAT[batch, seq, 2*d_model]	yes

test_cc_shape_inference_check_shape

opset: domain='' version=18
graph: name='test_cc_shape_inference_check_shape'
input: float[D32,D64] X
input: float[batch,channel,D128,D64] Y
init: int64[1] zero
init: int64[1] un
init: int64[1] c0
init: int64[1] cm1
0: Shape(X) -> x_last_dim
1: Concat(c0, cm1, x_last_dim) -> shape1
2: Shape(Y) -> y_dim2
3: Concat(cm1, x_last_dim, y_dim2) -> shape2
4: Shape(Y) -> y_first2
5: Shape(Y) -> y_last_dim
6: Concat(y_first2, x_last_dim, y_last_dim) -> shape3
7: Unsqueeze(X, zero) -> xu1
8: Unsqueeze(xu1, un) -> xu2
9: Reshape(xu2, shape1) -> xm1
10: Reshape(Y, shape2) -> xm2c
11: Cast(xm2c) -> xm2
12: MatMul(xm1, xm2) -> xm
13: Reshape(xm, shape3) -> Z
output: float[batch,channel,D64,D64] Z

Name	Role	Expected	Computed	Match
X	input	FLOAT[D32, D64]	FLOAT[D32, D64]	yes
Y	input	FLOAT[batch, channel, D128, D64]	FLOAT[batch, channel, D128, D64]	yes
x_last_dim	value_info	INT64[1]	INT64[1]	yes
y_dim2	value_info	INT64[1]	INT64[1]	yes
y_first2	value_info	INT64[2]	INT64[2]	yes
y_last_dim	value_info	INT64[1]	INT64[1]	yes
shape1	value_info	INT64[3]	INT64[3]	yes
shape2	value_info	INT64[3]	INT64[3]	yes
shape3	value_info	INT64[4]	INT64[4]	yes
xu1	value_info	FLOAT[1, D32, D64]	FLOAT[1, D32, D64]	yes
xu2	value_info	FLOAT[1, 1, D32, D64]	FLOAT[1, 1, D32, D64]	yes
xm1	value_info	FLOAT[1, D32, D64]	FLOAT[1, D32, D64]	yes
xm2c	value_info	FLOAT[batch*channel, D64, D128]	FLOAT[batch*channel, D64, D128]	yes
xm2	value_info	FLOAT[batch*channel, D64, D128]	FLOAT[batch*channel, D64, D128]	yes
xm	value_info	FLOAT[batch*channel, D32, D128]	FLOAT[batch*channel, D32, D128]	yes
Z	output	FLOAT[batch, channel, D64, D64]	FLOAT[batch, channel, D64, D64]	yes

test_cc_shape_inference_concat_split_even

opset: domain='' version=18
graph: name='test_cc_shape_inference_concat_split_even'
input: float[a,2*b] X
input: float[a,2*c] Y
0: Concat(X, Y) -> xy
1: Split(xy) -> S1, S2
2: Concat(S2, S1) -> zs
3: Relu(zs) -> Z
output: float[a,2*b+2*c] Z

Name	Role	Expected	Computed	Match
X	input	FLOAT[a, 2*b]	FLOAT[a, 2*b]	yes
Y	input	FLOAT[a, 2*c]	FLOAT[a, 2*c]	yes
xy	value_info	FLOAT[a, 2*b+2*c]	FLOAT[a, 2*b+2*c]	yes
S1	value_info	FLOAT[a, b+c]	FLOAT[a, b+c]	yes
S2	value_info	FLOAT[a, b+c]	FLOAT[a, b+c]	yes
zs	value_info	FLOAT[a, 2*b+2*c]	FLOAT[a, 2*b+2*c]	yes
Z	output	FLOAT[a, 2*b+2*c]	FLOAT[a, 2*b+2*c]	yes

test_cc_shape_inference_concat_split_odd

opset: domain='' version=18
graph: name='test_cc_shape_inference_concat_split_odd'
input: float[a,b] X
input: float[a,c] Y
0: Concat(X, Y) -> xy
1: Split(xy) -> S1, S2
2: Concat(S2, S1) -> zs
3: Relu(zs) -> Z
output: float[a,b+c] Z

Name	Role	Expected	Computed	Match
X	input	FLOAT[a, b]	FLOAT[a, b]	yes
Y	input	FLOAT[a, c]	FLOAT[a, c]	yes
xy	value_info	FLOAT[a, b+c]	FLOAT[a, b+c]	yes
S1	value_info	FLOAT[a, (1+b+c)//2]	FLOAT[a, (1+b+c)//2]	yes
S2	value_info	FLOAT[a, (b+c)//2]	FLOAT[a, (b+c)//2]	yes
zs	value_info	FLOAT[a, b+c]	FLOAT[a, b+c]	yes
Z	output	FLOAT[a, b+c]	FLOAT[a, b+c]	yes

test_cc_shape_inference_floordiv_offset_expression

opset: domain='' version=18
graph: name='test_cc_shape_inference_floordiv_offset_expression'
input: float[batch,channel,seq] X
0: MaxPool(X) -> Y
output: float[batch,channel,seq//5+2] Y

Name	Role	Expected	Computed	Match
X	input	FLOAT[batch, channel, seq]	FLOAT[batch, channel, seq]	yes
Y	output	FLOAT[batch, channel, seq//5+2]	FLOAT[batch, channel, seq//5+2]	yes

test_cc_shape_inference_gather_value_as_shape

opset: domain='' version=20
graph: name='test_cc_shape_inference_gather_value_as_shape'
input: float[N,D] x
input: float[1] y
init: int64[1] idx
0: Shape(x) -> shape_x
1: Gather(shape_x, idx) -> n_vec
2: Expand(y, n_vec) -> expanded
3: Abs(expanded) -> z
output: float[N] z

Name	Role	Expected	Computed	Match
x	input	FLOAT[N, D]	FLOAT[N, D]	yes
y	input	FLOAT[1]	FLOAT[1]	yes
shape_x	value_info	INT64[2]	INT64[2]	yes
n_vec	value_info	INT64[1]	INT64[1]	yes
expanded	value_info	FLOAT[N]	FLOAT[N]	yes
z	output	FLOAT[N]	FLOAT[N]	yes

test_cc_shape_inference_if_symbolic_shapes

opset: domain='' version=13
graph: name='test_cc_shape_inference_if_symbolic_shapes'
input: bool[] cond
input: float[3,4] a_then
input: float[5,4] a_else
input: bool[5] c_else
input: int64[3] b_then
input: int64[5] b_else
0: If(cond) -> I1, I2
1: Abs(I1) -> Y1
2: Neg(I2) -> Y2
output: float[B1,4] Y1
output: int64[B2] Y2

Name	Role	Expected	Computed	Match
cond	input	BOOL[]	BOOL[]	yes
a_then	input	FLOAT[3, 4]	FLOAT[3, 4]	yes
a_else	input	FLOAT[5, 4]	FLOAT[5, 4]	yes
c_else	input	BOOL[5]	BOOL[5]	yes
b_then	input	INT64[3]	INT64[3]	yes
b_else	input	INT64[5]	INT64[5]	yes
I1	value_info	FLOAT[B1, 4]	FLOAT[B1, 4]	yes
I2	value_info	INT64[B2]	INT64[B2]	yes
Y1	output	FLOAT[B1, 4]	FLOAT[B1, 4]	yes
Y2	output	INT64[B2]	INT64[B2]	yes

test_cc_shape_inference_local_function_add

opset: domain='' version=18
opset: domain='local' version=1
graph: name='test_cc_shape_inference_local_function_add'
input: float[batch,d_model] X
input: float[batch,d_model] Y
0: local.func_add(X, Y) -> Z_pre_abs
1: Abs(Z_pre_abs) -> Z
output: float[batch,d_model] Z

function: func_add[local]
input: a
input: b
0: Add(a, b) -> c
output: c

Name	Role	Expected	Computed	Match
X	input	FLOAT[batch, d_model]	FLOAT[batch, d_model]	yes
Y	input	FLOAT[batch, d_model]	FLOAT[batch, d_model]	yes
Z_pre_abs	value_info	FLOAT[batch, d_model]	FLOAT[batch, d_model]	yes
Z	output	FLOAT[batch, d_model]	FLOAT[batch, d_model]	yes

test_cc_shape_inference_loop_pairwise_distance

opset: domain='' version=18
graph: name='test_cc_shape_inference_loop_pairwise_distance'
input: float[batch,features] X
init: int64[1] zero_idx
init: int64[1] unsqueeze_axes
init: int64[1] reduce_axes
init: bool[] cond_init
0: Shape(X) -> shape_X
1: Gather(shape_X, zero_idx) -> trip_count
2: Loop(trip_count, cond_init) -> Y_pre_abs
3: Abs(Y_pre_abs) -> Y
output: float[batch,batch] Y

Name	Role	Expected	Computed	Match
X	input	FLOAT[batch, features]	FLOAT[batch, features]	yes
shape_X	value_info	INT64[2]	INT64[2]	yes
trip_count	value_info	INT64[1]	INT64[1]	yes
Y_pre_abs	value_info	FLOAT[batch, batch]	FLOAT[batch, batch]	yes
Y	output	FLOAT[batch, batch]	FLOAT[batch, batch]	yes

test_cc_shape_inference_loop_topk_pairwise_distance

opset: domain='' version=18
graph: name='test_cc_shape_inference_loop_topk_pairwise_distance'
input: float[N,D] X
input: int64[1] K
init: int64[1] zero_idx
init: int64[1] unsqueeze_axes
init: int64[1] reduce_axes
init: int64[1] mean_axes
init: bool[] cond_init
0: Shape(X) -> shape_X
1: Gather(shape_X, zero_idx) -> trip_count
2: Loop(trip_count, cond_init) -> dist
3: TopK(dist, K) -> topk_values, topk_indices
4: ReduceMean(topk_values, mean_axes) -> Y
output: float[N] Y

Name	Role	Expected	Computed	Match
X	input	FLOAT[N, D]	FLOAT[N, D]	yes
K	input	INT64[1]	INT64[1]	yes
shape_X	value_info	INT64[2]	INT64[2]	yes
trip_count	value_info	INT64[1]	INT64[1]	yes
dist	value_info	FLOAT[N, N]	FLOAT[N, N]	yes
topk_values	value_info	FLOAT[N, TopK_k]	FLOAT[N, TopK_k]	yes
topk_indices	value_info	INT64[N, TopK_k]	INT64[N, TopK_k]	yes
Y	output	FLOAT[N]	FLOAT[N]	yes

test_cc_shape_inference_nested_local_function_add

opset: domain='' version=18
opset: domain='local' version=1
graph: name='test_cc_shape_inference_nested_local_function_add'
input: float[batch,d_model] X
input: float[batch,d_model] Y
0: local.func_outer_add(X, Y) -> Z_pre_abs
1: Abs(Z_pre_abs) -> Z
output: float[batch,d_model] Z

function: func_inner_add[local]
input: a
input: b
0: Add(a, b) -> c
output: c

function: func_outer_add[local]
input: a
input: b
0: local.func_inner_add(a, b) -> c
output: c

Name	Role	Expected	Computed	Match
X	input	FLOAT[batch, d_model]	FLOAT[batch, d_model]	yes
Y	input	FLOAT[batch, d_model]	FLOAT[batch, d_model]	yes
Z_pre_abs	value_info	FLOAT[batch, d_model]	FLOAT[batch, d_model]	yes
Z	output	FLOAT[batch, d_model]	FLOAT[batch, d_model]	yes

test_cc_shape_inference_nonzero_chain_named

opset: domain='' version=18
graph: name='test_cc_shape_inference_nonzero_chain_named'
input: float[batch,seq] X
0: Abs(X) -> abs_out
1: Relu(abs_out) -> relu_out
2: Add(relu_out, relu_out) -> double_out
3: Mul(double_out, relu_out) -> mul_out
4: NonZero(mul_out) -> nz_pre_abs
5: Abs(nz_pre_abs) -> nz
6: Transpose(nz) -> transposed_nz
7: Cast(transposed_nz) -> nz_float_pre_abs
8: Abs(nz_float_pre_abs) -> nz_float
output: int64[2,do1] nz
output: float[do1,2] nz_float

Name	Role	Expected	Computed	Match
X	input	FLOAT[batch, seq]	FLOAT[batch, seq]	yes
abs_out	value_info	FLOAT[batch, seq]	FLOAT[batch, seq]	yes
relu_out	value_info	FLOAT[batch, seq]	FLOAT[batch, seq]	yes
double_out	value_info	FLOAT[batch, seq]	FLOAT[batch, seq]	yes
mul_out	value_info	FLOAT[batch, seq]	FLOAT[batch, seq]	yes
nz_pre_abs	value_info	INT64[2, do1]	INT64[2, do1]	yes
transposed_nz	value_info	INT64[do1, 2]	INT64[do1, 2]	yes
nz_float_pre_abs	value_info	FLOAT[do1, 2]	FLOAT[do1, 2]	yes
nz	output	INT64[2, do1]	INT64[2, do1]	yes
nz_float	output	FLOAT[do1, 2]	FLOAT[do1, 2]	yes

test_cc_shape_inference_nonzero_plus_expression

opset: domain='' version=18
graph: name='test_cc_shape_inference_nonzero_plus_expression'
input: float[batch,seq] X
init: int64[1] m1
0: Abs(X) -> abs_out
1: NonZero(abs_out) -> nz
2: Reshape(nz, m1) -> flat_nz
3: Neg(flat_nz) -> Y_pre_abs
4: Abs(Y_pre_abs) -> Y
output: int64[2*dnz] Y

Name	Role	Expected	Computed	Match
X	input	FLOAT[batch, seq]	FLOAT[batch, seq]	yes
abs_out	value_info	FLOAT[batch, seq]	FLOAT[batch, seq]	yes
nz	value_info	INT64[2, dnz]	INT64[2, dnz]	yes
flat_nz	value_info	INT64[2*dnz]	INT64[2*dnz]	yes
Y_pre_abs	value_info	INT64[2*dnz]	INT64[2*dnz]	yes
Y	output	INT64[2*dnz]	INT64[2*dnz]	yes

test_cc_shape_inference_pad_canny_average

opset: domain='' version=18
graph: name='test_cc_shape_inference_pad_canny_average'
input: float[N,1,H,W] X
init: float[1,1,3,3] W
init: int64[8] pads
init: int64[4] axes_mean
0: Pad(X, pads) -> padded
1: Conv(padded, W) -> filtered
2: ReduceMean(filtered, axes_mean) -> avg
3: Sub(filtered, avg) -> Y
output: float[N,1,H,W] Y

Name	Role	Expected	Computed	Match
X	input	FLOAT[N, 1, H, W]	FLOAT[N, 1, H, W]	yes
avg	value_info	FLOAT[1, 1, 1, 1]	FLOAT[1, 1, 1, 1]	yes
filtered	value_info	FLOAT[N, 1, H, W]	FLOAT[N, 1, H, W]	yes
padded	value_info	FLOAT[N, 1, H+2, W+2]	FLOAT[N, 1, H+2, W+2]	yes
Y	output	FLOAT[N, 1, H, W]	FLOAT[N, 1, H, W]	yes

test_cc_shape_inference_reshape_reshape

opset: domain='' version=18
graph: name='test_cc_shape_inference_reshape_reshape'
input: float[a,b,c] X
init: int64[4] shape1
init: int64[3] shape2
init: float[1] one
0: Reshape(X, shape1) -> xr
1: Reshape(xr, shape2) -> xrr
2: Add(xrr, one) -> Y
output: float[a,b,c] Y

Name	Role	Expected	Computed	Match
X	input	FLOAT[a, b, c]	FLOAT[a, b, c]	yes
xr	value_info	FLOAT[a, b, 2, c/:2]	FLOAT[a, b, 2, c/:2]	yes
xrr	value_info	FLOAT[a, b, c]	FLOAT[a, b, c]	yes
Y	output	FLOAT[a, b, c]	FLOAT[a, b, c]	yes

test_cc_shape_inference_resize_tile

opset: domain='' version=13
graph: name='test_cc_shape_inference_resize_tile'
input: float[H,2*h] X
init: float[2] scales
init: int64[2] repeats
init: float[] zeros_scalar
0: Resize(X, , scales) -> resized_out
1: Tile(resized_out, repeats) -> tile_out
2: Max(tile_out, zeros_scalar) -> output
output: float[2*(H//2),2*h] output

Name	Role	Expected	Computed	Match
X	input	FLOAT[H, 2*h]	FLOAT[H, 2*h]	yes
resized_out	value_info	FLOAT[H//2, h]	FLOAT[H//2, h]	yes
tile_out	value_info	FLOAT[2*(H//2), 2*h]	FLOAT[2*(H//2), 2*h]	yes
output	output	FLOAT[2*(H//2), 2*h]	FLOAT[2*(H//2), 2*h]	yes

test_cc_shape_inference_scan_running_sum

opset: domain='' version=18
graph: name='test_cc_shape_inference_scan_running_sum'
input: float[T,D] X
init: float[3] zero_acc
0: Scan(zero_acc, X) -> acc_final, Y_pre_abs
1: Abs(Y_pre_abs) -> Y
output: float[T,3] Y

Name	Role	Expected	Computed	Match
X	input	FLOAT[T, D]	FLOAT[T, D]	yes
acc_final	value_info	FLOAT[3]	FLOAT[3]	yes
Y_pre_abs	value_info	FLOAT[T, 3]	FLOAT[T, 3]	yes
Y	output	FLOAT[T, 3]	FLOAT[T, 3]	yes

test_cc_shape_inference_scan_topk_pairwise_distance

opset: domain='' version=18
graph: name='test_cc_shape_inference_scan_topk_pairwise_distance'
input: float[N,D] X
input: int64[1] K
init: int64[1] reduce_axes
init: int64[1] mean_axes
0: Scan(X, X) -> state_final, dist_sq
1: Sqrt(dist_sq) -> dist
2: TopK(dist, K) -> topk_values, topk_indices
3: ReduceMean(topk_values, mean_axes) -> Y
output: float[N] Y

Name	Role	Expected	Computed	Match
X	input	FLOAT[N, D]	FLOAT[N, D]	yes
K	input	INT64[1]	INT64[1]	yes
state_final	value_info	FLOAT[N, D]	FLOAT[N, D]	yes
dist_sq	value_info	FLOAT[N, N]	FLOAT[N, N]	yes
dist	value_info	FLOAT[N, N]	FLOAT[N, N]	yes
topk_values	value_info	FLOAT[N, TopK_k]	FLOAT[N, TopK_k]	yes
topk_indices	value_info	INT64[N, TopK_k]	INT64[N, TopK_k]	yes
Y	output	FLOAT[N]	FLOAT[N]	yes

test_cc_shape_inference_shape_identity_unsqueeze

opset: domain='' version=18
graph: name='test_cc_shape_inference_shape_identity_unsqueeze'
input: float[1,1,1,1,1,1,1,1,1,1,1,1,1,1,1] input
init: int64[15] unsq_axes
0: Shape(input) -> shape_out
1: Identity(shape_out) -> identity_out
2: Unsqueeze(identity_out, unsq_axes) -> output_pre_abs
3: Abs(output_pre_abs) -> output
output: int64[1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,15] output

Name	Role	Expected	Computed	Match
input	input	FLOAT[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]	FLOAT[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]	yes
shape_out	value_info	INT64[15]	INT64[15]	yes
identity_out	value_info	INT64[15]	INT64[15]	yes
output_pre_abs	value_info	INT64[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 15]	INT64[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 15]	yes
output	output	INT64[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 15]	INT64[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 15]	yes

test_cc_shape_inference_slice_symbolic_end

opset: domain='' version=13
graph: name='test_cc_shape_inference_slice_symbolic_end'
input: float[a,b,c] X
init: int64[1] starts
init: int64[1] ends
init: int64[1] axes
0: Slice(X, starts, ends, axes) -> sliced
1: Abs(sliced) -> Y
output: float[a,b,c-1] Y

Name	Role	Expected	Computed	Match
X	input	FLOAT[a, b, c]	FLOAT[a, b, c]	yes
sliced	value_info	FLOAT[a, b, c-1]	FLOAT[a, b, c-1]	yes
Y	output	FLOAT[a, b, c-1]	FLOAT[a, b, c-1]	yes

test_cc_shape_inference_tiny_llm

opset: domain='' version=23
graph: name='test_cc_shape_inference_tiny_llm'
input: int64[batch,seq] input_ids
input: int64[batch,total_seq] attention_mask
input: float[batch,4,past_seq,4] past_key
input: float[batch,4,past_seq,4] past_value
init: float[32,16] embed_tokens.weight
init: float[16] input_layernorm.weight
init: float[16,16] q_proj.weight
init: float[16,16] k_proj.weight
init: float[16,16] v_proj.weight
init: float[16,16] o_proj.weight
init: float[16] post_attention_layernorm.weight
init: float[16,32] gate_proj.weight
init: float[16,32] up_proj.weight
init: float[32,16] down_proj.weight
init: float[16] norm.weight
init: float[16,32] lm_head.weight
init: int64[2] mask_axes
init: float[1] mask_one
init: float[1] mask_neg
0: Gather(embed_tokens.weight, input_ids) -> hidden
1: RMSNormalization(hidden, input_layernorm.weight) -> normed1
2: MatMul(normed1, q_proj.weight) -> query
3: MatMul(normed1, k_proj.weight) -> key
4: MatMul(normed1, v_proj.weight) -> value
5: Cast(attention_mask) -> mask_float
6: Unsqueeze(mask_float, mask_axes) -> mask_4d
7: Sub(mask_one, mask_4d) -> mask_inv
8: Mul(mask_inv, mask_neg) -> attn_bias
9: Attention(query, key, value, attn_bias, past_key, past_value) -> attn_out, present_key, present_value
10: MatMul(attn_out, o_proj.weight) -> attn_proj
11: Add(hidden, attn_proj) -> hidden2
12: RMSNormalization(hidden2, post_attention_layernorm.weight) -> normed2
13: MatMul(normed2, gate_proj.weight) -> gate
14: Sigmoid(gate) -> gate_sigmoid
15: Mul(gate, gate_sigmoid) -> gate_silu
16: MatMul(normed2, up_proj.weight) -> up
17: Mul(gate_silu, up) -> mlp_hidden
18: MatMul(mlp_hidden, down_proj.weight) -> mlp_out
19: Add(hidden2, mlp_out) -> hidden3
20: RMSNormalization(hidden3, norm.weight) -> normed_final
21: MatMul(normed_final, lm_head.weight) -> logits
output: float[batch,seq,32] logits
output: float[batch,4,total_seq,4] present_key
output: float[batch,4,total_seq,4] present_value

Name	Role	Expected	Computed	Match
input_ids	input	INT64[batch, seq]	INT64[batch, seq]	yes
attention_mask	input	INT64[batch, total_seq]	INT64[batch, total_seq]	yes
past_key	input	FLOAT[batch, 4, past_seq, 4]	FLOAT[batch, 4, past_seq, 4]	yes
past_value	input	FLOAT[batch, 4, past_seq, 4]	FLOAT[batch, 4, past_seq, 4]	yes
attn_bias	value_info	FLOAT[batch, 1, 1, total_seq]	FLOAT[batch, 1, 1, total_seq]	yes
attn_out	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
attn_proj	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
gate	value_info	FLOAT[batch, seq, 32]	FLOAT[batch, seq, 32]	yes
gate_silu	value_info	FLOAT[batch, seq, 32]	FLOAT[batch, seq, 32]	yes
gate_sigmoid	value_info	FLOAT[batch, seq, 32]	FLOAT[batch, seq, 32]	yes
hidden	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
hidden2	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
hidden3	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
key	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
mask_4d	value_info	FLOAT[batch, 1, 1, total_seq]	FLOAT[batch, 1, 1, total_seq]	yes
mask_float	value_info	FLOAT[batch, total_seq]	FLOAT[batch, total_seq]	yes
mask_inv	value_info	FLOAT[batch, 1, 1, total_seq]	FLOAT[batch, 1, 1, total_seq]	yes
mlp_hidden	value_info	FLOAT[batch, seq, 32]	FLOAT[batch, seq, 32]	yes
mlp_out	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
normed1	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
normed2	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
normed_final	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
query	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
up	value_info	FLOAT[batch, seq, 32]	FLOAT[batch, seq, 32]	yes
value	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
logits	output	FLOAT[batch, seq, 32]	FLOAT[batch, seq, 32]	yes
present_key	output	FLOAT[batch, 4, total_seq, 4]	FLOAT[batch, 4, total_seq, 4]	yes
present_value	output	FLOAT[batch, 4, total_seq, 4]	FLOAT[batch, 4, total_seq, 4]	yes

test_cc_shape_inference_tiny_llm_inlined

opset: domain='' version=23
graph: name='test_cc_shape_inference_tiny_llm_inlined'
input: int64[batch,seq] input_ids
input: int64[batch,total_seq] attention_mask
input: float[batch,4,past_seq,4] past_key
input: float[batch,4,past_seq,4] past_value
init: float[32,16] embed_tokens.weight
init: float[16] input_layernorm.weight
init: float[16,16] q_proj.weight
init: float[16,16] k_proj.weight
init: float[16,16] v_proj.weight
init: float[16,16] o_proj.weight
init: float[16] post_attention_layernorm.weight
init: float[16,32] gate_proj.weight
init: float[16,32] up_proj.weight
init: float[32,16] down_proj.weight
init: float[16] norm.weight
init: float[16,32] lm_head.weight
init: int64[1] rms_axes
init: float[1] rms_eps
init: int64[4] head_shape
init: int64[3] merge_shape
init: float[1] attn_scale
init: int64[2] mask_axes
init: float[1] mask_one
init: float[1] mask_neg
0: Gather(embed_tokens.weight, input_ids) -> hidden
1: Mul(hidden, hidden) -> ln1_sq
2: ReduceMean(ln1_sq, rms_axes) -> ln1_mean
3: Add(ln1_mean, rms_eps) -> ln1_meaneps
4: Sqrt(ln1_meaneps) -> ln1_rms
5: Div(hidden, ln1_rms) -> ln1_norm
6: Mul(ln1_norm, input_layernorm.weight) -> normed1
7: MatMul(normed1, q_proj.weight) -> query
8: MatMul(normed1, k_proj.weight) -> key
9: MatMul(normed1, v_proj.weight) -> value
10: Cast(attention_mask) -> mask_float
11: Unsqueeze(mask_float, mask_axes) -> mask_4d
12: Sub(mask_one, mask_4d) -> mask_inv
13: Mul(mask_inv, mask_neg) -> attn_bias
14: Reshape(query, head_shape) -> query_4d
15: Transpose(query_4d) -> query_heads
16: Reshape(key, head_shape) -> key_4d
17: Transpose(key_4d) -> key_heads
18: Reshape(value, head_shape) -> value_4d
19: Transpose(value_4d) -> value_heads
20: Concat(past_key, key_heads) -> present_key
21: Concat(past_value, value_heads) -> present_value
22: Transpose(present_key) -> key_heads_t
23: MatMul(query_heads, key_heads_t) -> scores
24: Mul(scores, attn_scale) -> scores_scaled
25: Add(scores_scaled, attn_bias) -> scores_biased
26: Softmax(scores_biased) -> attn_weights
27: MatMul(attn_weights, present_value) -> context
28: Transpose(context) -> context_t
29: Reshape(context_t, merge_shape) -> attn_out
30: MatMul(attn_out, o_proj.weight) -> attn_proj
31: Add(hidden, attn_proj) -> hidden2
32: Mul(hidden2, hidden2) -> ln2_sq
33: ReduceMean(ln2_sq, rms_axes) -> ln2_mean
34: Add(ln2_mean, rms_eps) -> ln2_meaneps
35: Sqrt(ln2_meaneps) -> ln2_rms
36: Div(hidden2, ln2_rms) -> ln2_norm
37: Mul(ln2_norm, post_attention_layernorm.weight) -> normed2
38: MatMul(normed2, gate_proj.weight) -> gate
39: Sigmoid(gate) -> gate_sigmoid
40: Mul(gate, gate_sigmoid) -> gate_silu
41: MatMul(normed2, up_proj.weight) -> up
42: Mul(gate_silu, up) -> mlp_hidden
43: MatMul(mlp_hidden, down_proj.weight) -> mlp_out
44: Add(hidden2, mlp_out) -> hidden3
45: Mul(hidden3, hidden3) -> lnf_sq
46: ReduceMean(lnf_sq, rms_axes) -> lnf_mean
47: Add(lnf_mean, rms_eps) -> lnf_meaneps
48: Sqrt(lnf_meaneps) -> lnf_rms
49: Div(hidden3, lnf_rms) -> lnf_norm
50: Mul(lnf_norm, norm.weight) -> normed_final
51: MatMul(normed_final, lm_head.weight) -> logits
output: float[batch,seq,32] logits
output: float[batch,4,total_seq,4] present_key
output: float[batch,4,total_seq,4] present_value

Name	Role	Expected	Computed	Match
input_ids	input	INT64[batch, seq]	INT64[batch, seq]	yes
attention_mask	input	INT64[batch, total_seq]	INT64[batch, total_seq]	yes
past_key	input	FLOAT[batch, 4, past_seq, 4]	FLOAT[batch, 4, past_seq, 4]	yes
past_value	input	FLOAT[batch, 4, past_seq, 4]	FLOAT[batch, 4, past_seq, 4]	yes
attn_bias	value_info	FLOAT[batch, 1, 1, total_seq]	FLOAT[batch, 1, 1, total_seq]	yes
attn_out	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
attn_proj	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
attn_weights	value_info	FLOAT[batch, 4, seq, total_seq]	FLOAT[batch, 4, seq, total_seq]	yes
context	value_info	FLOAT[batch, 4, seq, 4]	FLOAT[batch, 4, seq, 4]	yes
context_t	value_info	FLOAT[batch, seq, 4, 4]	FLOAT[batch, seq, 4, 4]	yes
gate	value_info	FLOAT[batch, seq, 32]	FLOAT[batch, seq, 32]	yes
gate_silu	value_info	FLOAT[batch, seq, 32]	FLOAT[batch, seq, 32]	yes
gate_sigmoid	value_info	FLOAT[batch, seq, 32]	FLOAT[batch, seq, 32]	yes
hidden	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
hidden2	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
hidden3	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
key	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
key_4d	value_info	FLOAT[batch, seq, 4, 4]	FLOAT[batch, seq, 4, 4]	yes
key_heads	value_info	FLOAT[batch, 4, seq, 4]	FLOAT[batch, 4, seq, 4]	yes
key_heads_t	value_info	FLOAT[batch, 4, 4, total_seq]	FLOAT[batch, 4, 4, total_seq]	yes
ln1_mean	value_info	FLOAT[batch, seq, 1]	FLOAT[batch, seq, 1]	yes
ln1_meaneps	value_info	FLOAT[batch, seq, 1]	FLOAT[batch, seq, 1]	yes
ln1_norm	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
ln1_rms	value_info	FLOAT[batch, seq, 1]	FLOAT[batch, seq, 1]	yes
ln1_sq	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
ln2_mean	value_info	FLOAT[batch, seq, 1]	FLOAT[batch, seq, 1]	yes
ln2_meaneps	value_info	FLOAT[batch, seq, 1]	FLOAT[batch, seq, 1]	yes
ln2_norm	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
ln2_rms	value_info	FLOAT[batch, seq, 1]	FLOAT[batch, seq, 1]	yes
ln2_sq	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
lnf_mean	value_info	FLOAT[batch, seq, 1]	FLOAT[batch, seq, 1]	yes
lnf_meaneps	value_info	FLOAT[batch, seq, 1]	FLOAT[batch, seq, 1]	yes
lnf_norm	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
lnf_rms	value_info	FLOAT[batch, seq, 1]	FLOAT[batch, seq, 1]	yes
lnf_sq	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
mask_4d	value_info	FLOAT[batch, 1, 1, total_seq]	FLOAT[batch, 1, 1, total_seq]	yes
mask_float	value_info	FLOAT[batch, total_seq]	FLOAT[batch, total_seq]	yes
mask_inv	value_info	FLOAT[batch, 1, 1, total_seq]	FLOAT[batch, 1, 1, total_seq]	yes
mlp_hidden	value_info	FLOAT[batch, seq, 32]	FLOAT[batch, seq, 32]	yes
mlp_out	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
normed1	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
normed2	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
normed_final	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
query	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
query_4d	value_info	FLOAT[batch, seq, 4, 4]	FLOAT[batch, seq, 4, 4]	yes
query_heads	value_info	FLOAT[batch, 4, seq, 4]	FLOAT[batch, 4, seq, 4]	yes
scores	value_info	FLOAT[batch, 4, seq, total_seq]	FLOAT[batch, 4, seq, total_seq]	yes
scores_biased	value_info	FLOAT[batch, 4, seq, total_seq]	FLOAT[batch, 4, seq, total_seq]	yes
scores_scaled	value_info	FLOAT[batch, 4, seq, total_seq]	FLOAT[batch, 4, seq, total_seq]	yes
up	value_info	FLOAT[batch, seq, 32]	FLOAT[batch, seq, 32]	yes
value	value_info	FLOAT[batch, seq, 16]	FLOAT[batch, seq, 16]	yes
value_4d	value_info	FLOAT[batch, seq, 4, 4]	FLOAT[batch, seq, 4, 4]	yes
value_heads	value_info	FLOAT[batch, 4, seq, 4]	FLOAT[batch, 4, seq, 4]	yes
logits	output	FLOAT[batch, seq, 32]	FLOAT[batch, seq, 32]	yes
present_key	output	FLOAT[batch, 4, total_seq, 4]	FLOAT[batch, 4, total_seq, 4]	yes
present_value	output	FLOAT[batch, 4, total_seq, 4]	FLOAT[batch, 4, total_seq, 4]	yes

test_cc_shape_inference_topk_pairwise_distance

opset: domain='' version=18
graph: name='test_cc_shape_inference_topk_pairwise_distance'
input: float[N,D] X
input: int64[1] K
init: int64[1] axes_row
init: int64[1] axes_col
init: int64[1] reduce_axes
init: int64[1] mean_axes
0: Unsqueeze(X, axes_row) -> x_rows
1: Unsqueeze(X, axes_col) -> x_cols
2: Sub(x_rows, x_cols) -> diff
3: Mul(diff, diff) -> sq
4: ReduceSum(sq, reduce_axes) -> sum_sq
5: Sqrt(sum_sq) -> dist
6: TopK(dist, K) -> topk_values, topk_indices
7: ReduceMean(topk_values, mean_axes) -> Y
output: float[N] Y

Name	Role	Expected	Computed	Match
X	input	FLOAT[N, D]	FLOAT[N, D]	yes
K	input	INT64[1]	INT64[1]	yes
x_rows	value_info	FLOAT[N, 1, D]	FLOAT[N, 1, D]	yes
x_cols	value_info	FLOAT[1, N, D]	FLOAT[1, N, D]	yes
diff	value_info	FLOAT[N, N, D]	FLOAT[N, N, D]	yes
sq	value_info	FLOAT[N, N, D]	FLOAT[N, N, D]	yes
sum_sq	value_info	FLOAT[N, N]	FLOAT[N, N]	yes
dist	value_info	FLOAT[N, N]	FLOAT[N, N]	yes
topk_values	value_info	FLOAT[N, TopK_k]	FLOAT[N, TopK_k]	yes
topk_indices	value_info	INT64[N, TopK_k]	INT64[N, TopK_k]	yes
Y	output	FLOAT[N]	FLOAT[N]	yes

test_cc_shape_inference_two_topk_different_k

opset: domain='' version=18
graph: name='test_cc_shape_inference_two_topk_different_k'
input: float[N,5] X
input: int64[1] K1
input: int64[1] K2
init: int64[1] mean_axes
0: TopK(X, K1) -> values1, indices1
1: TopK(values1, K2) -> values2, indices2
2: ReduceMean(values2, mean_axes) -> Y
output: float[N] Y

Name	Role	Expected	Computed	Match
X	input	FLOAT[N, 5]	FLOAT[N, 5]	yes
K1	input	INT64[1]	INT64[1]	yes
K2	input	INT64[1]	INT64[1]	yes
values1	value_info	FLOAT[N, TopK_k]	FLOAT[N, TopK_k]	yes
indices1	value_info	INT64[N, TopK_k]	INT64[N, TopK_k]	yes
values2	value_info	FLOAT[N, TopK_k_2]	FLOAT[N, TopK_k_2]	yes
indices2	value_info	INT64[N, TopK_k_2]	INT64[N, TopK_k_2]	yes
Y	output	FLOAT[N]	FLOAT[N]	yes

test_cc_shape_inference_two_topk_same_k

opset: domain='' version=18
graph: name='test_cc_shape_inference_two_topk_same_k'
input: float[N,5] X
input: int64[1] K
init: int64[1] mean_axes
0: TopK(X, K) -> values1, indices1
1: TopK(values1, K) -> values2, indices2
2: ReduceMean(values2, mean_axes) -> Y
output: float[N] Y

Name	Role	Expected	Computed	Match
X	input	FLOAT[N, 5]	FLOAT[N, 5]	yes
K	input	INT64[1]	INT64[1]	yes
values1	value_info	FLOAT[N, TopK_k]	FLOAT[N, TopK_k]	yes
indices1	value_info	INT64[N, TopK_k]	INT64[N, TopK_k]	yes
values2	value_info	FLOAT[N, TopK_k]	FLOAT[N, TopK_k]	yes
indices2	value_info	INT64[N, TopK_k]	INT64[N, TopK_k]	yes
Y	output	FLOAT[N]	FLOAT[N]	yes

test_cc_shape_inference_unsqueeze_vas_reshape

opset: domain='' version=18
graph: name='test_cc_shape_inference_unsqueeze_vas_reshape'
input: float[D1,D2] x
input: float[M,D1] y
input: float[K,D2] z
init: int64[] idx
init: int64[1] axes0
0: Shape(y) -> shape_y
1: Shape(z) -> shape_z
2: Gather(shape_y, idx) -> d1
3: Gather(shape_z, idx) -> d2
4: Unsqueeze(d1, axes0) -> u1
5: Unsqueeze(d2, axes0) -> u2
6: Concat(u1, u2) -> new_shape
7: Reshape(x, new_shape) -> reshaped
8: Abs(reshaped) -> out
output: float[D1,D2] out

Name	Role	Expected	Computed	Match
x	input	FLOAT[D1, D2]	FLOAT[D1, D2]	yes
y	input	FLOAT[M, D1]	FLOAT[M, D1]	yes
z	input	FLOAT[K, D2]	FLOAT[K, D2]	yes
shape_y	value_info	INT64[2]	INT64[2]	yes
shape_z	value_info	INT64[2]	INT64[2]	yes
d1	value_info	INT64[]	INT64[]	yes
d2	value_info	INT64[]	INT64[]	yes
u1	value_info	INT64[1]	INT64[1]	yes
u2	value_info	INT64[1]	INT64[1]	yes
new_shape	value_info	INT64[2]	INT64[2]	yes
reshaped	value_info	FLOAT[D1, D2]	FLOAT[D1, D2]	yes
out	output	FLOAT[D1, D2]	FLOAT[D1, D2]	yes

test_cc_shape_inference_value_as_shape

opset: domain='' version=20
graph: name='test_cc_shape_inference_value_as_shape'
input: float[N,1] x
input: float[1,B] y1
input: float[1,B] y2
input: float[1,B] y3
init: int64[1] one
0: Shape(x) -> n
1: Shape(x) -> b
2: Concat(n, b) -> shape
3: Add(shape, one) -> shape1
4: Sub(shape1, one) -> shape2
5: Expand(x, shape2) -> expanded
6: Add(expanded, y1) -> z1
7: Add(expanded, y2) -> z2
8: Add(expanded, y3) -> z3
9: Add(z1, z2) -> z12
10: Add(z12, z3) -> z_pre_abs
11: Abs(z_pre_abs) -> z
output: float[N,B] z

Name	Role	Expected	Computed	Match
x	input	FLOAT[N, 1]	FLOAT[N, 1]	yes
y1	input	FLOAT[1, B]	FLOAT[1, B]	yes
y2	input	FLOAT[1, B]	FLOAT[1, B]	yes
y3	input	FLOAT[1, B]	FLOAT[1, B]	yes
n	value_info	INT64[1]	INT64[1]	yes
b	value_info	INT64[1]	INT64[1]	yes
shape	value_info	INT64[2]	INT64[2]	yes
shape1	value_info	INT64[2]	INT64[2]	yes
shape2	value_info	INT64[2]	INT64[2]	yes
expanded	value_info	FLOAT[N, 1]	FLOAT[N, 1]	yes
z1	value_info	FLOAT[N, B]	FLOAT[N, B]	yes
z2	value_info	FLOAT[N, B]	FLOAT[N, B]	yes
z12	value_info	FLOAT[N, B]	FLOAT[N, B]	yes
z3	value_info	FLOAT[N, B]	FLOAT[N, B]	yes
z_pre_abs	value_info	FLOAT[N, B]	FLOAT[N, B]	yes
z	output	FLOAT[N, B]	FLOAT[N, B]	yes

test_cc_shape_inference_value_as_shape_builder

opset: domain='' version=18
graph: name='test_cc_shape_inference_value_as_shape_builder'
input: float[batch,seq,256] ids_weight
init: int64[2] init328
init: float[256,256] A
init: float[256,256] B
init: float[256,256] C
0: Shape(ids_weight) -> shape
1: Concat(shape, init328) -> new_shape
2: MatMul(ids_weight, A) -> A1
3: MatMul(ids_weight, B) -> B1
4: MatMul(ids_weight, C) -> C1
5: Reshape(A1, new_shape) -> Areshaped
6: Reshape(B1, new_shape) -> Breshaped
7: Reshape(C1, new_shape) -> Creshaped
8: Transpose(Areshaped) -> At
9: Transpose(Breshaped) -> Bt
10: Transpose(Creshaped) -> Ct
output: float[batch,32,seq,8] At
output: float[batch,32,seq,8] Bt
output: float[batch,32,seq,8] Ct

Name	Role	Expected	Computed	Match
ids_weight	input	FLOAT[batch, seq, 256]	FLOAT[batch, seq, 256]	yes
shape	value_info	INT64[2]	INT64[2]	yes
new_shape	value_info	INT64[4]	INT64[4]	yes
A1	value_info	FLOAT[batch, seq, 256]	FLOAT[batch, seq, 256]	yes
B1	value_info	FLOAT[batch, seq, 256]	FLOAT[batch, seq, 256]	yes
C1	value_info	FLOAT[batch, seq, 256]	FLOAT[batch, seq, 256]	yes
Areshaped	value_info	FLOAT[batch, seq, 32, 8]	FLOAT[batch, seq, 32, 8]	yes
Breshaped	value_info	FLOAT[batch, seq, 32, 8]	FLOAT[batch, seq, 32, 8]	yes
Creshaped	value_info	FLOAT[batch, seq, 32, 8]	FLOAT[batch, seq, 32, 8]	yes
At	output	FLOAT[batch, 32, seq, 8]	FLOAT[batch, 32, seq, 8]	yes
Bt	output	FLOAT[batch, 32, seq, 8]	FLOAT[batch, 32, seq, 8]	yes
Ct	output	FLOAT[batch, 32, seq, 8]	FLOAT[batch, 32, seq, 8]	yes

See also

• Using backend tests to evaluate a runtime

• Runtime test coverage (onnxruntime and shape_inference)

• onnx_light.onnx_optim.shape_inference.infer_shapes_model()

• onnx_light.tools.pretty_onnx()