Next Steps

Performance roadmaps for the kernels that onnx-light-cpu plans to optimize next. Completed plans are kept in the 2026 archive.

Started

Plan	Contribution
AVX2 performance	Uses the explicit AVX2 SIMD ceiling to measure and rank the remaining gaps below the completed AVX-512 paths before optimizing matrix, Attention, activation, normalization, unary, and binary workloads. The isolated-runtime diagnostic baseline identifies matrix and Attention priorities and a thread-pool interference bias in the existing runner. The matrix kernel follow-up delivers measured FP16 packing and FP16/FP32 single-row improvements; full ORT parity remains open.
AVX2 activation and normalization gap closure	Extends the #604 AVX2 activation work (AVX2 PR04a): measured Sigmoid, Softmax, BiasGelu, and RMSNormalization against ONNX Runtime and closed the one found gap, a serialized Float16 RMSNormalization reduction; the full onnx-light backend-corpus acceptance run remains.
AVX2 Gemm and MatMul gap closure	Measures the AVX2-ceiling FP32/FP64 Gemm/MatMul corpus from #633 (AVX2 PR02a); dedicated AVX2-only hardware and an onnx-light checkout remain required for a genuine AVX2-vs-AVX2 ONNX Runtime parity gate.
Qwen3 missing kernels	Tracks optimized INT4 projections and integration work after the portable MatMulNBits foundation. Gather and SimplifiedLayerNormalization are complete; Cast, Split, and SkipSimplifiedLayerNormalization have delivered their principal kernels, with runtime metadata and persistent-cache ownership kept in onnx-light.

Planned

Plan	Contribution
Qwen3 inference	Freezes the audited end-to-end model benchmark, then builds the canonical batch-1 INT4 and persistent-decode path on top of the delivered GQA, RMSNormalization, and Sigmoid primitives.
Qwen3 non-MatMulNBits operators	Implements the metadata, input, layout, activation, and normalization slice required by the audited Qwen3 graph independently of packed INT4 projections; Sigmoid and the shared RMS engine are already available.

Discussion

Plan	Contribution
Conv	Defines prepared convolution plans and specialized algorithms that avoid a universal materialized im2col path.
Unary elementwise	Unifies scalar, SIMD, traversal, and scheduling implementations across unary operators.
Processor-aware tuning	Defines calibration and persistence of processor-specific elementwise thresholds.
SVM	Defines prepared SVM classification and regression kernels.
Persistent KV cache	Defines mutable, paged, and optionally quantized cache storage that avoids copying complete past K/V tensors during decode.