Next Steps

Date:

2026-08

Recommended implementation order

Runtime execution work should proceed in dependency order:

1. implement Profile PR01 from ParallelFor profiling and hardware counters, starting with portable bounded events and a disabled path with no instrumentation work;

2. add process CPU time, inspection, hardware counters, and calibration diagnostics only after that event contract is stable;

3. use that same executor when the fast-loading sequence reaches Prepared and asynchronous execution, rather than creating another scheduler pool.

Model-format work such as custom types, quantization, compiled tensors, and model resolution may proceed independently until it reaches prepared execution. Within the runtime track, the order above is mandatory: profiling or tuning a hidden global pool would produce profiles that a session cannot reproduce.

Large-model startup follows the four-plan sequence documented in Fast-loading implementation sequence.

All Next Steps

Use the search field to filter by status or text, and select a column heading to sort the table.

Status	Next step	Planned work	Why
Started	Fast-loading implementation sequence	Orchestrate the four-step large-model startup roadmap.	Define one dependency order for fast loading.
Started	ParallelFor profiling and hardware counters	Add bounded, opt-in ParallelFor diagnostics and hardware counters.	Explain CPU under-utilization before tuning prepared execution.
Discussed	Structured views over byte buffers	Define structured byte-buffer types for custom formats.	TypeProto.Opaque does not describe serialized layouts.
Discussed	Proto schema inheritance	Add schema inheritance while retaining flat wire encoding.	Reuse common fields without duplicating them across proto families.
Discussed	Quantization	Describe quantized data families and their proto mappings.	Represent quantization consistently as structured custom types.
Discussed	Quantized values in GraphBuilder	Preserve quantized initializers in graph storage.	Avoid implicit dequantization or rewriting by GraphBuilder.
Discussed	Compact GraphBuilder authoring and runtime walkthroughs	Add compact graph authoring and non-gallery runtime walkthroughs.	Make models easier to build, inspect, optimize, and execute.
Discussed	Mutable execution cache	Support in-place KV-cache updates with controlled aliasing.	Avoid duplicating large caches on every update.
Discussed	CompiledTensorProto	Persist packed tensor representations as caches.	Avoid repeating expensive prepacking when a model is reloaded.
Discussed	Model resolution before weight loading	Resolve the final graph and required payloads before I/O.	Load weights only after transformations and liveness analysis.
Discussed	Splitting onnx-light into composable wheels	Split public features into composable Python wheels.	Let users install only the components they need.
Completed	Building onnx_proto, the protobuf-free ONNX schema	Build the protobuf-free ONNX message layer.	Provide the project’s independent base schema layer.
Completed	Porting the ONNX C++ library on top of onnx_proto	Port the ONNX C++ library to onnx_proto.	Run the upstream library without libprotobuf.
Completed	Operator kernels and the C++ backend tests	Provide native kernels and backend tests in C++.	Validate the runtime natively without depending on Python.
Completed	Symbolic gradients for ONNX graphs	Generate backward-pass graphs symbolically.	Support training with a native graph-based gradient pass.
Completed	Integrating onnx-light into onnxruntime (PR #29723)	Route onnxruntime protobuf usage through onnx-light.	Provide a build-time alternative to protobuf in onnxruntime.
Completed	Reducing the lib_onnx_proto binary size	Reduce the lib_onnx_proto shared-library footprint.	Avoid shipping unused wrapper overhead.
Completed	Processor-aware kernel tuning	Make kernel thresholds processor-specific and persistent.	Adapt thresholds to hardware instead of fixed constants.
Completed	Buffer-reuse arenas	Reuse execution and I/O buffers safely.	Reduce allocations without breaking NumPy ownership.
Completed	Pattern-based optimization in GraphBuilder	Rewrite local graph patterns to cheaper equivalents.	Add the native optimization engine missing from GraphBuilder.
Completed	Session execution policies and shared CPU pools	Manage CPU policies and shared executor pools.	Give sessions deterministic, shareable execution resources.