Differences with Mobius (onnxruntime/mobius)

Mobius is a newer model builder for onnxruntime-genai. Like mbext, it constructs ONNX graphs declaratively and applies Hugging Face or custom weights, rather than tracing an existing PyTorch graph. Both projects target the same runtime (onnxruntime-genai) and overlap on the same kinds of models (LLMs, MoE, multimodal, …).

How Mobius is organised

Mobius builds the ONNX graph with the onnxscript.nn API (onnxscript), composing onnxscript.nn.Module objects the way one would compose torch.nn.Module objects. Its pipeline is:

HuggingFace Hub
      │
      ▼
ArchitectureConfig  ◄── from_transformers() / from_diffusers()
      │
      ▼
Model Module        ◄── reusable Components (Attention, MLP, RMSNorm, RoPE, MoELayer, …)
      │
      ▼
Task                ◄── CausalLMTask, VisionLanguageTask, VAETask, DenoisingTask, …
      │
      ▼
ONNX Model          ◄── preprocess_weights() + apply_weights()

The codebase is split into four layers:

• Components — onnxscript.nn.Module building blocks (Attention, MLP, DecoderLayer, RoPE, VisionEncoder, MoELayer, …).

• Models — full architectures composed from those components.

• Tasks — declare the ONNX graph’s I/O contract (inputs, outputs, KV cache), e.g. CausalLMTask, VisionLanguageTask, VAETask.

• Registry — maps a Hugging Face model_type string to a model class.

It also layers on execution-provider-aware optimization (CUDA / DirectML / WebGPU graphs with the right fused kernels), an opt-in static KV cache, and coverage that reaches well beyond text generation: MoE, multimodal, encoder-only, encoder-decoder, speech, audio, vision and diffusion pipelines (hundreds of Transformers model types plus Diffusers components).

How mbext is organised

mbext is a fork of the original onnxruntime_genai.models.builder and keeps its command line and conventions (see Differences with the onnxruntime-genai model builder). A single entry point, modelbuilder.builder.create_model(), dispatches on config.architectures[0] to a Model subclass in modelbuilder/builders/. Each subclass overrides only the few methods that differ for its family (make_layer, make_attention, make_moe, …) on top of a large shared base class that emits the graph imperatively (see Design). Its scope is deliberately narrower — decoder LLMs and MoE for onnxruntime-genai — and its defining features are:

• Fast, offline discrepancy tests. Every architecture ships a tiny, random-weight test that converts a one-layer model and compares ONNX vs. PyTorch logits within a precision-aware tolerance, so CI stays short and deterministic and new architectures get immediate feedback.

• A --private escape hatch to convert proprietary architectures that live in external files, without modifying the package (see Custom model in a separate file (--private)).

Side by side

	mbext	Mobius
Lineage	fork of onnxruntime-genai builder	separate, newer codebase
Graph construction	imperative shared base class	onnxscript.nn.Module composition
Extension model	subclass a Model, override a few hooks	compose Components → Model → Task, register
Scope	decoder LLMs and MoE | LLM, MoE, multimodal, vision, audio, encoder-decoder, diffusion
Testing	tiny random-weight, fully offline	unit + integration (downloads models)
Private / external models	built-in --private option	via registry / custom components

Which one to choose to support a new model?

Both are reasonable; the answer depends on the model and the goal.

• Choose mbext when the new model is a decoder-only LLM or MoE close to a family that already exists here. You subclass the nearest builder, override the one or two methods that differ, add a dispatch branch, and drop in a tiny random-weight test — the offline suite then verifies the conversion in seconds, with no downloads. It is also the pragmatic choice for proprietary models thanks to --private, and for teams already standardised on the onnxruntime-genai builder command line.

• Choose Mobius when the model needs a modality or pipeline that mbext does not target — vision, audio, encoder-decoder or diffusion — or when you want execution-provider-tuned graphs and a static cache out of the box. Its Component/Model/Task/Registry layering, and its large library of reusable components, make novel modalities cheaper to assemble than extending mbext’s base class would be.

In short: for a new LLM/MoE variant where fast, offline verification matters most, prefer mbext; for a genuinely new modality or when you need EP-specific graph optimization, prefer Mobius.

Note

Mobius is evolving quickly. For its current feature set and supported architectures, refer to the Mobius repository directly.