Qwen · Qwen3.6

Qwen3.6-35B-A3B

A 35B-total / 3B-active multimodal MoE checkpoint with a 27-block vision encoder feeding a 2,048-dimensional language stream and a 40-layer language tower that repeats three Gated DeltaNet blocks followed by one gated grouped-query attention block.

Total35BActive3.0BLayers40Context256K

Checkpoint architecture · source-backed layout

Model anatomyToken path · block microscope · cost surface

Architecture signature30 Gated DeltaNet · 10 Gated Attention

Gated DeltaNet30 · 75%Gated Attention10 · 25%

Layer blueprintAttention · FFN · residual, aligned by transformer block40 layers

Gated DeltaNet30 Gated Attention10 Learned MoE40Residual

Layer mix: 30× Gated DeltaNet · 10× Gated Attention
FFN schedule: 40 learned-routing MoE
Experts / token: 8 / 256 routed + 1 shared
Vision tower: 27 blocks · 1152d → 2048d
Residual topology: single residual stream
Active parameter share: 8.6% of checkpoint

Compare attention Follow attention path Multimodal boundary Vision tower

Each blueprint column is one transformer block. Hover or keyboard-focus a linked cell to read attention, feed-forward and residual structure as one aligned layer; linked cells open the corresponding walkthrough or inspector. DSA cells additionally preserve whether the layer runs a full indexer or reuses a shared IndexShare selection. QSA cells preserve their per-layer sparse retrieval identity, while PLE is marked on the decoder layer receiving N-gram features. MoE cells preserve any layer-level routing transition encoded by the checkpoint manifest. Structural encoding only — it does not imply benchmark quality, throughput or FLOPs.