← Back to 3D modelQwen · Qwen3.6
Model anatomy
Qwen3.6-35B-A3B
A 35B-total / 3B-active multimodal MoE checkpoint with a 27-block vision encoder feeding a 2,048-dimensional language stream and a 40-layer language tower that repeats three Gated DeltaNet blocks followed by one gated grouped-query attention block.
Token path
From token id to next-token logits
Follow the structural path first, then inspect one decoder block at full resolution.
Embedding matrix size is reported as shape × elements only. This view does not assume whether the output head is tied to the input embedding unless the manifest explicitly records it.
Decoder microscope
Layer 1
Select any layer to see the checkpoint-backed attention, feed-forward and residual anatomy.
AttentionGated DeltaNet
- Key heads
- 16
- Value heads
- 32
- Key / value dim
- 128 / 128
- Short conv
- kernel 4
Recurrent state: token history is compressed into a fixed-size state update rather than a full KV history for this layer.
Feed-forwardMixture of Experts
- Routed experts
- 256
- Active routed
- 8 / token
- Shared experts
- 1
- Expert width
- 512
- Routing
- learned
linear router logits → softmax → top-8 → selected-weight renormalization
Residual topologyStandard stream
- Topology
- standard
The residual path carries the block output forward; this panel reports only topology represented by the manifest.
Parameter & memory surface
What is stored, and what participates per token?
Source-backed parameter pools and active-path quantities; stored inventory is not assumed to be read in full for every token.
The full checkpoint parameter set exists in storage / device memory planning; this does not imply every stored weight is read for one token.
Parameters reported as active per token by the model manifest.
Recurrent state: token history is compressed into a fixed-size state update rather than a full KV history for this layer.
Active routed + shared experts for the selected layer.
Evidence for layer 1
Trace every structural claim back to source
2 sources currently support this layer view.
config · rev 7da1103Qwen3.6-35B-A3B released config.json
Concrete text and vision dimensions, including 27 vision blocks, 1,152d vision width, 4,304d vision MLP, 16 heads, patch/merge sizes, 2,048d vision output and the absence of deepstack visual indexes.
implementation · rev 95940bfTransformers Qwen3.5/Qwen3.6 MoE implementationReleased-family implementation for the inherited Qwen3.5 vision tower, decoder branching, Gated DeltaNet recurrence, gated attention, softmax top-k routed experts and sigmoid-gated shared-expert path.