Architecture comparison · source-backed checkpoints

See the design choices side by side.

Compare the current Atlas specimens across attention strategy, sparse feed-forward routing and residual topology, then jump directly into the mechanism that explains each difference.

Back to atlas ↖

Read attention, sparse FFN and residual topology together.

InclusionAI

Ling 3.0 tiny

7.9B total · 1.3B active · 24 layers
Attention strategy
Finite-state KDA recurrence is periodically refreshed by latent MLA blocks.
layer mix24 layers
Sparse FFN
1 dense layer · 23 MoE layers · 128 routed experts · top-8 · 1 shared expert
8 active / 128 routed6.3%
Residual topology
Standard single residual stream
residual width1× stream
Architectural character
Finite-state recurrent attention alternating with latent attention.
2 cinematic walkthroughs2 indexed mechanisms

Qwen

Qwen3.6-35B-A3B

35B total · 3.0B active · 40 layers
Attention strategy
Recurrent Gated DeltaNet periodically yields to gated full-attention refresh blocks.
layer mix40 layers
Sparse FFN
40 MoE layers · 256 routed experts · top-8 · 1 sigmoid-gated shared expert
8 active / 256 routed3.1%
Residual topology
Standard single residual stream
residual width1× stream
Architectural character
Recurrent state dominates the stack, with periodic full-attention refresh blocks.
3 cinematic walkthroughs5 indexed mechanisms

Qwen

Qwen3.8-Flash-Next

125B main · 180B stored · 6.0B active · 48 layers
Attention strategy
Recurrent Gated DeltaNet periodically yields to independently indexed sparse QSA.
layer mix48 layers
Sparse FFN
48 MoE layers · 512 routed experts · top-10 · 1 sigmoid-gated shared expert
10 active / 512 routed2.0%
Residual topology
Gated Residual four-branch state
residual width4× stream
Architectural character
A recurrent/sparse-attention rhythm combines per-layer QSA retrieval, ultra-sparse MoE, dynamically gated residual branches and a deterministic PLE side-memory pool.
0 cinematic walkthroughs5 indexed mechanisms

DeepSeek

DeepSeek V4 Flash

284B total · 13B active · 43 layers
Attention strategy
A local exact-attention stem transitions into compressed and hybrid long-context attention.
layer mix43 layers
Sparse FFN
43 MoE layers · 256 routed experts · top-6 · 1 shared expert
6 active / 256 routed2.3%
Residual topology
mHC four-stream residual state
residual width4× stream
Architectural character
Local exact attention plus aggressively compressed long-context retrieval and depth-wise residual routing.
6 cinematic walkthroughs8 indexed mechanisms

Z.ai

GLM-5.2

744B total · 40B active · 78 layers
Attention strategy
MLA-backed sparse attention runs throughout, with IndexShare reusing sparse address selections across declared groups.
layer mix78 layers
Sparse FFN
3 dense layers · 75 MoE layers · 256 routed experts · top-8 · 1 always-on shared expert
8 active / 256 routed3.1%
Residual topology
Standard single residual stream
residual width1× stream
Architectural character
Million-token sparse retrieval couples MLA with a lightweight indexer whose selections are reused across IndexShare groups.
3 cinematic walkthroughs3 indexed mechanisms

Bars encode architecture, not benchmark performance: attention width shows layer counts, expert fill shows routed experts active per token, and residual lines show simultaneous residual streams. Qwen3.8 GR and DeepSeek mHC both use four streams here, but the comparison text preserves their different control mechanisms.