ByQwen3.8-Flash-Next Launches: A Preview of Qwen4 Architecture Focused on Training and Inference Efficiency
Alibaba's Qwen team released **Qwen3.8-Flash-Next** on August 26, 2026, as an open-weight architectural preview of the upcoming Qwen4 series. The model uses a multimodal MoE architecture with **125B main parameters**, an additional **51B N-gram embedding table**, and only **6B active parameters** per inference step. It natively supports a **262,144-token context**, extensible to 1M tokens via YaRN
Alibaba's Qwen team officially released Qwen3.8-Flash-Next on August 26, 2026. The model serves as an early architectural preview of the Qwen4 series, making key design innovations available to the open-source community ahead of the full Qwen4 release .
Qwen3.8-Flash-Next uses a Mixture-of-Experts (MoE) architecture with 125 billion main parameters, supplemented by an additional 51 billion N-gram embedding parameters, while activating only 6 billion parameters per inference step . It natively supports a context window of up to 262,144 tokens, which can be extended to 1 million tokens via YaRN .
Architecture: Four Areas of Systematic Upgrade
The model introduces several architectural innovations that will underpin the full Qwen4 series :
- Attention: The model employs a hybrid GDN (Gated DeltaNet) + QSA (Qwen Sparse Attention) architecture. GDN compresses historical information efficiently, while QSA uses a lightweight indexer to select important context at the micro-block level. At 1M-token context length, QSA achieves up to 7.6x prefill and 4.9x decode attention-kernel speedups .
- Residual: A Gated Residual (GR) mechanism expands the residual stream into 4 parallel branches, with dynamic gates controlling information flow. Residual states support FP8 storage to reduce memory bandwidth overhead .
- Embedding: A 51B-parameter N-gram embedding table adds model capacity through local-context lookups. These parameters can be offloaded to host memory and asynchronously prefetched, avoiding permanent GPU memory occupation .
- Optimization: The model uses the Muon Optimizer with strategies optimized for orthogonalization precision, parameter division, and fused matrix splitting, with scaling laws refitted for the new architecture .
Coding and Agentic Performance
Published benchmark results—all self-reported by the Qwen team—show strong performance across software engineering and agentic tasks :
| Benchmark | Qwen3.8-Flash-Next | Qwen3.8-27B | Qwen3.7-Plus | DeepSeek-V4-Flash-0731 | Claude Opus 4.6 Max |
|---|---|---|---|---|---|
| DeepSWE 1.1 | 58.7 | 42.2 | 16.5 | 54.4 | -- |
| SWE-bench Pro | 62.5 | 61.7 | 55.8 | 56.0 | 53.4 |
| LiveCodeBench v6 | 91.9 | 90.3 | 89.6 | 90.6 | 88.8 |
| GPQA Diamond | 91.7 | 89.2 | 90.3 | 90.8 | 91.3 |
| CoWorkBench | 73.9 | 70.7 | 65.1 | 45.1 | 68.2 |
Source: Qwen team self-reported benchmarks, as published on August 26, 2026
The model achieved a 58.7% score on DeepSWE 1.1 and 62.5% on SWE-bench Pro, outperforming DeepSeek-V4-Flash-0731 by 4.3 and 6.5 percentage points respectively . On LiveCodeBench v6, it scored 91.9%, slightly ahead of competing models .
These figures are self-reported and should be interpreted as early performance signals rather than independently verified benchmarks. Benchmark configurations, reasoning settings, and evaluation harnesses can vary across models.
Multimodal Capabilities
As a multimodal model, Qwen3.8-Flash-Next accepts text, image, and video inputs while generating text outputs . Vision-related benchmarks show competitive results:
- AndroidWorld: 84.5%
- MathVision (with CI): 95.7%
- RealWorldQA: 88.5%
The model's vision capabilities extend to agentic tasks such as screenshot understanding, mobile and computer operation, and multimodal tool calling .
Efficiency and Cost
Qwen3.8-Flash-Next was designed with efficiency as a primary focus. The Qwen team states that training costs are approximately one-ninth those of Qwen3.7-Plus, with inference costs similarly reduced .
The production version, Qwen3.8-Flash, will be offered through Qwen Cloud with API pricing of:
- Input: 1 RMB per million tokens (~$0.16)
- Output: 3 RMB per million tokens (~$0.47)
This pricing is approximately 33% lower than DeepSeek-V4-Flash's off-peak rates .
Open Weights and Ecosystem Support
The model weights have been made publicly available on both Hugging Face and ModelScope under the Qwen Community 1.0 license . A FP8 quantized version is also available to reduce memory requirements .
Supported inference frameworks include vLLM (version 0.28.0+) and SGLang . For deployment, vLLM recommends:
- GB300: TP2 minimum; TP4 recommended for full-tray configuration
- 8x H200: TEP8 with Triton MoE backend
The N-gram embedding table can be offloaded to host memory, requiring approximately 51GB of host RAM plus runtime headroom .
Where Qwen3.8-Flash-Next Fits
Qwen3.8-Flash-Next is positioned as a technical preview of Qwen4 architecture rather than a production-ready chat model. Its primary value lies in allowing the open-source community early access to architectural innovations ahead of the full Qwen4 release .
For developers, the model is particularly relevant for testing coding agents, software engineering workflows, and long-context processing, where it demonstrates competitive performance with only 6B active parameters .
Overall, Qwen3.8-Flash-Next represents a significant step in efficiency-oriented model design, combining sparse activation, novel attention mechanisms, and N-gram memory to reduce costs while maintaining strong coding and agentic capabilities. As independent evaluations accumulate, its position relative to other frontier models will become clearer. For now, its most notable characteristics are its strong coding performance, multimodal input support, 1M-token context, open weights, and low inference cost.