February 25, 2025
DeepSeek has released DeepEP, the second open-source project in its “Open Source Week” initiative. This new library provides an efficient expert-parallel communication system designed specifically for Mixture-of-Experts (MoE) model training and inference.
DeepEP: Optimizing MoE Communication
DeepEP addresses a critical challenge in large-scale AI development by optimizing the communication patterns needed for MoE architectures. These models, which selectively activate only relevant “expert” neural networks for specific inputs, require specialized communication protocols to operate efficiently across multiple GPUs and nodes.
According to DeepSeek’s announcement, DeepEP provides:
- Efficient and optimized all-to-all communication between model components
- Support for both intranode (via NVLink) and internode communication (via RDMA)
- High-throughput kernels optimized for training and inference prefilling
- Low-latency kernels specifically designed for inference decoding
- Native FP8 precision support
- Flexible GPU resource control for communication-computation overlapping
DeepEP’s GitHub repository indicates the library can approach the theoretical bandwidth limits of modern GPU interconnects, showing 153-158 GB/s throughput on NVLink (approaching the ~160 GB/s maximum) and 39-47 GB/s on RDMA networks (approaching the ~50 GB/s maximum).
Part of a Larger Open-Source Initiative
This release follows DeepSeek’s February 24th launch of FlashMLA, the first project in its “Open Source Week” campaign. On February 21st, DeepSeek announced the formation of a dedicated AGI (Artificial General Intelligence) exploration team and committed to sharing their research progress through five open-source code repositories.
DeepEP aligns with the group-limited gating algorithm described in DeepSeek’s recent V3 paper, providing kernel optimizations for asymmetric-domain bandwidth forwarding. The library appears particularly focused on large-scale deployment scenarios, offering performance tuning for various network configurations.
Technical Details
The library provides two main sets of kernels:
- Normal kernels: Optimized for high throughput in both training and inference prefilling, supporting NVLink for intranode communication and RDMA for internode communication.
- Low-latency kernels: Designed for inference decoding with minimal delays, using pure RDMA to optimize latency.
DeepEP also introduces a hook-based communication-computation overlapping method that DeepSeek claims doesn’t occupy any SM (Streaming Multiprocessor) resources, potentially allowing better utilization of GPU compute capacity.
Availability
DeepEP is available immediately on GitHub under an MIT license. The library requires Hopper GPUs, Python 3.8+, CUDA 12.3+, PyTorch 2.1+, and a modified version of NVSHMEM that DeepSeek has also made available.
As the second of five planned releases during DeepSeek’s Open Source Week, the AI community will be watching closely to see what other tools and libraries will follow in the coming days.