https://guijiagi.com/tags/%E5%88%86%E5%B8%83%E5%BC%8F%E8%AE%AD%E7%BB%83/Recent content in 分布式训练 on 硅基 AGI · 智能体学习与测评Hugozh-cn本站内容采用 CC BY-NC-SA 4.0 国际许可协议授权Mon, 13 Jul 2026 09:00:00 +0800https://guijiagi.com/posts/article-97/Mon, 13 Jul 2026 09:00:00 +0800https://guijiagi.com/posts/article-97/系统解析大模型分布式训练的并行策略，涵盖数据并行、张量并行、流水线并行及混合策略https://guijiagi.com/posts/sequence-parallelism-explained/Thu, 02 Jul 2026 11:07:00 +0800https://guijiagi.com/posts/sequence-parallelism-explained/解析序列并行的核心原理、DeepSpeed Ulysses与Megatron SP的区别与联系https://guijiagi.com/posts/pipeline-parallelism-2026/Thu, 02 Jul 2026 11:06:00 +0800https://guijiagi.com/posts/pipeline-parallelism-2026/从GPipe的朴素流水线到PipeDream的1F1B调度，梳理流水线并行的演进与2026年方案https://guijiagi.com/posts/tensor-parallelism-deep/Thu, 02 Jul 2026 11:05:00 +0800https://guijiagi.com/posts/tensor-parallelism-deep/从原理到实现，深入解析Megatron-LM张量并行方案的设计与工程细节https://guijiagi.com/posts/ring-attention-distributed-context/Thu, 02 Jul 2026 10:57:00 +0800https://guijiagi.com/posts/ring-attention-distributed-context/解析Ring Attention如何通过环形通信实现超长序列的分布式注意力计算https://guijiagi.com/posts/federated-learning-2026/Thu, 02 Jul 2026 10:27:00 +0800https://guijiagi.com/posts/federated-learning-2026/系统梳理2026年联邦学习技术的最新进展、挑战与产业应用实践https://guijiagi.com/posts/ring-attention-million-token-context/Sun, 28 Jun 2026 11:09:00 +0800https://guijiagi.com/posts/ring-attention-million-token-context/深入解析Ring Attention的分布式注意力计算原理、Flash Attention集成与超长上下文实现