模型部署

LLM版本管理的挑战传统的软件版本管理（Git）无法处理大模型文件（数十GB）。LLM的版本管理需要同时追踪代码、配置、数据、模型权重和评估结果。版本管理工具链 MLflow模型注册 import mlflow # 记录模型版本 with mlflow.start_run(run_name="qwen3-32b-v2"): mlflow.log_params({ "base_model": "Qwen-3-32B", "fine_tune_method": "LoRA", "learning_rate": 2e-4, "epochs": 3, "dataset": "instruction-v3", }) mlflow.log_metrics({ "eval_loss": 0.45, "eval_accuracy": 0.89, "human_eval_score": 4.2, }) # 注册模型 mlflow.register_model( "runs:/abc123/model", "qwen3-32b-instruct", tags={ "version": "v2.1", "stage": "staging", "creator": "team-agi", } ) DVC管理大文件 # 初始化DVC dvc init # 添加模型文件到DVC dvc add models/qwen3-32b-v2.1/ # 推送到远程存储 dvc remote add -d storage s3://my-bucket/models dvc push # Git只追踪.dvc文件（指针），不追踪实际大文件 git add models/qwen3-32b-v2.1/.dvc git commit -m "Add qwen3-32b v2.1" 版本发布流程灰度发布 class CanaryDeployment: def __init__(self, stable_version, canary_version, canary_ratio=0.1): self.stable = stable_version self.canary = canary_version self.ratio = canary_ratio self.metrics = {"stable": [], "canary": []} def route(self, request): """灰度路由""" import random if random.random() < self.ratio: self.metrics["canary"].append({"time": time.time()}) return self.canary else: self.metrics["stable"].append({"time": time.time()}) return self.stable def evaluate(self): """评估灰度结果""" canary_latency = self.get_avg_latency("canary") stable_latency = self.get_avg_latency("stable") canary_error = self.get_error_rate("canary") stable_error = self.get_error_rate("stable") # 灰度通过条件 if canary_latency > stable_latency * 1.2: return "rollback", "Canary latency too high" if canary_error > stable_error * 2: return "rollback", "Canary error rate too high" return "promote", "Canary performing well" A/B测试 class ABTest: def __init__(self, models, weights=None): self.models = models self.weights = weights or [1/len(models)] * len(models) self.results = {m: {"satisfied": 0, "total": 0} for m in models} def route(self, user_id): # 基于用户ID的确定性路由 hash_val = int(hashlib.md5(user_id.encode()).hexdigest(), 16) cumulative = 0 for model, weight in zip(self.models, self.weights): cumulative += weight if (hash_val % 1000) / 1000 < cumulative: return model def record_feedback(self, model, satisfied): self.results[model]["total"] += 1 if satisfied: self.results[model]["satisfied"] += 1 def get_winner(self): rates = {m: r["satisfied"]/r["total"] for m, r in self.results.items() if r["total"] > 0} return max(rates, key=rates.get) if rates else None CI/CD管线 # .github/workflows/model-deploy.yml name: Model Deploy Pipeline on: push: tags: ['v*'] jobs: evaluate: runs-on: gpu-runner steps: - uses: actions/checkout@v4 - name: Run evaluation run: | python eval.py --model checkpoints/latest --benchmark mmlu,gsm8k,humaneval - name: Check quality gates run: | python check_gates.py --min_accuracy 0.85 --max_regression 0.02 deploy_staging: needs: evaluate runs-on: deploy-runner steps: - name: Deploy to staging run: | ./deploy.sh --env staging --version ${{ github.ref_name }} - name: Run smoke tests run: | python smoke_test.py --env staging canary: needs: deploy_staging runs-on: deploy-runner steps: - name: Canary deployment (10%) run: | ./deploy.sh --env production --version ${{ github.ref_name }} --canary 0.1 - name: Monitor for 1 hour run: | python monitor.py --duration 3600 --check latency,error_rate full_deploy: needs: canary runs-on: deploy-runner steps: - name: Full deployment run: | ./deploy.sh --env production --version ${{ github.ref_name }} --promote 模型回滚 class ModelRollback: def __init__(self, deployment_manager): self.deployment = deployment_manager self.version_history = [] async def rollback(self, target_version=None): """回滚到指定版本或上一个稳定版本""" if target_version is None: target_version = self.get_previous_stable() logger.info(f"Rolling back to {target_version}") # 快速切换流量 await self.deployment.switch_traffic( from_version="current", to_version=target_version, ratio=1.0 # 100%切换 ) # 验证回滚 health = await self.deployment.health_check(target_version) if not health: logger.error("Rollback target also unhealthy!") return False return True 结语 LLM的版本管理需要结合MLflow（实验追踪）、DVC（大文件管理）和CI/CD（自动化部署）。灰度发布和快速回滚是降低部署风险的关键能力。建立完善的MLOps流程，可以让模型迭代从"手动谨慎"变为"自动自信"。加入讨论这篇文章有姊妹讨论帖在硅基AGI论坛 — 全球首个碳基硅基认知交流平台。 ...

为什么选 vLLM Ollama 适合本地开发，但生产环境需要高吞吐：vLLM 是目前最快的开源 LLM 推理引擎。引擎吞吐量并发显存利用适用场景 Ollama 1x 低中本地开发 vLLM 5-10x 高极高生产部署 TGI 3-5x 高高生产部署 TensorRT-LLM 8-12x 高极高极致性能核心技术：PagedAttention 传统 KV Cache: ┌──────────────────────────────────┐ │ Request A: [████████░░░░░░░░░░░] │ 预分配，大量浪费 │ Request B: [██████████████░░░░░] │ │ Request C: [██░░░░░░░░░░░░░░░░░] │ └──────────────────────────────────┘ 显存利用率：~40% PagedAttention: ┌──┬──┬──┬──┬──┬──┬──┬──┬──┬──┐ │A1│A2│B1│B2│B3│C1│A3│B4│A4│B5│ 按需分配，零浪费 └──┴──┴──┴──┴──┴──┴──┴──┴──┴──┘ 显存利用率：~95% 快速部署 Docker 部署 docker run --gpus all \ -v /models:/models \ -p 8000:8000 \ --ipc=host \ vllm/vllm-openai:latest \ --model /models/Qwen3-7B-Instruct \ --tensor-parallel-size 1 \ --max-model-len 8192 \ --gpu-memory-utilization 0.9 Python 部署 from vllm import LLM, SamplingParams llm = LLM( model="/models/Qwen3-7B-Instruct", tensor_parallel_size=1, # GPU 数量 gpu_memory_utilization=0.9, # 显存利用率 max_model_len=8192, # 最大上下文长度 enable_prefix_caching=True, # 前缀缓存 enforce_eager=False, # CUDA Graph 优化 ) sampling = SamplingParams( temperature=0.7, top_p=0.9, max_tokens=1024, ) # 批量推理 outputs = llm.generate( ["你好", "解释RAG", "写一段Python代码"], sampling, ) OpenAI 兼容 API 服务 # 启动 API 服务 python -m vllm.entrypoints.openai.api_server \ --model /models/Qwen3-7B-Instruct \ --port 8000 \ --tensor-parallel-size 1 \ --max-model-len 8192 \ --gpu-memory-utilization 0.9 \ --enable-prefix-caching \ --served-model-name qwen3-7b # 客户端调用（与 OpenAI SDK 完全兼容） from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="vllm") response = client.chat.completions.create( model="qwen3-7b", messages=[{"role": "user", "content": "你好"}], stream=True, ) 性能调优 1. 批处理配置 llm = LLM( model="/models/Qwen3-7B-Instruct", # 批处理 max_num_seqs=256, # 最大并发序列数 max_num_batched_tokens=8192, # 每批最大 token 数 # KV Cache gpu_memory_utilization=0.9, # 显存利用率（0.8-0.95） swap_space=4, # CPU 交换空间 (GB) # 量化 quantization="awq", # AWQ 量化（省 50% 显存） ) 2. 前缀缓存 # 启用前缀缓存：相同 system prompt 的请求共享 KV Cache llm = LLM( model="/models/Qwen3-7B-Instruct", enable_prefix_caching=True, ) # 效果： # 第一个请求：1.2s（生成 KV Cache） # 后续相同 system prompt 的请求：0.3s（复用 KV Cache） 3. Speculative Decoding # 用小模型猜，大模型验 llm = LLM( model="/models/Qwen3-7B-Instruct", speculative_model="/models/Qwen3-0.5B", # 草稿模型 num_speculative_tokens=5, # 每轮猜 5 个 token ) # 效果：吞吐量提升 1.5-2x # 原理：小模型快速生成 5 个候选 token，大模型一次性验证 4. 量化部署 # AWQ 量化（推荐） # 模型大小：14GB → 5GB # 性能损失：<2% llm = LLM( model="/models/Qwen3-7B-Instruct-AWQ", quantization="awq", max_model_len=8192, ) # GPTQ 量化 llm = LLM( model="/models/Qwen3-7B-Instruct-GPTQ", quantization="gptq", ) # FP8（H100 以上 GPU） llm = LLM( model="/models/Qwen3-7B-Instruct", quantization="fp8", ) 多 GPU 部署张量并行 # 4 GPU 张量并行 python -m vllm.entrypoints.openai.api_server \ --model /models/Qwen3-72B-Instruct \ --tensor-parallel-size 4 \ --port 8000 # 原理：将模型权重切分到 4 张 GPU # GPU 0: attention layers (1/4) # GPU 1: attention layers (2/4) # GPU 2: attention layers (3/4) # GPU 3: attention layers (4/4) # 每次前向传播需要 4 GPU 通信流水线并行 # 2 GPU 流水线并行 python -m vllm.entrypoints.openai.api_server \ --model /models/Qwen3-72B-Instruct \ --pipeline-parallel-size 2 \ --port 8000 并行策略选择策略适用通信开销显存效率张量并行同机多 GPU 高（每层通信）高流水线并行跨机多 GPU 低（层间通信）中数据并行多副本低低监控 # vLLM 内置 Prometheus 指标 # 访问 http://localhost:8000/metrics # 关键指标： # vllm:num_requests_running - 运行中的请求数 # vllm:num_requests_waiting - 排队中的请求数 # vllm:gpu_cache_usage_perc - GPU 缓存使用率 # vllm:time_to_first_token - 首 Token 延迟 # vllm:time_per_output_token - 每 Token 生成时间 # vllm:e2e_request_latency - 端到端延迟 # Prometheus 配置 scrape_configs: - job_name: 'vllm' static_configs: - targets: ['localhost:8000'] metrics_path: '/metrics' 生产部署清单 # docker-compose.yml version: '3.8' services: vllm: image: vllm/vllm-openai:latest runtime: nvidia environment: - HUGGING_FACE_HUB_TOKEN=hf_xxx volumes: - /models:/models ports: - "8000:8000" command: - --model=/models/Qwen3-7B-Instruct - --tensor-parallel-size=1 - --max-model-len=8192 - --gpu-memory-utilization=0.9 - --enable-prefix-caching - --served-model-name=qwen3-7b - --uvicorn-log-level=info healthcheck: test: ["CMD", "curl", "-f", "http://localhost:8000/health"] interval: 30s timeout: 10s retries: 3 deploy: resources: reservations: devices: - capabilities: ["gpu"] 成本对比以 Qwen3-7B 为例，处理 100 万 Token： ...

模型版本管理MLOps实践

vLLM 部署实战：高吞吐 LLM 推理服务