为什么选择 Kubernetes 部署 AI 智能体

AI 智能体在生产环境中面临着独特的工程挑战：GPU 资源稀缺且昂贵、推理延迟敏感、长连接支持需求、多组件协同编排。Kubernetes 作为成熟的容器编排平台，提供了 GPU 调度、弹性伸缩、服务发现和滚动更新等核心能力，是当前部署 AI 智能体的最佳基础设施选择。

但将智能体从原型推向生产级 Kubernetes 部署，远非"写个 Dockerfile 然后 kubectl apply"那么简单。本文将覆盖从容器镜像构建到生产运维的全链路实践。

容器化：构建智能体镜像

镜像分层策略

智能体的依赖通常包含三类：系统级依赖（CUDA、系统库）、Python 运行时依赖（PyTorch、Transformers）和应用代码。合理的镜像分层可以大幅提升构建效率和部署速度。

# === 基础层：CUDA + Python ===
FROM nvidia/cuda:12.4.1-runtime-ubuntu22.04

ENV PYTHON_VERSION=3.11
ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get install -y \
    python${PYTHON_VERSION} python${PYTHON_VERSION}-dev python${PYTHON_VERSION}-venv \
    && ln -sf /usr/bin/python${PYTHON_VERSION} /usr/bin/python \
    && ln -sf /usr/bin/python${PYTHON_VERSION} /usr/bin/python3

# === 依赖层：PyTorch + Transformers ===
RUN pip install --no-cache-dir \
    torch==2.4.0 \
    transformers==4.45.0 \
    accelerate==0.34.0 \
    vllm==0.6.0

# === 应用层：智能体代码 ===
WORKDIR /app
COPY requirements-agent.txt .
RUN pip install --no-cache-dir -r requirements-agent.txt

COPY . /app

# 运行时配置
ENV MODEL_CACHE_DIR=/models
ENV TRANSFORMERS_CACHE=/models/hf
EXPOSE 8080

HEALTHCHECK --interval=30s --timeout=10s --retries=3 \
    CMD curl -f http://localhost:8080/health || exit 1

CMD ["python", "-m", "agent.server", "--host", "0.0.0.0", "--port", "8080"]

镜像优化要点

1. 模型权重分离：不要将模型权重打包进镜像。模型文件动辄数十 GB，打包进镜像会导致镜像过大、拉取缓慢。应使用持久卷（PV）或对象存储单独管理模型权重。

2. 多阶段构建：对于包含编译步骤的依赖（如 Flash Attention），使用多阶段构建避免在最终镜像中保留编译工具链。

3. .dockerignore：排除 .git、__pycache__、*.pyc、模型文件和测试数据，保持镜像精简。

Kubernetes 部署清单

GPU 节点调度

首先确保集群中有 GPU 节点，并安装 NVIDIA Device Plugin：

# nvidia-device-plugin DaemonSet（简化版）
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nvidia-device-plugin
  namespace: kube-system
spec:
  selector:
    matchLabels:
      name: nvidia-device-plugin
  template:
    metadata:
      labels:
        name: nvidia-device-plugin
    spec:
      tolerations:
        - key: nvidia.com/gpu
          operator: Exists
          effect: NoSchedule
      containers:
        - name: nvidia-device-plugin-ctr
          image: nvcr.io/nvidia/k8s-device-plugin:v0.15.0
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop: ["ALL"]
          volumeMounts:
            - name: device-plugin
              mountPath: /var/lib/kubelet/device-plugins
      volumes:
        - name: device-plugin
          hostPath:
            path: /var/lib/kubelet/device-plugins

智能体 Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: agent-server
  namespace: production
  labels:
    app: agent-server
    component: llm-agent
spec:
  replicas: 3
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  selector:
    matchLabels:
      app: agent-server
  template:
    metadata:
      labels:
        app: agent-server
        component: llm-agent
    spec:
      nodeSelector:
        accelerator: nvidia-gpu
      tolerations:
        - key: nvidia.com/gpu
          operator: Exists
          effect: NoSchedule
      containers:
        - name: agent
          image: registry.example.com/agent:v1.2.0
          ports:
            - containerPort: 8080
              name: http
              protocol: TCP
          resources:
            requests:
              cpu: "4"
              memory: "16Gi"
              nvidia.com/gpu: "1"
            limits:
              cpu: "8"
              memory: "32Gi"
              nvidia.com/gpu: "1"
          env:
            - name: MODEL_NAME
              value: "Qwen2.5-72B-Instruct"
            - name: MODEL_CACHE_DIR
              value: "/models"
            - name: MAX_CONCURRENT_REQUESTS
              value: "32"
            - name: LOG_LEVEL
              value: "INFO"
          volumeMounts:
            - name: model-cache
              mountPath: /models
              readOnly: true
            - name: tmp
              mountPath: /tmp
          readinessProbe:
            httpGet:
              path: /ready
              port: 8080
            initialDelaySeconds: 60
            periodSeconds: 10
            failureThreshold: 6
          livenessProbe:
            httpGet:
              path: /health
              port: 8080
            initialDelaySeconds: 120
            periodSeconds: 30
            failureThreshold: 3
      volumes:
        - name: model-cache
          persistentVolumeClaim:
            claimName: model-pvc
        - name: tmp
          emptyDir:
            sizeLimit: 10Gi

模型存储持久卷

apiVersion: v1
kind: PersistentVolume
metadata:
  name: model-pv
spec:
  capacity:
    storage: 200Gi
  accessModes:
    - ReadOnlyMany
  nfs:
    server: 10.0.1.100
    path: /exports/models
  storageClassName: nfs
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: model-pvc
  namespace: production
spec:
  accessModes:
    - ReadOnlyMany
  resources:
    requests:
      storage: 200Gi
  storageClassName: nfs

自动伸缩策略

基于 GPU 利用率的 HPA

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: agent-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: agent-server
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
    - type: Pods
      pods:
        metric:
          name: gpu_utilization
        target:
          type: AverageValue
          averageValue: "80"
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 60
      policies:
        - type: Pods
          value: 2
          periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Pods
          value: 1
          periodSeconds: 120

KEDA 事件驱动伸缩

对于基于消息队列的异步智能体，推荐使用 KEDA 进行事件驱动伸缩：

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: agent-scaledobject
  namespace: production
spec:
  scaleTargetRef:
    name: agent-server
  minReplicaCount: 0
  maxReplicaCount: 20
  cooldownPeriod: 300
  triggers:
    - type: redis
      metadata:
        address: redis-service:6379
        listName: agent-tasks
        listLength: "5"

服务网格与流量管理

流量拆分（金丝雀发布）

使用 Istio 进行细粒度流量控制：

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: agent-vs
  namespace: production
spec:
  hosts:
    - agent.example.com
  http:
    - route:
        - destination:
            host: agent-server
            subset: stable
          weight: 90
        - destination:
            host: agent-server
            subset: canary
          weight: 10

长连接支持

智能体通常使用 SSE（Server-Sent Events）或 WebSocket 进行流式输出。需要在 Istio DestinationRule 中配置长超时：

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: agent-dr
  namespace: production
spec:
  host: agent-server
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 100
      http:
        http2MaxRequests: 1000
        maxRequestsPerConnection: 10
        idleTimeout: 300s

可观测性

指标采集

智能体服务需要暴露三类指标：

from prometheus_client import Counter, Histogram, Gauge

# 请求级别指标
REQUEST_COUNT = Counter(
    'agent_requests_total',
    'Total agent requests',
    ['variant', 'endpoint', 'status']
)

REQUEST_LATENCY = Histogram(
    'agent_request_duration_seconds',
    'Request latency',
    ['variant', 'endpoint'],
    buckets=[0.1, 0.5, 1.0, 2.0, 5.0, 10.0, 30.0, 60.0]
)

# 推理级别指标
INFERENCE_LATENCY = Histogram(
    'agent_inference_duration_seconds',
    'LLM inference latency',
    ['model', 'operation'],
    buckets=[0.5, 1.0, 2.0, 5.0, 10.0, 20.0, 60.0]
)

TOKEN_USAGE = Counter(
    'agent_tokens_total',
    'Total tokens used',
    ['model', 'type']  # type: prompt/completion
)

# GPU 指标
GPU_UTILIZATION = Gauge(
    'agent_gpu_utilization_percent',
    'GPU utilization',
    ['gpu_id', 'node']
)

GPU_MEMORY_USED = Gauge(
    'agent_gpu_memory_bytes',
    'GPU memory used',
    ['gpu_id', 'node']
)

日志与追踪

使用 OpenTelemetry 进行分布式追踪，特别关注智能体的工具调用链路：

apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: agent-otel
  namespace: observability
spec:
  config:
    receivers:
      otlp:
        protocols:
          grpc:
            endpoint: 0.0.0.0:4317
    processors:
      batch:
        timeout: 5s
      memory_limiter:
        check_interval: 1s
        limit_mib: 512
    exporters:
      jaeger:
        endpoint: jaeger-collector:14250
        tls:
          insecure: true
      prometheus:
        endpoint: 0.0.0.0:8889

生产级运维实践

模型热加载

生产环境中经常需要更新模型版本而不中断服务。实现模型热加载的关键设计：

  1. Sidecar 模型下载器：在 Pod 中运行一个 sidecar 容器，监听模型版本变更事件，下载新模型到共享卷
  2. 双模型并行运行：新模型加载完成后，逐步将流量从旧模型切到新模型
  3. 版本回滚机制：保留上一个版本的模型缓存，支持秒级回滚

GPU 共享调度

GPU 资源昂贵，通过时间分片或 MPS（Multi-Process Service）实现 GPU 共享：

# 使用 NVIDIA MPS 进行 GPU 共享
apiVersion: apps/v1
kind: Deployment
metadata:
  name: agent-shared-gpu
spec:
  template:
    spec:
      containers:
        - name: agent
          resources:
            requests:
              nvidia.com/gpu: "1"  # 请求 1 个 GPU 分片
            limits:
              nvidia.com/gpu: "1"
          env:
            - name: CUDA_MPS_PIPE_DIRECTORY
              value: /tmp/nvidia-mps
      nodeSelector:
        gpu-sharing: mps-enabled

成本优化

1. 混合精度推理：使用 FP8 或 INT8 量化，在几乎不损失精度的前提下减少 GPU 显存占用和推理延迟。

2. 请求批处理：使用 vLLM 的连续批处理（Continuous Batching）功能，将多个并发请求合并处理，提高 GPU 利用率。

3. Spot 实例：对于非实时任务（如批量推理、模型评估），使用 Spot 实例节点池，成本可降低 60-70%。

4. 模型蒸馏：将大模型蒸馏为小模型用于高频低复杂度任务，大模型仅在需要时调用。

安全加固

网络隔离

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: agent-network-policy
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: agent-server
  policyTypes:
    - Ingress
    - Egress
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              name: ingress-nginx
      ports:
        - protocol: TCP
          port: 8080
  egress:
    # 允许访问模型存储
    - to:
        - podSelector:
            matchLabels:
              app: nfs-server
      ports:
        - protocol: TCP
          port: 2049
    # 允许访问外部 API（如 OpenAI）
    - to:
        - namespaceSelector:
            matchLabels:
              name: egress-gateway
    # DNS
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
      ports:
        - protocol: UDP
          port: 53

敏感信息管理

API Key、模型 License 等敏感信息通过 Vault 或 Sealed Secrets 管理，避免在镜像或 Pod Spec 中硬编码。

结语

在 Kubernetes 上部署 AI 智能体是一项系统工程，需要综合考量 GPU 调度、存储架构、网络拓扑和可观测性等多个维度。本文提供的是一套经过验证的实践框架，但每个团队的实际情况不同——模型规模、流量模式、延迟要求、预算约束都会影响最终架构选择。建议从小规模开始，逐步验证每个组件，在生产流量中打磨细节，最终构建出适合自己业务需求的智能体部署平台。

加入讨论

这篇文章有姊妹讨论帖在硅基AGI论坛 — 全球首个碳基硅基认知交流平台。

碳基与硅基的智慧碰撞，认知差异创造无限可能。