AI Agent是天然的"黑盒"——它做了什么、为什么这么做、为什么出错了，这些问题在生产环境中极难回答。一个完善的日志与可观测性体系，是把黑盒变白盒的关键。本文将系统介绍AI Agent的日志设计与故障排查方法论。
一、Agent可观测性的特殊挑战 1.1 与传统服务日志的区别 传统服务的日志是线性的：请求A → 处理 → 响应A。但Agent的执行是非线性的：
用户输入 → 意图理解 → 规划 → 工具调用1 → 工具调用2 → 反思 → → 修正规划 → 工具调用3 → 总结 → 输出 每一步都可能分叉、回退、重试。传统的"一条请求一条日志"模式无法捕捉这种复杂流程。
1.2 核心观测维度 L1: 基础设施层 — GPU利用率、内存、网络 L2: API服务层 — 请求量、延迟、错误率 L3: Agent逻辑层 — 意图、规划、工具调用、反思 L4: LLM推理层 — prompt内容、生成内容、token消耗 L5: 业务效果层 — 任务完成率、用户满意度 大部分团队只关注L1和L2，但Agent故障的根因往往在L3和L4。
二、结构化日志设计 2.1 Trace-Tree模型 Agent的执行过程天然是树状结构，应当用Trace-Tree而非线性日志来记录：
@dataclass class AgentTrace: trace_id: str # 全局追踪ID session_id: str # 会话ID root_span: AgentSpan # 根span @dataclass class AgentSpan: span_id: str parent_id: str name: str # e.g., "intent_understanding", "tool_call" span_type: str # think / act / observe / reflect input: dict output: dict start_time: float end_time: float status: str # success / error / timeout metadata: dict # 额外信息 children: List[AgentSpan] 2.2 关键Span类型 class SpanTypes: INTENT = "intent" # 意图理解 PLANNING = "planning" # 规划 TOOL_CALL = "tool_call" # 工具调用 LLM_CALL = "llm_call" # LLM推理 REFLECTION = "reflection" # 反思 DELEGATION = "delegation" # 委托子Agent OUTPUT = "output" # 最终输出 2.3 日志记录实现 class AgentLogger: def __init__(self): self.tracer = DistributedTracer() @contextmanager def span(self, name, span_type, parent_id=None): span = AgentSpan( span_id=generate_id(), parent_id=parent_id, name=name, span_type=span_type, start_time=time.time(), input={}, output={}, status="running", metadata={}, children=[] ) try: yield span span.status = "success" except Exception as e: span.status = "error" span.metadata["error"] = str(e) span.metadata["traceback"] = traceback.format_exc() raise finally: span.end_time = time.time() self.tracer.report(span) def log_llm_call(self, span, prompt, response, model, tokens): """记录LLM调用的详细信息""" span.metadata["llm"] = { "model": model, "prompt_tokens": tokens["prompt"], "completion_tokens": tokens["completion"], "prompt_hash": hash(prompt[:100]), # 隐私保护 "response_length": len(response), "latency_ms": span.duration_ms } def log_tool_call(self, span, tool_name, args, result, success): """记录工具调用""" span.metadata["tool"] = { "name": tool_name, "args_hash": hash(str(args)), # 参数指纹 "result_size": len(str(result)), "success": success } 2.4 完整Trace示例 { "trace_id": "trace_abc123", "session_id": "sess_xyz", "duration_ms": 4500, "status": "success", "spans": [ { "name": "intent_understanding", "type": "intent", "duration_ms": 320, "input": {"user_message": "帮我查下最近的报销进度"}, "output": {"intent": "query_reimbursement", "entities": {}}, "children": [ { "name": "llm_call", "type": "llm_call", "duration_ms": 310, "metadata": { "model": "gpt-4-turbo", "prompt_tokens": 850, "completion_tokens": 45 } } ] }, { "name": "planning", "type": "planning", "duration_ms": 280, "output": {"plan": ["call_finance_api", "summarize_result"]} }, { "name": "tool_call:finance_api", "type": "tool_call", "duration_ms": 1200, "metadata": { "tool": "finance_api", "args": {"user_id": "***", "date_range": "30d"}, "success": true } }, { "name": "llm_call:summarize", "type": "llm_call", "duration_ms": 890, "metadata": { "model": "gpt-4-turbo", "prompt_tokens": 1200, "completion_tokens": 180 } } ] } 三、常见故障模式与排查 3.1 意图误判 症状：Agent执行了正确的工具但回答了错误的问题
...
