[{"data":1,"prerenderedAt":691},["ShallowReactive",2],{"article:rag-03-rerank-citation-eval":3},{"meta":4,"raw":18,"references":19,"prev":42,"next":53,"body":62,"toc":682},{"slug":5,"title":6,"date":7,"updated":8,"tags":9,"summary":14,"draft":15,"lang":16,"minutes":17},"rag-03-rerank-citation-eval","RAG 全链路（三）：重排、引用溯源与效果评估","2026-02-17","2026-10-11",[10,11,12,13],"RAG","Rerank","评测","可信","按 token 预算选择重排后的证据，区分引用存在、引用支撑与事实正确；分层评估检索、生成和业务结果，并处理资料冲突、缺证据与超时。",false,"zh-CN",5,"[混合检索](\u002Farticles\u002Frag-02-vector-hybrid-search)给出了候选，但候选相关不代表足以回答问题。这一篇把三件事接起来：**重排决定优先读哪些材料，引用让结论可追溯，评测检验整条链路是否有效。**\n\n## 重排是第二次、更细的相关性判断\n\nBi-encoder 分别编码查询与文档，文档向量可离线计算，适合大范围检索。Cross-encoder 将查询与候选一起处理，让它们直接交互，逐对评估相关性，适合较少候选的重排。它通常比简单向量相似度昂贵，收益和延迟依赖模型与文本长度。\n\n一个**实验起点**可以是两路召回各 50 条、去重后重排，再按 token 预算选择最终上下文。候选数、输出条数都要调，不能把 `150 → 20` 视为通用标准。\n\n![重排漏斗与验证：候选召回、相关性重排、预算选择、引用和最终答案评测](\u002Fdiagrams\u002Frag-rerank.svg)\n\n*图 1 · 数量是待调参数，最终供给还要考虑去重、证据覆盖与 token 预算。*\n\n## 多给几段并不总是更好\n\n候选过少可能漏掉多跳证据，供给过多则会增加延迟、引入冲突和分散注意力。比较不同 top-k 时，固定模型与资料版本，记录最终正确率、证据覆盖、输入 token 和 p95。\n\n选择上下文时还应：合并同父文档相邻段落、限制重复、保留互补证据、优先有效版本。若预算不够，把检索到的三份重复说明改成一份，往往比简单砍掉最后一条更合理。\n\n若 reranker 超时，可使用已验证的粗排结果降级；若证据不足，选择追问或说明资料不足。不能为达到固定 top-k 把低相关内容强行凑满。\n\n## 引用需要四项验证\n\n“有一个角标”不代表正确。服务端至少检查：\n\n1. 引用 ID 确实在本次授权检索结果中。\n2. 引用能定位到当时的文档版本与原文片段。\n3. 原文支持对应结论，不能只与主题相近。\n4. 关键结论覆盖充分，不能引用第一句后让后文自由发挥。\n\n可以让模型输出以下业务结构；这是中立示例，并非特定厂商 API 请求：\n\n```json\n{\n  \"claims\": [\n    { \"text\": \"该版本的连接超时配置为 5 秒。\", \"evidence_ids\": [\"runbook-42:rev-3:p-2\"] }\n  ],\n  \"missing_evidence\": [],\n  \"status\": \"answered\"\n}\n```\n\n模型不得生成任意 URL；服务端用已知 ID 解析链接和定位信息。文档内容也属于不受信任输入，其中“忽略规则”类指令不能获得系统权限。\n\nClaude 的原生 Citations 可提供结构化来源位置，减少手工编造定位的空间，但**不能保证结论正确，也不能自动满足合规**。支持格式与其他功能的组合限制需查 [官方文档](https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fbuild-with-claude\u002Fcitations)。\n\n## 忠实与正确：两个问题\n\n若旧规范写错了一条定义，模型完整复述并正确引用，答案可以“忠实于资料”却“事实错误”。资料治理要独立维护生效版本、勘误、权威级别与更新时间；冲突时展示差异、优先经过确认的来源，或拒绝给唯一答案。\n\n同样，引用覆盖率高只说明形式覆盖。真正要抽检的是引用支撑率和结论正确率。对数值与配置还可用计算器、配置校验器或确定性规则检查结果，不能只靠另一模型看起来认可。\n\n## 分层评测：指标不是一对一修复按钮\n\n| 层次 \u002F 指标 | 回答的问题 | 需要注意 |\n| --- | --- | --- |\n| 检索 Recall@k、MRR、nDCG | 证据是否找到，排序是否合理 | 需要人工或可信标注 |\n| Faithfulness | 结论是否被上下文支持 | 不是世界事实正确率 |\n| Answer relevancy | 回答是否切题 | 切题也可能答错 |\n| Context precision \u002F recall | 上下文是否相关、证据是否完整 | 不同实现需要的参考数据不同 |\n| 业务正确率、拒答适当率 | 任务是否完成、该答与该拒是否合理 | 按业务定义标准答案和边界 |\n\nRagas 提供多种带或不带参考答案的指标实现，不能说所有指标都是同一种 LLM 裁判。[Ragas 指标文档](https:\u002F\u002Fdocs.ragas.io\u002Fen\u002Fstable\u002Fconcepts\u002Fmetrics\u002Favailable_metrics\u002F)\n\n指标与修复手段也不是一一对应。Faithfulness 低可能是检索无关、证据冲突、提示没约束或模型推理失误。排查时用**对照输入**：同一问题分别提供实际检索结果与人工确认的完整证据；若后者仍答错，继续调检索通常解决不了核心问题。\n\n## 一个完整实验怎么记录\n\n用例同时保存问题、必要证据位置、标准答案、允许拒答条件和资料版本。运行时保存召回、重排、最终上下文、回答、引用、延迟与费用。每次只改变一个主要变量，比较逐用例差异，不只看总分。\n\n例如 100 个标注问题中，重排后证据覆盖增加，但最终正确率不变：检查新增证据是否太长、重复、与旧资料冲突，或模型没有利用它。若回答正确而引用错误，修引用机制而非靠提高 top-k 掩盖。\n\nLLM 裁判要与人工样本校准，高风险结论由人工或确定性工具复核。评测本身可能慢且有误差，不能把未经校准的离线分数直接当每条在线回答的阻断阈值。\n\n## 落地检查\n\n1. 候选数和 top-k 通过实验确定，按 token 预算选择互补证据。\n2. 重排有超时、降级与独立观测，权限过滤贯穿每一路。\n3. 引用 ID、版本、定位、支撑和覆盖都验证。\n4. 无证据则追问或拒答，来源冲突有明确策略。\n5. 同时报检索质量、事实正确、引用质量、延迟和费用。\n6. 筛选并依法脱敏的线上坏案例进入 [评测闭环](\u002Farticles\u002Fllm-eval-loop)。\n\n## 参考资料\n\n- [Sentence Transformers — Retrieve and rerank](https:\u002F\u002Fwww.sbert.net\u002Fexamples\u002Fsentence_transformer\u002Fapplications\u002Fretrieve_rerank\u002FREADME.html)\n- [Cohere — Rerank](https:\u002F\u002Fdocs.cohere.com\u002Fdocs\u002Frerank-overview)\n- [Claude — Citations](https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fbuild-with-claude\u002Fcitations)\n- [Ragas — Available metrics](https:\u002F\u002Fdocs.ragas.io\u002Fen\u002Fstable\u002Fconcepts\u002Fmetrics\u002Favailable_metrics\u002F)\n- [Anthropic — Contextual Retrieval 实验](https:\u002F\u002Fwww.anthropic.com\u002Fengineering\u002Fcontextual-retrieval)",[20,25,29,33,37],{"title":21,"url":22,"type":23,"note":24,"checked":8},"Sentence Transformers — Retrieve and rerank","https:\u002F\u002Fwww.sbert.net\u002Fexamples\u002Fsentence_transformer\u002Fapplications\u002Fretrieve_rerank\u002FREADME.html","官方文档","召回加交叉编码器重排的实现思路；需比较质量、延迟与资源代价。",{"title":26,"url":27,"type":23,"note":28,"checked":8},"Cohere — Rerank","https:\u002F\u002Fdocs.cohere.com\u002Fdocs\u002Frerank-overview","重排 API 能力与用法；具体模型、输入和成本以所用版本为准。",{"title":30,"url":31,"type":23,"note":32,"checked":8},"Claude — Citations","https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fbuild-with-claude\u002Fcitations","原生来源定位与支持限制；引用存在不保证结论正确或自动合规。",{"title":34,"url":35,"type":23,"note":36,"checked":8},"Ragas — Available metrics","https:\u002F\u002Fdocs.ragas.io\u002Fen\u002Fstable\u002Fconcepts\u002Fmetrics\u002Favailable_metrics\u002F","RAG 评测指标的定义与适用问题；指标不能当一对一修复指令。",{"title":38,"url":39,"type":40,"note":41,"checked":8},"Anthropic — Contextual Retrieval 实验","https:\u002F\u002Fwww.anthropic.com\u002Fengineering\u002Fcontextual-retrieval","工程报告","为块补充定位上下文并组合向量、BM25 与重排的实验；检索失败率下降不等于答案正确率同比提高。",{"slug":43,"title":44,"date":45,"updated":8,"tags":46,"summary":51,"draft":15,"lang":16,"minutes":52},"function-calling-structured-output","Function Calling 与结构化输出：让 LLM 可靠地动手","2026-03-17",[47,48,49,50],"LLM","Tool Use","Agent","结构化输出","分清调用协议与输出格式，比较 OpenAI、Claude 的字段；用完整 schema 和工具循环说明参数验证、授权、并发、幂等与失败处理。",6,{"slug":54,"title":55,"date":56,"updated":8,"tags":57,"summary":61,"draft":15,"lang":16,"minutes":52},"rag-02-vector-hybrid-search","RAG 全链路（二）：向量库选型与混合检索","2026-01-20",[10,58,59,60],"向量数据库","检索","pgvector","按查询、权限过滤和运维条件选择向量库；解释向量与词法检索的互补、RRF 融合，以及 ANN 召回和业务证据召回的区别。",{"type":63,"children":64},"root",[65,86,93,98,119,128,137,142,147,152,157,162,167,192,197,419,424,444,450,455,460,466,589,600,612,617,622,627,632,637,677],{"type":66,"tag":67,"props":68,"children":69},"element","p",{},[70,78,80],{"type":66,"tag":71,"props":72,"children":74},"a",{"href":73},"\u002Farticles\u002Frag-02-vector-hybrid-search",[75],{"type":76,"value":77},"text","混合检索",{"type":76,"value":79},"给出了候选，但候选相关不代表足以回答问题。这一篇把三件事接起来：",{"type":66,"tag":81,"props":82,"children":83},"strong",{},[84],{"type":76,"value":85},"重排决定优先读哪些材料，引用让结论可追溯，评测检验整条链路是否有效。",{"type":66,"tag":87,"props":88,"children":90},"h2",{"id":89},"重排是第二次更细的相关性判断",[91],{"type":76,"value":92},"重排是第二次、更细的相关性判断",{"type":66,"tag":67,"props":94,"children":95},{},[96],{"type":76,"value":97},"Bi-encoder 分别编码查询与文档，文档向量可离线计算，适合大范围检索。Cross-encoder 将查询与候选一起处理，让它们直接交互，逐对评估相关性，适合较少候选的重排。它通常比简单向量相似度昂贵，收益和延迟依赖模型与文本长度。",{"type":66,"tag":67,"props":99,"children":100},{},[101,103,108,110,117],{"type":76,"value":102},"一个",{"type":66,"tag":81,"props":104,"children":105},{},[106],{"type":76,"value":107},"实验起点",{"type":76,"value":109},"可以是两路召回各 50 条、去重后重排，再按 token 预算选择最终上下文。候选数、输出条数都要调，不能把 ",{"type":66,"tag":111,"props":112,"children":114},"code",{"className":113},[],[115],{"type":76,"value":116},"150 → 20",{"type":76,"value":118}," 视为通用标准。",{"type":66,"tag":67,"props":120,"children":121},{},[122],{"type":66,"tag":123,"props":124,"children":127},"img",{"alt":125,"src":126},"重排漏斗与验证：候选召回、相关性重排、预算选择、引用和最终答案评测","\u002Fdiagrams\u002Frag-rerank.svg",[],{"type":66,"tag":67,"props":129,"children":130},{},[131],{"type":66,"tag":132,"props":133,"children":134},"em",{},[135],{"type":76,"value":136},"图 1 · 数量是待调参数，最终供给还要考虑去重、证据覆盖与 token 预算。",{"type":66,"tag":87,"props":138,"children":140},{"id":139},"多给几段并不总是更好",[141],{"type":76,"value":139},{"type":66,"tag":67,"props":143,"children":144},{},[145],{"type":76,"value":146},"候选过少可能漏掉多跳证据，供给过多则会增加延迟、引入冲突和分散注意力。比较不同 top-k 时，固定模型与资料版本，记录最终正确率、证据覆盖、输入 token 和 p95。",{"type":66,"tag":67,"props":148,"children":149},{},[150],{"type":76,"value":151},"选择上下文时还应：合并同父文档相邻段落、限制重复、保留互补证据、优先有效版本。若预算不够，把检索到的三份重复说明改成一份，往往比简单砍掉最后一条更合理。",{"type":66,"tag":67,"props":153,"children":154},{},[155],{"type":76,"value":156},"若 reranker 超时，可使用已验证的粗排结果降级；若证据不足，选择追问或说明资料不足。不能为达到固定 top-k 把低相关内容强行凑满。",{"type":66,"tag":87,"props":158,"children":160},{"id":159},"引用需要四项验证",[161],{"type":76,"value":159},{"type":66,"tag":67,"props":163,"children":164},{},[165],{"type":76,"value":166},"“有一个角标”不代表正确。服务端至少检查：",{"type":66,"tag":168,"props":169,"children":170},"ol",{},[171,177,182,187],{"type":66,"tag":172,"props":173,"children":174},"li",{},[175],{"type":76,"value":176},"引用 ID 确实在本次授权检索结果中。",{"type":66,"tag":172,"props":178,"children":179},{},[180],{"type":76,"value":181},"引用能定位到当时的文档版本与原文片段。",{"type":66,"tag":172,"props":183,"children":184},{},[185],{"type":76,"value":186},"原文支持对应结论，不能只与主题相近。",{"type":66,"tag":172,"props":188,"children":189},{},[190],{"type":76,"value":191},"关键结论覆盖充分，不能引用第一句后让后文自由发挥。",{"type":66,"tag":67,"props":193,"children":194},{},[195],{"type":76,"value":196},"可以让模型输出以下业务结构；这是中立示例，并非特定厂商 API 请求：",{"type":66,"tag":198,"props":199,"children":204},"pre",{"className":200,"code":201,"language":202,"meta":203,"style":203},"language-json shiki shiki-themes vitesse-light vitesse-light","{\n  \"claims\": [\n    { \"text\": \"该版本的连接超时配置为 5 秒。\", \"evidence_ids\": [\"runbook-42:rev-3:p-2\"] }\n  ],\n  \"missing_evidence\": [],\n  \"status\": \"answered\"\n}\n","json","",[205],{"type":66,"tag":111,"props":206,"children":207},{"__ignoreMap":203},[208,220,251,342,351,376,410],{"type":66,"tag":209,"props":210,"children":213},"span",{"class":211,"line":212},"line",1,[214],{"type":66,"tag":209,"props":215,"children":217},{"style":216},"--shiki-default:#999999;--shiki-dark:#999999",[218],{"type":76,"value":219},"{\n",{"type":66,"tag":209,"props":221,"children":223},{"class":211,"line":222},2,[224,230,236,241,246],{"type":66,"tag":209,"props":225,"children":227},{"style":226},"--shiki-default:#99841877;--shiki-dark:#99841877",[228],{"type":76,"value":229},"  \"",{"type":66,"tag":209,"props":231,"children":233},{"style":232},"--shiki-default:#998418;--shiki-dark:#998418",[234],{"type":76,"value":235},"claims",{"type":66,"tag":209,"props":237,"children":238},{"style":226},[239],{"type":76,"value":240},"\"",{"type":66,"tag":209,"props":242,"children":243},{"style":216},[244],{"type":76,"value":245},":",{"type":66,"tag":209,"props":247,"children":248},{"style":216},[249],{"type":76,"value":250}," [\n",{"type":66,"tag":209,"props":252,"children":254},{"class":211,"line":253},3,[255,260,265,269,273,277,282,288,292,297,301,306,310,314,319,323,328,332,337],{"type":66,"tag":209,"props":256,"children":257},{"style":216},[258],{"type":76,"value":259},"    {",{"type":66,"tag":209,"props":261,"children":262},{"style":226},[263],{"type":76,"value":264}," \"",{"type":66,"tag":209,"props":266,"children":267},{"style":232},[268],{"type":76,"value":76},{"type":66,"tag":209,"props":270,"children":271},{"style":226},[272],{"type":76,"value":240},{"type":66,"tag":209,"props":274,"children":275},{"style":216},[276],{"type":76,"value":245},{"type":66,"tag":209,"props":278,"children":280},{"style":279},"--shiki-default:#B5695977;--shiki-dark:#B5695977",[281],{"type":76,"value":264},{"type":66,"tag":209,"props":283,"children":285},{"style":284},"--shiki-default:#B56959;--shiki-dark:#B56959",[286],{"type":76,"value":287},"该版本的连接超时配置为 5 秒。",{"type":66,"tag":209,"props":289,"children":290},{"style":279},[291],{"type":76,"value":240},{"type":66,"tag":209,"props":293,"children":294},{"style":216},[295],{"type":76,"value":296},",",{"type":66,"tag":209,"props":298,"children":299},{"style":226},[300],{"type":76,"value":264},{"type":66,"tag":209,"props":302,"children":303},{"style":232},[304],{"type":76,"value":305},"evidence_ids",{"type":66,"tag":209,"props":307,"children":308},{"style":226},[309],{"type":76,"value":240},{"type":66,"tag":209,"props":311,"children":312},{"style":216},[313],{"type":76,"value":245},{"type":66,"tag":209,"props":315,"children":316},{"style":216},[317],{"type":76,"value":318}," [",{"type":66,"tag":209,"props":320,"children":321},{"style":279},[322],{"type":76,"value":240},{"type":66,"tag":209,"props":324,"children":325},{"style":284},[326],{"type":76,"value":327},"runbook-42:rev-3:p-2",{"type":66,"tag":209,"props":329,"children":330},{"style":279},[331],{"type":76,"value":240},{"type":66,"tag":209,"props":333,"children":334},{"style":216},[335],{"type":76,"value":336},"]",{"type":66,"tag":209,"props":338,"children":339},{"style":216},[340],{"type":76,"value":341}," }\n",{"type":66,"tag":209,"props":343,"children":345},{"class":211,"line":344},4,[346],{"type":66,"tag":209,"props":347,"children":348},{"style":216},[349],{"type":76,"value":350},"  ],\n",{"type":66,"tag":209,"props":352,"children":353},{"class":211,"line":17},[354,358,363,367,371],{"type":66,"tag":209,"props":355,"children":356},{"style":226},[357],{"type":76,"value":229},{"type":66,"tag":209,"props":359,"children":360},{"style":232},[361],{"type":76,"value":362},"missing_evidence",{"type":66,"tag":209,"props":364,"children":365},{"style":226},[366],{"type":76,"value":240},{"type":66,"tag":209,"props":368,"children":369},{"style":216},[370],{"type":76,"value":245},{"type":66,"tag":209,"props":372,"children":373},{"style":216},[374],{"type":76,"value":375}," [],\n",{"type":66,"tag":209,"props":377,"children":378},{"class":211,"line":52},[379,383,388,392,396,400,405],{"type":66,"tag":209,"props":380,"children":381},{"style":226},[382],{"type":76,"value":229},{"type":66,"tag":209,"props":384,"children":385},{"style":232},[386],{"type":76,"value":387},"status",{"type":66,"tag":209,"props":389,"children":390},{"style":226},[391],{"type":76,"value":240},{"type":66,"tag":209,"props":393,"children":394},{"style":216},[395],{"type":76,"value":245},{"type":66,"tag":209,"props":397,"children":398},{"style":279},[399],{"type":76,"value":264},{"type":66,"tag":209,"props":401,"children":402},{"style":284},[403],{"type":76,"value":404},"answered",{"type":66,"tag":209,"props":406,"children":407},{"style":279},[408],{"type":76,"value":409},"\"\n",{"type":66,"tag":209,"props":411,"children":413},{"class":211,"line":412},7,[414],{"type":66,"tag":209,"props":415,"children":416},{"style":216},[417],{"type":76,"value":418},"}\n",{"type":66,"tag":67,"props":420,"children":421},{},[422],{"type":76,"value":423},"模型不得生成任意 URL；服务端用已知 ID 解析链接和定位信息。文档内容也属于不受信任输入，其中“忽略规则”类指令不能获得系统权限。",{"type":66,"tag":67,"props":425,"children":426},{},[427,429,434,436,442],{"type":76,"value":428},"Claude 的原生 Citations 可提供结构化来源位置，减少手工编造定位的空间，但",{"type":66,"tag":81,"props":430,"children":431},{},[432],{"type":76,"value":433},"不能保证结论正确，也不能自动满足合规",{"type":76,"value":435},"。支持格式与其他功能的组合限制需查 ",{"type":66,"tag":71,"props":437,"children":440},{"href":31,"rel":438},[439],"nofollow",[441],{"type":76,"value":23},{"type":76,"value":443},"。",{"type":66,"tag":87,"props":445,"children":447},{"id":446},"忠实与正确两个问题",[448],{"type":76,"value":449},"忠实与正确：两个问题",{"type":66,"tag":67,"props":451,"children":452},{},[453],{"type":76,"value":454},"若旧规范写错了一条定义，模型完整复述并正确引用，答案可以“忠实于资料”却“事实错误”。资料治理要独立维护生效版本、勘误、权威级别与更新时间；冲突时展示差异、优先经过确认的来源，或拒绝给唯一答案。",{"type":66,"tag":67,"props":456,"children":457},{},[458],{"type":76,"value":459},"同样，引用覆盖率高只说明形式覆盖。真正要抽检的是引用支撑率和结论正确率。对数值与配置还可用计算器、配置校验器或确定性规则检查结果，不能只靠另一模型看起来认可。",{"type":66,"tag":87,"props":461,"children":463},{"id":462},"分层评测指标不是一对一修复按钮",[464],{"type":76,"value":465},"分层评测：指标不是一对一修复按钮",{"type":66,"tag":467,"props":468,"children":469},"table",{},[470,494],{"type":66,"tag":471,"props":472,"children":473},"thead",{},[474],{"type":66,"tag":475,"props":476,"children":477},"tr",{},[478,484,489],{"type":66,"tag":479,"props":480,"children":481},"th",{},[482],{"type":76,"value":483},"层次 \u002F 指标",{"type":66,"tag":479,"props":485,"children":486},{},[487],{"type":76,"value":488},"回答的问题",{"type":66,"tag":479,"props":490,"children":491},{},[492],{"type":76,"value":493},"需要注意",{"type":66,"tag":495,"props":496,"children":497},"tbody",{},[498,517,535,553,571],{"type":66,"tag":475,"props":499,"children":500},{},[501,507,512],{"type":66,"tag":502,"props":503,"children":504},"td",{},[505],{"type":76,"value":506},"检索 Recall@k、MRR、nDCG",{"type":66,"tag":502,"props":508,"children":509},{},[510],{"type":76,"value":511},"证据是否找到，排序是否合理",{"type":66,"tag":502,"props":513,"children":514},{},[515],{"type":76,"value":516},"需要人工或可信标注",{"type":66,"tag":475,"props":518,"children":519},{},[520,525,530],{"type":66,"tag":502,"props":521,"children":522},{},[523],{"type":76,"value":524},"Faithfulness",{"type":66,"tag":502,"props":526,"children":527},{},[528],{"type":76,"value":529},"结论是否被上下文支持",{"type":66,"tag":502,"props":531,"children":532},{},[533],{"type":76,"value":534},"不是世界事实正确率",{"type":66,"tag":475,"props":536,"children":537},{},[538,543,548],{"type":66,"tag":502,"props":539,"children":540},{},[541],{"type":76,"value":542},"Answer relevancy",{"type":66,"tag":502,"props":544,"children":545},{},[546],{"type":76,"value":547},"回答是否切题",{"type":66,"tag":502,"props":549,"children":550},{},[551],{"type":76,"value":552},"切题也可能答错",{"type":66,"tag":475,"props":554,"children":555},{},[556,561,566],{"type":66,"tag":502,"props":557,"children":558},{},[559],{"type":76,"value":560},"Context precision \u002F recall",{"type":66,"tag":502,"props":562,"children":563},{},[564],{"type":76,"value":565},"上下文是否相关、证据是否完整",{"type":66,"tag":502,"props":567,"children":568},{},[569],{"type":76,"value":570},"不同实现需要的参考数据不同",{"type":66,"tag":475,"props":572,"children":573},{},[574,579,584],{"type":66,"tag":502,"props":575,"children":576},{},[577],{"type":76,"value":578},"业务正确率、拒答适当率",{"type":66,"tag":502,"props":580,"children":581},{},[582],{"type":76,"value":583},"任务是否完成、该答与该拒是否合理",{"type":66,"tag":502,"props":585,"children":586},{},[587],{"type":76,"value":588},"按业务定义标准答案和边界",{"type":66,"tag":67,"props":590,"children":591},{},[592,594],{"type":76,"value":593},"Ragas 提供多种带或不带参考答案的指标实现，不能说所有指标都是同一种 LLM 裁判。",{"type":66,"tag":71,"props":595,"children":597},{"href":35,"rel":596},[439],[598],{"type":76,"value":599},"Ragas 指标文档",{"type":66,"tag":67,"props":601,"children":602},{},[603,605,610],{"type":76,"value":604},"指标与修复手段也不是一一对应。Faithfulness 低可能是检索无关、证据冲突、提示没约束或模型推理失误。排查时用",{"type":66,"tag":81,"props":606,"children":607},{},[608],{"type":76,"value":609},"对照输入",{"type":76,"value":611},"：同一问题分别提供实际检索结果与人工确认的完整证据；若后者仍答错，继续调检索通常解决不了核心问题。",{"type":66,"tag":87,"props":613,"children":615},{"id":614},"一个完整实验怎么记录",[616],{"type":76,"value":614},{"type":66,"tag":67,"props":618,"children":619},{},[620],{"type":76,"value":621},"用例同时保存问题、必要证据位置、标准答案、允许拒答条件和资料版本。运行时保存召回、重排、最终上下文、回答、引用、延迟与费用。每次只改变一个主要变量，比较逐用例差异，不只看总分。",{"type":66,"tag":67,"props":623,"children":624},{},[625],{"type":76,"value":626},"例如 100 个标注问题中，重排后证据覆盖增加，但最终正确率不变：检查新增证据是否太长、重复、与旧资料冲突，或模型没有利用它。若回答正确而引用错误，修引用机制而非靠提高 top-k 掩盖。",{"type":66,"tag":67,"props":628,"children":629},{},[630],{"type":76,"value":631},"LLM 裁判要与人工样本校准，高风险结论由人工或确定性工具复核。评测本身可能慢且有误差，不能把未经校准的离线分数直接当每条在线回答的阻断阈值。",{"type":66,"tag":87,"props":633,"children":635},{"id":634},"落地检查",[636],{"type":76,"value":634},{"type":66,"tag":168,"props":638,"children":639},{},[640,645,650,655,660,665],{"type":66,"tag":172,"props":641,"children":642},{},[643],{"type":76,"value":644},"候选数和 top-k 通过实验确定，按 token 预算选择互补证据。",{"type":66,"tag":172,"props":646,"children":647},{},[648],{"type":76,"value":649},"重排有超时、降级与独立观测，权限过滤贯穿每一路。",{"type":66,"tag":172,"props":651,"children":652},{},[653],{"type":76,"value":654},"引用 ID、版本、定位、支撑和覆盖都验证。",{"type":66,"tag":172,"props":656,"children":657},{},[658],{"type":76,"value":659},"无证据则追问或拒答，来源冲突有明确策略。",{"type":66,"tag":172,"props":661,"children":662},{},[663],{"type":76,"value":664},"同时报检索质量、事实正确、引用质量、延迟和费用。",{"type":66,"tag":172,"props":666,"children":667},{},[668,670,676],{"type":76,"value":669},"筛选并依法脱敏的线上坏案例进入 ",{"type":66,"tag":71,"props":671,"children":673},{"href":672},"\u002Farticles\u002Fllm-eval-loop",[674],{"type":76,"value":675},"评测闭环",{"type":76,"value":443},{"type":66,"tag":87,"props":678,"children":680},{"id":679},"参考资料",[681],{"type":76,"value":679},[683,684,685,686,687,688,689,690],{"id":89,"depth":222,"text":92},{"id":139,"depth":222,"text":139},{"id":159,"depth":222,"text":159},{"id":446,"depth":222,"text":449},{"id":462,"depth":222,"text":465},{"id":614,"depth":222,"text":614},{"id":634,"depth":222,"text":634},{"id":679,"depth":222,"text":679},1792400815607]