[{"data":1,"prerenderedAt":926},["ShallowReactive",2],{"article:rag-01-chunking":3},{"meta":4,"raw":18,"references":19,"prev":39,"next":48,"body":57,"toc":918},{"slug":5,"title":6,"date":7,"updated":8,"tags":9,"summary":14,"draft":15,"lang":16,"minutes":17},"rag-01-chunking","RAG 全链路（一）：文档切分——让检索结果保留完整语境","2025-12-16","2026-10-11",[10,11,12,13],"RAG","检索","Embedding","LLM","从解析和元数据开始，比较固定窗口、结构化切分、父子块与上下文补注；用一份故障手册实例解释检索粒度、供给粒度和版本追踪。",false,"zh-CN",5,"RAG（Retrieval-Augmented Generation，检索增强生成）先从外部资料找证据，再让模型回答。资料被切成的小块叫 chunk；Embedding 则把文本映射成可比较的向量。**切分决定检索单元里保留多少语境**，但答案质量还取决于解析、检索、模型和评测，不能只归因于切分。\n\n比如一段只写“营收增长 3%”，丢掉了公司与季度，检索和回答都会更难。保留标题路径或父文档关系，比只改变 chunk 大小更直接。本文是三篇系列的起点，后续分别讲 [检索](\u002Farticles\u002Frag-02-vector-hybrid-search)与 [重排和引用](\u002Farticles\u002Frag-03-rerank-citation-eval)。\n\n## 先解析正确，再谈怎么切\n\nPDF 的阅读顺序、表格列名、公式、脚注和 OCR 错误，都可能在切分前损坏。双栏报告若被逐行横向拼接，再精细的切分也只是把乱码切得整齐。\n\n入库前先做结构抽取、重复页眉清理和抽样核验；图片保留原图引用及可检索图注，公式保留表达式与文字解释。OCR 和自动图注会出错，不能替代原图作为最终证据。版权、租户和访问权限也要跟随文档进入元数据。\n\n## 四种策略，各有适用条件\n\n| 方法 | 适合什么 | 主要代价 |\n| --- | --- | --- |\n| 固定窗口加重叠 | 结构弱的文本、快速基线 | 可能切断句子，重叠增加索引量 |\n| 按标题、段落、条目结构切 | 规范、技术文档、操作手册 | 需要可靠解析，超长段仍需递归拆分 |\n| 父子块 | 小块检索、大块解释的任务 | 父块去重、长度控制和权限继承 |\n| 上下文补注 | 孤立段落缺文档语境 | 额外生成成本，补注自身可能出错 |\n\n固定窗口也可优先在句子边界截断，不一定要机械按字符拦腰切。可先比较几组 token 长度与重叠比例；这些是实验变量，没有适用于所有语料的固定最佳值。\n\n父子块把**检索粒度**与**供给粒度**解耦：用小块找准位置，返回父块提供语境。父块可能很大，需要按 token 预算截取；多个子块命中同一父块要去重，不能把一个章节重复喂五次。\n\n![切分方法对比：固定窗口、结构边界、父子块与上下文补注](\u002Fdiagrams\u002Frag-chunking.svg)\n\n*图 1 · 先保住原始结构，再按查询需求决定检索和供给粒度。*\n\n## 一份故障手册怎么拆\n\n一份故障手册可能包含症状、前置条件、诊断步骤、修复命令与验证方法。可以把单个故障条目作为父文档，把诊断与修复步骤作为子块，保留统一的 `entry_id`。只读用户与运维人员可访问的段落可能不同，取回父块时仍要逐段检查权限，不能因需要完整语境就泄露受限命令。\n\n```json\n{\n  \"chunk_id\": \"runbook-42:rev-3:diagnosis-2\",\n  \"parent_id\": \"runbook-42:rev-3\",\n  \"document_id\": \"service-runbook-8\",\n  \"source_revision\": \"rev-3\",\n  \"section_path\": [\"连接故障\", \"请求超时\"],\n  \"content_type\": \"diagnosis\",\n  \"tenant_id\": \"tenant-a\",\n  \"source_locator\": { \"page\": 27, \"paragraph\": \"p-8\" }\n}\n```\n\n身份、版本和来源定位分开保存：同一条目修订后产生新 revision；引用必须能定位当时使用的版本。删除或撤权时，父子块、向量、全文索引和答案缓存一起处理。标题路径不是权限控制，客户端的授权过滤需在服务端执行。\n\n## 上下文补注：公开实验怎么读\n\nAnthropic 的 Contextual Retrieval 为块生成简短定位说明，再与原块一起建向量和词法索引。其公开实验中，top-20 检索失败率依次如下：[原始报告](https:\u002F\u002Fwww.anthropic.com\u002Fengineering\u002Fcontextual-retrieval)\n\n| 配置 | 失败率 | 相对 5.7% 基线降幅 |\n| --- | --- | --- |\n| 基础检索 | 5.7% | — |\n| 上下文向量检索 | 3.7% | 约 35% |\n| 再结合上下文 BM25 | 2.9% | 约 49% |\n| 再结合重排 | 1.9% | 约 67% |\n\n5.7% 到 1.9% 是减少 **3.8 个百分点**，或相对减少约 67%；不是答案正确率提高了 67%。这些是报告语料、模型和流程下的结果，不能直接推广到其他语言和领域的资料库。\n\n补注要单独标记为派生内容，保留原文；它能帮助检索，但不能伪装成权威证据。用同一评测集比较结构标题补全、父子块和 LLM 补注，再决定额外成本是否值得。\n\nLate Chunking 是另一条路线：先在长上下文模型里编码，再按块池化向量，使块保留更多全文语境。它要求相应模型和实现，不是所有 Embedding API 都能支持。[Jina 原始介绍](https:\u002F\u002Fjina.ai\u002Fnews\u002Flate-chunking-in-long-context-embedding-models\u002F)\n\n## 切分实验怎么做\n\n固定语料版本、Embedding 和检索参数，比较不同切分配置。用真实问题标注支持答案的证据位置，避免只标 chunk ID：重新切分后 chunk 变了，证据位置仍能对应。\n\n至少看证据召回、最终正确率、引用支撑率、输入 token、索引体积和延迟。对跨段推理、表格问题、图题单独分组。小样本抽检能发现坏块，却不能代替全链路评测。\n\n若题目需要三条证据，Recall@k 可能不错但仍缺关键一条；再看“全部必要证据是否齐全”。若重叠导致多个结果几乎相同，先去重和限制父文档占比，不要立刻增加 top-k。\n\n## 落地检查\n\n1. 检查解析顺序、表格、公式与 OCR，保留原始载体。\n2. 每块记录父文档、内容类型、版本、来源与权限。\n3. 结构化语料优先比较结构切分，超长单元用父子关系处理。\n4. 补注标记为派生内容，抽检生成错误，不作为独立证据。\n5. 固定其他变量比较切分，用最终答案质量和成本一起选参数。\n6. 更新、删除与撤权同步覆盖各索引和缓存。\n\n## 参考资料\n\n- [Anthropic — Contextual Retrieval](https:\u002F\u002Fwww.anthropic.com\u002Fengineering\u002Fcontextual-retrieval)\n- [LlamaIndex — Node parsers](https:\u002F\u002Fdevelopers.llamaindex.ai\u002Fpython\u002Fframework\u002Fmodule_guides\u002Floading\u002Fnode_parsers\u002F)\n- [Pinecone — Chunking strategies](https:\u002F\u002Fwww.pinecone.io\u002Flearn\u002Fchunking-strategies\u002F)\n- [Jina — Late Chunking](https:\u002F\u002Fjina.ai\u002Fnews\u002Flate-chunking-in-long-context-embedding-models\u002F)",[20,25,30,35],{"title":21,"url":22,"type":23,"note":24,"checked":8},"Anthropic — Contextual Retrieval","https:\u002F\u002Fwww.anthropic.com\u002Fengineering\u002Fcontextual-retrieval","工程报告","为块补充定位上下文并组合向量、BM25 与重排的实验；检索失败率下降不等于答案正确率同比提高。",{"title":26,"url":27,"type":28,"note":29,"checked":8},"LlamaIndex — Node parsers","https:\u002F\u002Fdevelopers.llamaindex.ai\u002Fpython\u002Fframework\u002Fmodule_guides\u002Floading\u002Fnode_parsers\u002F","官方文档","结构化与递归 node parser 的接口；解析质量和权限由数据链路保证。",{"title":31,"url":32,"type":33,"note":34,"checked":8},"Pinecone — Chunking strategies","https:\u002F\u002Fwww.pinecone.io\u002Flearn\u002Fchunking-strategies\u002F","技术文章","切分策略的工程讨论；块大小不是可跨任务照搬的常数。",{"title":36,"url":37,"type":33,"note":38,"checked":8},"Jina — Late Chunking","https:\u002F\u002Fjina.ai\u002Fnews\u002Flate-chunking-in-long-context-embedding-models\u002F","Late Chunking 的模型上下文与池化方法介绍；区别于普通先切分后编码。",{"slug":40,"title":41,"date":42,"updated":8,"tags":43,"summary":46,"draft":15,"lang":16,"minutes":47},"rag-02-vector-hybrid-search","RAG 全链路（二）：向量库选型与混合检索","2026-01-20",[10,44,11,45],"向量数据库","pgvector","按查询、权限过滤和运维条件选择向量库；解释向量与词法检索的互补、RRF 融合，以及 ANN 召回和业务证据召回的区别。",6,{"slug":49,"title":50,"date":51,"updated":8,"tags":52,"summary":56,"draft":15,"lang":16,"minutes":17},"llm-fallback-design","把失败当默认前提：LLM 服务的重试、降级与兜底设计","2025-11-18",[13,53,54,55],"可靠性","架构","SLA","按错误语义决定重试，用总截止时间控制等待；分清模型重复调用与业务幂等，处理流式中断、缓存串用和降级质量，并用 SLO 验证兜底。",{"type":58,"children":59},"root",[60,76,97,104,109,114,120,225,230,249,258,267,272,286,703,708,714,726,825,837,842,853,858,863,868,873,878,913],{"type":61,"tag":62,"props":63,"children":64},"element","p",{},[65,68,74],{"type":66,"value":67},"text","RAG（Retrieval-Augmented Generation，检索增强生成）先从外部资料找证据，再让模型回答。资料被切成的小块叫 chunk；Embedding 则把文本映射成可比较的向量。",{"type":61,"tag":69,"props":70,"children":71},"strong",{},[72],{"type":66,"value":73},"切分决定检索单元里保留多少语境",{"type":66,"value":75},"，但答案质量还取决于解析、检索、模型和评测，不能只归因于切分。",{"type":61,"tag":62,"props":77,"children":78},{},[79,81,87,89,95],{"type":66,"value":80},"比如一段只写“营收增长 3%”，丢掉了公司与季度，检索和回答都会更难。保留标题路径或父文档关系，比只改变 chunk 大小更直接。本文是三篇系列的起点，后续分别讲 ",{"type":61,"tag":82,"props":83,"children":85},"a",{"href":84},"\u002Farticles\u002Frag-02-vector-hybrid-search",[86],{"type":66,"value":11},{"type":66,"value":88},"与 ",{"type":61,"tag":82,"props":90,"children":92},{"href":91},"\u002Farticles\u002Frag-03-rerank-citation-eval",[93],{"type":66,"value":94},"重排和引用",{"type":66,"value":96},"。",{"type":61,"tag":98,"props":99,"children":101},"h2",{"id":100},"先解析正确再谈怎么切",[102],{"type":66,"value":103},"先解析正确，再谈怎么切",{"type":61,"tag":62,"props":105,"children":106},{},[107],{"type":66,"value":108},"PDF 的阅读顺序、表格列名、公式、脚注和 OCR 错误，都可能在切分前损坏。双栏报告若被逐行横向拼接，再精细的切分也只是把乱码切得整齐。",{"type":61,"tag":62,"props":110,"children":111},{},[112],{"type":66,"value":113},"入库前先做结构抽取、重复页眉清理和抽样核验；图片保留原图引用及可检索图注，公式保留表达式与文字解释。OCR 和自动图注会出错，不能替代原图作为最终证据。版权、租户和访问权限也要跟随文档进入元数据。",{"type":61,"tag":98,"props":115,"children":117},{"id":116},"四种策略各有适用条件",[118],{"type":66,"value":119},"四种策略，各有适用条件",{"type":61,"tag":121,"props":122,"children":123},"table",{},[124,148],{"type":61,"tag":125,"props":126,"children":127},"thead",{},[128],{"type":61,"tag":129,"props":130,"children":131},"tr",{},[132,138,143],{"type":61,"tag":133,"props":134,"children":135},"th",{},[136],{"type":66,"value":137},"方法",{"type":61,"tag":133,"props":139,"children":140},{},[141],{"type":66,"value":142},"适合什么",{"type":61,"tag":133,"props":144,"children":145},{},[146],{"type":66,"value":147},"主要代价",{"type":61,"tag":149,"props":150,"children":151},"tbody",{},[152,171,189,207],{"type":61,"tag":129,"props":153,"children":154},{},[155,161,166],{"type":61,"tag":156,"props":157,"children":158},"td",{},[159],{"type":66,"value":160},"固定窗口加重叠",{"type":61,"tag":156,"props":162,"children":163},{},[164],{"type":66,"value":165},"结构弱的文本、快速基线",{"type":61,"tag":156,"props":167,"children":168},{},[169],{"type":66,"value":170},"可能切断句子，重叠增加索引量",{"type":61,"tag":129,"props":172,"children":173},{},[174,179,184],{"type":61,"tag":156,"props":175,"children":176},{},[177],{"type":66,"value":178},"按标题、段落、条目结构切",{"type":61,"tag":156,"props":180,"children":181},{},[182],{"type":66,"value":183},"规范、技术文档、操作手册",{"type":61,"tag":156,"props":185,"children":186},{},[187],{"type":66,"value":188},"需要可靠解析，超长段仍需递归拆分",{"type":61,"tag":129,"props":190,"children":191},{},[192,197,202],{"type":61,"tag":156,"props":193,"children":194},{},[195],{"type":66,"value":196},"父子块",{"type":61,"tag":156,"props":198,"children":199},{},[200],{"type":66,"value":201},"小块检索、大块解释的任务",{"type":61,"tag":156,"props":203,"children":204},{},[205],{"type":66,"value":206},"父块去重、长度控制和权限继承",{"type":61,"tag":129,"props":208,"children":209},{},[210,215,220],{"type":61,"tag":156,"props":211,"children":212},{},[213],{"type":66,"value":214},"上下文补注",{"type":61,"tag":156,"props":216,"children":217},{},[218],{"type":66,"value":219},"孤立段落缺文档语境",{"type":61,"tag":156,"props":221,"children":222},{},[223],{"type":66,"value":224},"额外生成成本，补注自身可能出错",{"type":61,"tag":62,"props":226,"children":227},{},[228],{"type":66,"value":229},"固定窗口也可优先在句子边界截断，不一定要机械按字符拦腰切。可先比较几组 token 长度与重叠比例；这些是实验变量，没有适用于所有语料的固定最佳值。",{"type":61,"tag":62,"props":231,"children":232},{},[233,235,240,242,247],{"type":66,"value":234},"父子块把",{"type":61,"tag":69,"props":236,"children":237},{},[238],{"type":66,"value":239},"检索粒度",{"type":66,"value":241},"与",{"type":61,"tag":69,"props":243,"children":244},{},[245],{"type":66,"value":246},"供给粒度",{"type":66,"value":248},"解耦：用小块找准位置，返回父块提供语境。父块可能很大，需要按 token 预算截取；多个子块命中同一父块要去重，不能把一个章节重复喂五次。",{"type":61,"tag":62,"props":250,"children":251},{},[252],{"type":61,"tag":253,"props":254,"children":257},"img",{"alt":255,"src":256},"切分方法对比：固定窗口、结构边界、父子块与上下文补注","\u002Fdiagrams\u002Frag-chunking.svg",[],{"type":61,"tag":62,"props":259,"children":260},{},[261],{"type":61,"tag":262,"props":263,"children":264},"em",{},[265],{"type":66,"value":266},"图 1 · 先保住原始结构，再按查询需求决定检索和供给粒度。",{"type":61,"tag":98,"props":268,"children":270},{"id":269},"一份故障手册怎么拆",[271],{"type":66,"value":269},{"type":61,"tag":62,"props":273,"children":274},{},[275,277,284],{"type":66,"value":276},"一份故障手册可能包含症状、前置条件、诊断步骤、修复命令与验证方法。可以把单个故障条目作为父文档，把诊断与修复步骤作为子块，保留统一的 ",{"type":61,"tag":278,"props":279,"children":281},"code",{"className":280},[],[282],{"type":66,"value":283},"entry_id",{"type":66,"value":285},"。只读用户与运维人员可访问的段落可能不同，取回父块时仍要逐段检查权限，不能因需要完整语境就泄露受限命令。",{"type":61,"tag":287,"props":288,"children":293},"pre",{"className":289,"code":290,"language":291,"meta":292,"style":292},"language-json shiki shiki-themes vitesse-light vitesse-light","{\n  \"chunk_id\": \"runbook-42:rev-3:diagnosis-2\",\n  \"parent_id\": \"runbook-42:rev-3\",\n  \"document_id\": \"service-runbook-8\",\n  \"source_revision\": \"rev-3\",\n  \"section_path\": [\"连接故障\", \"请求超时\"],\n  \"content_type\": \"diagnosis\",\n  \"tenant_id\": \"tenant-a\",\n  \"source_locator\": { \"page\": 27, \"paragraph\": \"p-8\" }\n}\n","json","",[294],{"type":61,"tag":278,"props":295,"children":296},{"__ignoreMap":292},[297,309,356,394,432,469,530,568,606,694],{"type":61,"tag":298,"props":299,"children":302},"span",{"class":300,"line":301},"line",1,[303],{"type":61,"tag":298,"props":304,"children":306},{"style":305},"--shiki-default:#999999;--shiki-dark:#999999",[307],{"type":66,"value":308},"{\n",{"type":61,"tag":298,"props":310,"children":312},{"class":300,"line":311},2,[313,319,325,330,335,341,347,351],{"type":61,"tag":298,"props":314,"children":316},{"style":315},"--shiki-default:#99841877;--shiki-dark:#99841877",[317],{"type":66,"value":318},"  \"",{"type":61,"tag":298,"props":320,"children":322},{"style":321},"--shiki-default:#998418;--shiki-dark:#998418",[323],{"type":66,"value":324},"chunk_id",{"type":61,"tag":298,"props":326,"children":327},{"style":315},[328],{"type":66,"value":329},"\"",{"type":61,"tag":298,"props":331,"children":332},{"style":305},[333],{"type":66,"value":334},":",{"type":61,"tag":298,"props":336,"children":338},{"style":337},"--shiki-default:#B5695977;--shiki-dark:#B5695977",[339],{"type":66,"value":340}," \"",{"type":61,"tag":298,"props":342,"children":344},{"style":343},"--shiki-default:#B56959;--shiki-dark:#B56959",[345],{"type":66,"value":346},"runbook-42:rev-3:diagnosis-2",{"type":61,"tag":298,"props":348,"children":349},{"style":337},[350],{"type":66,"value":329},{"type":61,"tag":298,"props":352,"children":353},{"style":305},[354],{"type":66,"value":355},",\n",{"type":61,"tag":298,"props":357,"children":359},{"class":300,"line":358},3,[360,364,369,373,377,381,386,390],{"type":61,"tag":298,"props":361,"children":362},{"style":315},[363],{"type":66,"value":318},{"type":61,"tag":298,"props":365,"children":366},{"style":321},[367],{"type":66,"value":368},"parent_id",{"type":61,"tag":298,"props":370,"children":371},{"style":315},[372],{"type":66,"value":329},{"type":61,"tag":298,"props":374,"children":375},{"style":305},[376],{"type":66,"value":334},{"type":61,"tag":298,"props":378,"children":379},{"style":337},[380],{"type":66,"value":340},{"type":61,"tag":298,"props":382,"children":383},{"style":343},[384],{"type":66,"value":385},"runbook-42:rev-3",{"type":61,"tag":298,"props":387,"children":388},{"style":337},[389],{"type":66,"value":329},{"type":61,"tag":298,"props":391,"children":392},{"style":305},[393],{"type":66,"value":355},{"type":61,"tag":298,"props":395,"children":397},{"class":300,"line":396},4,[398,402,407,411,415,419,424,428],{"type":61,"tag":298,"props":399,"children":400},{"style":315},[401],{"type":66,"value":318},{"type":61,"tag":298,"props":403,"children":404},{"style":321},[405],{"type":66,"value":406},"document_id",{"type":61,"tag":298,"props":408,"children":409},{"style":315},[410],{"type":66,"value":329},{"type":61,"tag":298,"props":412,"children":413},{"style":305},[414],{"type":66,"value":334},{"type":61,"tag":298,"props":416,"children":417},{"style":337},[418],{"type":66,"value":340},{"type":61,"tag":298,"props":420,"children":421},{"style":343},[422],{"type":66,"value":423},"service-runbook-8",{"type":61,"tag":298,"props":425,"children":426},{"style":337},[427],{"type":66,"value":329},{"type":61,"tag":298,"props":429,"children":430},{"style":305},[431],{"type":66,"value":355},{"type":61,"tag":298,"props":433,"children":434},{"class":300,"line":17},[435,439,444,448,452,456,461,465],{"type":61,"tag":298,"props":436,"children":437},{"style":315},[438],{"type":66,"value":318},{"type":61,"tag":298,"props":440,"children":441},{"style":321},[442],{"type":66,"value":443},"source_revision",{"type":61,"tag":298,"props":445,"children":446},{"style":315},[447],{"type":66,"value":329},{"type":61,"tag":298,"props":449,"children":450},{"style":305},[451],{"type":66,"value":334},{"type":61,"tag":298,"props":453,"children":454},{"style":337},[455],{"type":66,"value":340},{"type":61,"tag":298,"props":457,"children":458},{"style":343},[459],{"type":66,"value":460},"rev-3",{"type":61,"tag":298,"props":462,"children":463},{"style":337},[464],{"type":66,"value":329},{"type":61,"tag":298,"props":466,"children":467},{"style":305},[468],{"type":66,"value":355},{"type":61,"tag":298,"props":470,"children":471},{"class":300,"line":47},[472,476,481,485,489,494,498,503,507,512,516,521,525],{"type":61,"tag":298,"props":473,"children":474},{"style":315},[475],{"type":66,"value":318},{"type":61,"tag":298,"props":477,"children":478},{"style":321},[479],{"type":66,"value":480},"section_path",{"type":61,"tag":298,"props":482,"children":483},{"style":315},[484],{"type":66,"value":329},{"type":61,"tag":298,"props":486,"children":487},{"style":305},[488],{"type":66,"value":334},{"type":61,"tag":298,"props":490,"children":491},{"style":305},[492],{"type":66,"value":493}," [",{"type":61,"tag":298,"props":495,"children":496},{"style":337},[497],{"type":66,"value":329},{"type":61,"tag":298,"props":499,"children":500},{"style":343},[501],{"type":66,"value":502},"连接故障",{"type":61,"tag":298,"props":504,"children":505},{"style":337},[506],{"type":66,"value":329},{"type":61,"tag":298,"props":508,"children":509},{"style":305},[510],{"type":66,"value":511},",",{"type":61,"tag":298,"props":513,"children":514},{"style":337},[515],{"type":66,"value":340},{"type":61,"tag":298,"props":517,"children":518},{"style":343},[519],{"type":66,"value":520},"请求超时",{"type":61,"tag":298,"props":522,"children":523},{"style":337},[524],{"type":66,"value":329},{"type":61,"tag":298,"props":526,"children":527},{"style":305},[528],{"type":66,"value":529},"],\n",{"type":61,"tag":298,"props":531,"children":533},{"class":300,"line":532},7,[534,538,543,547,551,555,560,564],{"type":61,"tag":298,"props":535,"children":536},{"style":315},[537],{"type":66,"value":318},{"type":61,"tag":298,"props":539,"children":540},{"style":321},[541],{"type":66,"value":542},"content_type",{"type":61,"tag":298,"props":544,"children":545},{"style":315},[546],{"type":66,"value":329},{"type":61,"tag":298,"props":548,"children":549},{"style":305},[550],{"type":66,"value":334},{"type":61,"tag":298,"props":552,"children":553},{"style":337},[554],{"type":66,"value":340},{"type":61,"tag":298,"props":556,"children":557},{"style":343},[558],{"type":66,"value":559},"diagnosis",{"type":61,"tag":298,"props":561,"children":562},{"style":337},[563],{"type":66,"value":329},{"type":61,"tag":298,"props":565,"children":566},{"style":305},[567],{"type":66,"value":355},{"type":61,"tag":298,"props":569,"children":571},{"class":300,"line":570},8,[572,576,581,585,589,593,598,602],{"type":61,"tag":298,"props":573,"children":574},{"style":315},[575],{"type":66,"value":318},{"type":61,"tag":298,"props":577,"children":578},{"style":321},[579],{"type":66,"value":580},"tenant_id",{"type":61,"tag":298,"props":582,"children":583},{"style":315},[584],{"type":66,"value":329},{"type":61,"tag":298,"props":586,"children":587},{"style":305},[588],{"type":66,"value":334},{"type":61,"tag":298,"props":590,"children":591},{"style":337},[592],{"type":66,"value":340},{"type":61,"tag":298,"props":594,"children":595},{"style":343},[596],{"type":66,"value":597},"tenant-a",{"type":61,"tag":298,"props":599,"children":600},{"style":337},[601],{"type":66,"value":329},{"type":61,"tag":298,"props":603,"children":604},{"style":305},[605],{"type":66,"value":355},{"type":61,"tag":298,"props":607,"children":609},{"class":300,"line":608},9,[610,614,619,623,627,632,636,641,645,649,655,659,663,668,672,676,680,685,689],{"type":61,"tag":298,"props":611,"children":612},{"style":315},[613],{"type":66,"value":318},{"type":61,"tag":298,"props":615,"children":616},{"style":321},[617],{"type":66,"value":618},"source_locator",{"type":61,"tag":298,"props":620,"children":621},{"style":315},[622],{"type":66,"value":329},{"type":61,"tag":298,"props":624,"children":625},{"style":305},[626],{"type":66,"value":334},{"type":61,"tag":298,"props":628,"children":629},{"style":305},[630],{"type":66,"value":631}," {",{"type":61,"tag":298,"props":633,"children":634},{"style":315},[635],{"type":66,"value":340},{"type":61,"tag":298,"props":637,"children":638},{"style":321},[639],{"type":66,"value":640},"page",{"type":61,"tag":298,"props":642,"children":643},{"style":315},[644],{"type":66,"value":329},{"type":61,"tag":298,"props":646,"children":647},{"style":305},[648],{"type":66,"value":334},{"type":61,"tag":298,"props":650,"children":652},{"style":651},"--shiki-default:#2F798A;--shiki-dark:#2F798A",[653],{"type":66,"value":654}," 27",{"type":61,"tag":298,"props":656,"children":657},{"style":305},[658],{"type":66,"value":511},{"type":61,"tag":298,"props":660,"children":661},{"style":315},[662],{"type":66,"value":340},{"type":61,"tag":298,"props":664,"children":665},{"style":321},[666],{"type":66,"value":667},"paragraph",{"type":61,"tag":298,"props":669,"children":670},{"style":315},[671],{"type":66,"value":329},{"type":61,"tag":298,"props":673,"children":674},{"style":305},[675],{"type":66,"value":334},{"type":61,"tag":298,"props":677,"children":678},{"style":337},[679],{"type":66,"value":340},{"type":61,"tag":298,"props":681,"children":682},{"style":343},[683],{"type":66,"value":684},"p-8",{"type":61,"tag":298,"props":686,"children":687},{"style":337},[688],{"type":66,"value":329},{"type":61,"tag":298,"props":690,"children":691},{"style":305},[692],{"type":66,"value":693}," }\n",{"type":61,"tag":298,"props":695,"children":697},{"class":300,"line":696},10,[698],{"type":61,"tag":298,"props":699,"children":700},{"style":305},[701],{"type":66,"value":702},"}\n",{"type":61,"tag":62,"props":704,"children":705},{},[706],{"type":66,"value":707},"身份、版本和来源定位分开保存：同一条目修订后产生新 revision；引用必须能定位当时使用的版本。删除或撤权时，父子块、向量、全文索引和答案缓存一起处理。标题路径不是权限控制，客户端的授权过滤需在服务端执行。",{"type":61,"tag":98,"props":709,"children":711},{"id":710},"上下文补注公开实验怎么读",[712],{"type":66,"value":713},"上下文补注：公开实验怎么读",{"type":61,"tag":62,"props":715,"children":716},{},[717,719],{"type":66,"value":718},"Anthropic 的 Contextual Retrieval 为块生成简短定位说明，再与原块一起建向量和词法索引。其公开实验中，top-20 检索失败率依次如下：",{"type":61,"tag":82,"props":720,"children":723},{"href":22,"rel":721},[722],"nofollow",[724],{"type":66,"value":725},"原始报告",{"type":61,"tag":121,"props":727,"children":728},{},[729,750],{"type":61,"tag":125,"props":730,"children":731},{},[732],{"type":61,"tag":129,"props":733,"children":734},{},[735,740,745],{"type":61,"tag":133,"props":736,"children":737},{},[738],{"type":66,"value":739},"配置",{"type":61,"tag":133,"props":741,"children":742},{},[743],{"type":66,"value":744},"失败率",{"type":61,"tag":133,"props":746,"children":747},{},[748],{"type":66,"value":749},"相对 5.7% 基线降幅",{"type":61,"tag":149,"props":751,"children":752},{},[753,771,789,807],{"type":61,"tag":129,"props":754,"children":755},{},[756,761,766],{"type":61,"tag":156,"props":757,"children":758},{},[759],{"type":66,"value":760},"基础检索",{"type":61,"tag":156,"props":762,"children":763},{},[764],{"type":66,"value":765},"5.7%",{"type":61,"tag":156,"props":767,"children":768},{},[769],{"type":66,"value":770},"—",{"type":61,"tag":129,"props":772,"children":773},{},[774,779,784],{"type":61,"tag":156,"props":775,"children":776},{},[777],{"type":66,"value":778},"上下文向量检索",{"type":61,"tag":156,"props":780,"children":781},{},[782],{"type":66,"value":783},"3.7%",{"type":61,"tag":156,"props":785,"children":786},{},[787],{"type":66,"value":788},"约 35%",{"type":61,"tag":129,"props":790,"children":791},{},[792,797,802],{"type":61,"tag":156,"props":793,"children":794},{},[795],{"type":66,"value":796},"再结合上下文 BM25",{"type":61,"tag":156,"props":798,"children":799},{},[800],{"type":66,"value":801},"2.9%",{"type":61,"tag":156,"props":803,"children":804},{},[805],{"type":66,"value":806},"约 49%",{"type":61,"tag":129,"props":808,"children":809},{},[810,815,820],{"type":61,"tag":156,"props":811,"children":812},{},[813],{"type":66,"value":814},"再结合重排",{"type":61,"tag":156,"props":816,"children":817},{},[818],{"type":66,"value":819},"1.9%",{"type":61,"tag":156,"props":821,"children":822},{},[823],{"type":66,"value":824},"约 67%",{"type":61,"tag":62,"props":826,"children":827},{},[828,830,835],{"type":66,"value":829},"5.7% 到 1.9% 是减少 ",{"type":61,"tag":69,"props":831,"children":832},{},[833],{"type":66,"value":834},"3.8 个百分点",{"type":66,"value":836},"，或相对减少约 67%；不是答案正确率提高了 67%。这些是报告语料、模型和流程下的结果，不能直接推广到其他语言和领域的资料库。",{"type":61,"tag":62,"props":838,"children":839},{},[840],{"type":66,"value":841},"补注要单独标记为派生内容，保留原文；它能帮助检索，但不能伪装成权威证据。用同一评测集比较结构标题补全、父子块和 LLM 补注，再决定额外成本是否值得。",{"type":61,"tag":62,"props":843,"children":844},{},[845,847],{"type":66,"value":846},"Late Chunking 是另一条路线：先在长上下文模型里编码，再按块池化向量，使块保留更多全文语境。它要求相应模型和实现，不是所有 Embedding API 都能支持。",{"type":61,"tag":82,"props":848,"children":850},{"href":37,"rel":849},[722],[851],{"type":66,"value":852},"Jina 原始介绍",{"type":61,"tag":98,"props":854,"children":856},{"id":855},"切分实验怎么做",[857],{"type":66,"value":855},{"type":61,"tag":62,"props":859,"children":860},{},[861],{"type":66,"value":862},"固定语料版本、Embedding 和检索参数，比较不同切分配置。用真实问题标注支持答案的证据位置，避免只标 chunk ID：重新切分后 chunk 变了，证据位置仍能对应。",{"type":61,"tag":62,"props":864,"children":865},{},[866],{"type":66,"value":867},"至少看证据召回、最终正确率、引用支撑率、输入 token、索引体积和延迟。对跨段推理、表格问题、图题单独分组。小样本抽检能发现坏块，却不能代替全链路评测。",{"type":61,"tag":62,"props":869,"children":870},{},[871],{"type":66,"value":872},"若题目需要三条证据，Recall@k 可能不错但仍缺关键一条；再看“全部必要证据是否齐全”。若重叠导致多个结果几乎相同，先去重和限制父文档占比，不要立刻增加 top-k。",{"type":61,"tag":98,"props":874,"children":876},{"id":875},"落地检查",[877],{"type":66,"value":875},{"type":61,"tag":879,"props":880,"children":881},"ol",{},[882,888,893,898,903,908],{"type":61,"tag":883,"props":884,"children":885},"li",{},[886],{"type":66,"value":887},"检查解析顺序、表格、公式与 OCR，保留原始载体。",{"type":61,"tag":883,"props":889,"children":890},{},[891],{"type":66,"value":892},"每块记录父文档、内容类型、版本、来源与权限。",{"type":61,"tag":883,"props":894,"children":895},{},[896],{"type":66,"value":897},"结构化语料优先比较结构切分，超长单元用父子关系处理。",{"type":61,"tag":883,"props":899,"children":900},{},[901],{"type":66,"value":902},"补注标记为派生内容，抽检生成错误，不作为独立证据。",{"type":61,"tag":883,"props":904,"children":905},{},[906],{"type":66,"value":907},"固定其他变量比较切分，用最终答案质量和成本一起选参数。",{"type":61,"tag":883,"props":909,"children":910},{},[911],{"type":66,"value":912},"更新、删除与撤权同步覆盖各索引和缓存。",{"type":61,"tag":98,"props":914,"children":916},{"id":915},"参考资料",[917],{"type":66,"value":915},[919,920,921,922,923,924,925],{"id":100,"depth":311,"text":103},{"id":116,"depth":311,"text":119},{"id":269,"depth":311,"text":269},{"id":710,"depth":311,"text":713},{"id":855,"depth":311,"text":855},{"id":875,"depth":311,"text":875},{"id":915,"depth":311,"text":915},1792400815607]