JianguoDing
EN
← Back to the RAG series

RAG 01 · The Paper Room · Reproduction notes

The Paper Room: reproduction notes for all ten levels

Use the same material and actions as the game, then check why the evidence is sufficient—or why it is not.

These notes follow the ten levels of the basic game, The Paper Room (in Chinese). Open level 1 →

What you can do now

Open “View steps and results” under any level: the question, the candidate order, every action, each evidence verdict (unsuitable / more needed / sufficient) and the final reply. Nothing to install.

If you have your own RAG project

Reproduce the method: take a small set of questions and documents, fix the candidate order, record the evidence handed over at each step, and judge it on the same three levels.

The state of this site’s source

The site’s source, including the game rules and the script that produces these records, is not public, and there is no download. The commands below only run in the author’s repository.

These records are produced by executing the game rules at build time. Documents, word weights, candidate lists and replies are hand-authored teaching material. They are neither live model answers nor vector-search records from the old shop assistant.

Action labels and source documents stay in Chinese to match the game. Trilingual concept names and explanations are below.

01 Find the page

Open this level →
View steps and results

Order 1 · Customer question

几点开门？

  1. 查看「营业时间」，拿起并交给顾客

    营业时间 — 上午九点开门。

    Sufficient evidence

Evidence finally delivered

  • 营业时间：上午九点开门。

Teaching reply / customer feedback (prewritten)

九点开门，好的，明早见！

The command that produces this level’s record (runs only in the author’s repository):

npm run experiment:rag -- --level 1

02 Choose among several pages

Open this level →
View steps and results

Order 1 · Customer question

周几休息？

  1. 先试「饮品价格」，观察为什么不够

    饮品价格 — 珍珠奶茶 15 元，杨枝甘露 18 元，加料 2 元。

    Unsuitable evidence · 这页是价格，我想知道哪天休息。

  2. 改选「营业安排」，交给顾客

    营业安排 — 周一至周六营业，周日休息。

    Sufficient evidence

Evidence finally delivered

  • 营业安排：周一至周六营业，周日休息。

Teaching reply / customer feedback (prewritten)

周日休息，记住了。

The command that produces this level’s record (runs only in the author’s repository):

npm run experiment:rag -- --level 2

03 Recognize a paraphrased question

Open this level →
View steps and results

Order 1 · Customer question

周日能来买奶茶吗？

  1. 先试含有“奶茶”字样的价格页

    饮品价格 — 珍珠奶茶 15 元，杨枝甘露 18 元，加料 2 元。

    Unsuitable evidence · 这页是奶茶价格。我不是问多少钱，是问周日能不能来。

  2. 改选能说明周日是否营业的资料

    营业安排 — 周一至周六营业，周日休息。

    Sufficient evidence

Evidence finally delivered

  • 营业安排：周一至周六营业，周日休息。

Teaching reply / customer feedback (prewritten)

周日不开门啊，知道了，我周六来。

The command that produces this level’s record (runs only in the author’s repository):

npm run experiment:rag -- --level 3

04 Ask the LLM to reply

Open this level →
View steps and results

Order 1 · Customer question

我周六九点半来，开门了吗？

  1. 取收件窗口的问题，交给 LLM 写信台

  2. 查看「营业时间」，拿起并交给写信台；取回信后交给顾客

    营业时间 — 平日九点开门；周末十点开门。

    Sufficient evidence

Evidence finally delivered

  • 营业时间：平日九点开门；周末十点开门。

Teaching reply / customer feedback (prewritten)

还没开门。周六属于周末，10点开门。

The command that produces this level’s record (runs only in the author’s repository):

npm run experiment:rag -- --level 4

05 Reorder candidates

Open this level →
View steps and results

Order 1 · Customer question

珍珠奶茶多少钱？

  1. 先取默认第一张「新品宣传」，交给写信台

    Current candidate order：新品宣传 → 活动海报 → 制作说明 → 饮品价格表

    新品宣传 — 新品芋泥波波上市！老顾客买珍珠奶茶，第二杯半价。

    Unsuitable evidence · 这些资料没有提供价格：这页讲新品上市和第二杯半价，没写珍珠奶茶本身多少钱。

  2. 回到检索台，把「饮品价格表」上移一位

    Current candidate order：新品宣传 → 活动海报 → 饮品价格表 → 制作说明

  3. 回到检索台，把「饮品价格表」上移一位

    Current candidate order：新品宣传 → 饮品价格表 → 活动海报 → 制作说明

  4. 回到检索台，把「饮品价格表」上移一位

    Current candidate order：饮品价格表 → 新品宣传 → 活动海报 → 制作说明

  5. 取新的第一张，交给写信台

    Current candidate order：饮品价格表 → 新品宣传 → 活动海报 → 制作说明

    饮品价格表 — 珍珠奶茶 18 元 · 杨枝甘露 20 元 · 加料 2 元

    Sufficient evidence

Evidence finally delivered

  • 饮品价格表：珍珠奶茶 18 元 · 杨枝甘露 20 元 · 加料 2 元

Teaching reply / customer feedback (prewritten)

珍珠奶茶 18 元。

没有被找回的资料，不能靠重新排序凭空出现。

The command that produces this level’s record (runs only in the author’s repository):

npm run experiment:rag -- --level 5

06 Rewrite the retrieval query

Open this level →
View steps and results

Order 1 · Customer question

那个QQ的黑色小丸子，晚上几点就没啦？

  1. 执行检索，比较候选顺序

    Current candidate order：冬季限定 → 小料介绍 → 营业时间

  2. 先取原查询的第一条，检查证据

    Current candidate order：冬季限定 → 小料介绍 → 营业时间

    冬季限定 — 黑芝麻汤圆：黑色的小丸子，冬季晚上供应。

    Unsuitable evidence · 这页是黑芝麻汤圆。客人指着的是珍珠奶茶里的珍珠。

  3. 回到检索台，拿掉查询词「QQ」

  4. 回到检索台，拿掉查询词「黑色小丸子」

  5. 回到检索台，拿掉查询词「晚上」

  6. 回到检索台，拿掉查询词「几点」

  7. 加入查询词「珍珠」

  8. 加入查询词「停止供应」

  9. 加入查询词「时间」

  10. 执行检索，比较候选顺序

    Current candidate order：珍珠 · 供应时间 → 营业时间 → 小料介绍

  11. 取改写后的第一条，交给写信台

    Current candidate order：珍珠 · 供应时间 → 营业时间 → 小料介绍

    珍珠 · 供应时间 — 珍珠每日19:30停止供应。

    Sufficient evidence

Evidence finally delivered

  • 珍珠 · 供应时间：珍珠每日19:30停止供应。

Teaching reply / customer feedback (prewritten)

您说的是珍珠吧？珍珠每天19:30停止供应，七点半以后就没有了。

The command that produces this level’s record (runs only in the author’s repository):

npm run experiment:rag -- --level 6

07 Restore a fragment’s context

Open this level →
View steps and results

Order 1 · Customer question

新宿店周六几点开门？

  1. 先取没有店名的片段，交给写信台

    Current candidate order：片段 · 来源：门店营业时间 第 2 段 → 门店地址 → 节假日安排

    片段 · 来源：门店营业时间 第 2 段 — ……周末10点开门。

    Unsuitable evidence · 这段没有说明是哪家店。不能直接把 10 点当成新宿店的开门时间。

  2. 回到检索台，点击「查看来源（展开）」

  3. 取「新宿店」那一行，交给写信台

    门店营业时间 · 新宿店 — 新宿店：周末9点开门。

    Sufficient evidence

Evidence finally delivered

  • 门店营业时间 · 新宿店：新宿店：周末9点开门。

Teaching reply / customer feedback (prewritten)

新宿店周六9点开门（周六属于周末）。

切分（Chunking）方式也会影响这个问题：切的时候就把店名和时间留在一起，片段本身就够用。重新切分和展开上下文是两种不同的做法。

The command that produces this level’s record (runs only in the author’s repository):

npm run experiment:rag -- --level 7

08 Compare two retrieval routes

Open this level →
View steps and results

Order 1 · Customer question

SKU-MT-042 现在多少钱？

  1. 先选「按意思找」

  2. 执行检索，比较候选顺序

    Current candidate order：价格表 · SKU-MT-024 → 新品预告 → 价格表 · SKU-MT-042

  3. 试取 SKU-MT-024，核对编号

    Current candidate order：价格表 · SKU-MT-024 → 新品预告 → 价格表 · SKU-MT-042

    价格表 · SKU-MT-024 — SKU-MT-024 茉香奶绿（大杯）：19 元。

    Unsuitable evidence · 这页是 SKU-MT-024，客人问的是 042。编号差一位，不是同一件商品。

  4. 回到检索台，也启用「按字找」

  5. 执行检索，比较候选顺序

    Current candidate order：价格表 · SKU-MT-042 → 价格表 · SKU-MT-024 → 编号说明 → 新品预告

  6. 取 SKU-MT-042，交给写信台

    Current candidate order：价格表 · SKU-MT-042 → 价格表 · SKU-MT-024 → 编号说明 → 新品预告

    价格表 · SKU-MT-042 — SKU-MT-042 茉莉奶绿：16 元。

    Sufficient evidence

Evidence finally delivered

  • 价格表 · SKU-MT-042：SKU-MT-042 茉莉奶绿：16 元。

Teaching reply / customer feedback (prewritten)

SKU-MT-042 是茉莉奶绿，16 元。

Order 2 · Customer question

有没有口感清爽、带点酸味的饮品？

  1. 第二单先选「按字找」

  2. 执行检索，比较候选顺序

    Current candidate order：焦糖奶茶 → 加料推荐 → 柠檬绿茶

  3. 试取「焦糖奶茶」，核对口味描述

    Current candidate order：焦糖奶茶 → 加料推荐 → 柠檬绿茶

    焦糖奶茶 — 口感醇厚，香甜浓郁。

    Unsuitable evidence · 这页只是也用了“口感”两个字。焦糖奶茶香甜浓郁，不清爽也不酸。

  4. 回到检索台，也启用「按意思找」

  5. 执行检索，比较候选顺序

    Current candidate order：焦糖奶茶 → 柠檬绿茶 → 加料推荐 → 百香果多多

  6. 取「柠檬绿茶」，交给写信台

    Current candidate order：焦糖奶茶 → 柠檬绿茶 → 加料推荐 → 百香果多多

    柠檬绿茶 — 鲜柠檬配茉莉绿茶，入口爽口，微酸回甘。

    Sufficient evidence

Evidence finally delivered

  • 柠檬绿茶：鲜柠檬配茉莉绿茶，入口爽口，微酸回甘。

Teaching reply / customer feedback (prewritten)

推荐柠檬绿茶：资料写它入口爽口，微酸回甘。

意思接近不等于事实成立。不管哪一路找到的，都要资料本身明确支持答案。

The command that produces this level’s record (runs only in the author’s repository):

npm run experiment:rag -- --level 8

09 Find a gap and search again

Open this level →
View steps and results

Order 1 · Customer question

今晚8点还能买到珍珠奶茶吗？

  1. 按顾客问题检索

    Current candidate order：营业时间

  2. 取营业时间，交给写信台，阅读信息缺口

    Current candidate order：营业时间

    营业时间 — 门店营业到21:00。

    Accepted; more evidence needed · 知道门店开着，但还不知道珍珠供应到几点。

  3. 回到检索台，先试补查价格

    Current candidate order：饮品价格

  4. 把价格交给写信台，观察缺口仍然存在

    Current candidate order：饮品价格

    饮品价格 — 珍珠奶茶 18 元。

    Unsuitable evidence · 这是价格，填不上缺口：还不知道珍珠供应到几点。

  5. 改查「珍珠 · 供应时间」

    Current candidate order：珍珠 · 供应时间

  6. 补齐珍珠供应时间，取回信交给顾客

    Current candidate order：珍珠 · 供应时间

    珍珠 · 供应时间 — 珍珠19:30停止供应。

    Sufficient evidence

Evidence finally delivered

  • 营业时间：门店营业到21:00。
  • 珍珠 · 供应时间：珍珠19:30停止供应。

Teaching reply / customer feedback (prewritten)

门店开到21:00，但珍珠19:30就停止供应了，今晚8点买不到珍珠奶茶，可以换别的饮品。

这展示了“判断不足—继续检索”的过程，是理解 Agentic RAG 的一个入口；Agentic RAG 不只是重复检索。

The command that produces this level’s record (runs only in the author’s repository):

npm run experiment:rag -- --level 9

10 Follow relations to evidence

Open this level →
View steps and results

Order 1 · Customer question

A供应商今晚无法按时送货，明天哪些饮品可能受影响？

  1. 展开 A 供应商

  2. 展开木薯粉

  3. 展开珍珠

  4. 把珍珠奶茶路径加入证据

  5. 先只带一条路径给写信台

    关系证据 — A 供应商 → 木薯粉 → 珍珠 → 珍珠奶茶（进货单、配方、菜单）

    Unsuitable evidence · 我只拿到一条受影响的路径。A 供应商还连着别的原料吗？再去关系板看看。

  6. 回到关系板，展开 A 供应商的芋头

  7. 展开芋泥

  8. 把芋泥奶茶路径加入证据

  9. 带两条路径给写信台；留意回信中的“可能”和库存限制

    关系证据 — A 供应商 → 木薯粉 → 珍珠 → 珍珠奶茶（进货单、配方、菜单）；A 供应商 → 芋头 → 芋泥 → 芋泥奶茶（进货单、配方、菜单）

    Sufficient evidence

Evidence finally delivered

  • 关系证据：A 供应商 → 木薯粉 → 珍珠 → 珍珠奶茶（进货单、配方、菜单）；A 供应商 → 芋头 → 芋泥 → 芋泥奶茶（进货单、配方、菜单）

Teaching reply / customer feedback (prewritten)

可能受影响：珍珠奶茶（木薯粉 → 珍珠）和芋泥奶茶（芋头 → 芋泥），都依赖 A 供应商。资料里没有库存信息，是否停售要先查库存。

物流延误只能说明“可能受影响”；没有库存信息，就不能断言一定停售。完整的 GraphRAG 专题还会探索建图和不同的查询方式，这一关只展示关系能提供关联证据。

The command that produces this level’s record (runs only in the author’s repository):

npm run experiment:rag -- --level 10

How these records are produced

Run from the root of the site’s repository with Node 22.12 or later; no model, API key or network connection is needed. The repository is not public yet, so these commands only run in the author’s own copy. They are listed to show where the records on this page come from.

Run all ten levels

npm run experiment:rag
Data and walkthrough locations
  • experiments/rag/room/README.md
  • src/scripts/rag-game/levels.ts
  • src/data/rag-reproduction.ts
Inspect the complete build-time record
[
  {
    "level": 1,
    "title": "找到那一页",
    "skill": "找到唯一资料",
    "learn": [
      "先找到和问题有关的那一页，再交出去。整个游戏都从这一步开始：检索（Retrieval）。"
    ],
    "source": "teaching-simulation",
    "orders": [
      {
        "question": "几点开门？",
        "steps": [
          {
            "action": "take",
            "arg": "hours",
            "label": "查看「营业时间」，拿起并交给顾客",
            "candidates": [],
            "item": {
              "id": "hours",
              "t": "营业时间",
              "b": "上午九点开门。",
              "kind": "doc"
            },
            "judgement": {
              "verdict": "accept",
              "complete": true
            }
          }
        ],
        "evidence": [
          {
            "id": "hours",
            "t": "营业时间",
            "b": "上午九点开门。",
            "kind": "doc"
          }
        ],
        "complete": true,
        "reply": "九点开门，好的，明早见！"
      }
    ]
  },
  {
    "level": 2,
    "title": "从几页里挑出对的",
    "skill": "从不同资料中选择",
    "learn": [
      "资料一多，就要读一读、比一比，挑出真正能回答问题的那一页。"
    ],
    "source": "teaching-simulation",
    "orders": [
      {
        "question": "周几休息？",
        "steps": [
          {
            "action": "take",
            "arg": "price",
            "label": "先试「饮品价格」，观察为什么不够",
            "candidates": [],
            "item": {
              "id": "price",
              "t": "饮品价格",
              "b": "珍珠奶茶 15 元，杨枝甘露 18 元，加料 2 元。",
              "kind": "doc"
            },
            "judgement": {
              "verdict": "reject",
              "say": "这页是价格，我想知道哪天休息。"
            }
          },
          {
            "action": "take",
            "arg": "schedule",
            "label": "改选「营业安排」，交给顾客",
            "candidates": [],
            "item": {
              "id": "schedule",
              "t": "营业安排",
              "b": "周一至周六营业，周日休息。",
              "kind": "doc"
            },
            "judgement": {
              "verdict": "accept",
              "complete": true
            }
          }
        ],
        "evidence": [
          {
            "id": "schedule",
            "t": "营业安排",
            "b": "周一至周六营业，周日休息。",
            "kind": "doc"
          }
        ],
        "complete": true,
        "reply": "周日休息，记住了。"
      }
    ]
  },
  {
    "level": 3,
    "title": "换了问法也能找到",
    "skill": "理解不同问法",
    "learn": [
      "价格页上也写着“奶茶”，但能回答“周日能不能来”的是营业安排。找资料要看意思，不只是看有没有同样的字。"
    ],
    "source": "teaching-simulation",
    "orders": [
      {
        "question": "周日能来买奶茶吗？",
        "steps": [
          {
            "action": "take",
            "arg": "price",
            "label": "先试含有“奶茶”字样的价格页",
            "candidates": [],
            "item": {
              "id": "price",
              "t": "饮品价格",
              "b": "珍珠奶茶 15 元，杨枝甘露 18 元，加料 2 元。",
              "kind": "doc"
            },
            "judgement": {
              "verdict": "reject",
              "say": "这页是奶茶价格。我不是问多少钱，是问周日能不能来。"
            }
          },
          {
            "action": "take",
            "arg": "schedule",
            "label": "改选能说明周日是否营业的资料",
            "candidates": [],
            "item": {
              "id": "schedule",
              "t": "营业安排",
              "b": "周一至周六营业，周日休息。",
              "kind": "doc"
            },
            "judgement": {
              "verdict": "accept",
              "complete": true
            }
          }
        ],
        "evidence": [
          {
            "id": "schedule",
            "t": "营业安排",
            "b": "周一至周六营业，周日休息。",
            "kind": "doc"
          }
        ],
        "complete": true,
        "reply": "周日不开门啊，知道了，我周六来。"
      }
    ]
  },
  {
    "level": 4,
    "title": "请 LLM 写回信",
    "skill": "把问题和资料交给 LLM，生成回答",
    "learn": [
      "之前：找到资料，交回资料。",
      "现在：把问题和资料交给 LLM，生成回答。",
      "RAG：检索增强生成。"
    ],
    "source": "teaching-simulation",
    "orders": [
      {
        "question": "我周六九点半来，开门了吗？",
        "steps": [
          {
            "action": "question",
            "arg": "",
            "label": "取收件窗口的问题，交给 LLM 写信台",
            "candidates": []
          },
          {
            "action": "take",
            "arg": "weekend",
            "label": "查看「营业时间」，拿起并交给写信台；取回信后交给顾客",
            "candidates": [],
            "item": {
              "id": "weekend",
              "t": "营业时间",
              "b": "平日九点开门；周末十点开门。",
              "kind": "doc"
            },
            "judgement": {
              "verdict": "accept",
              "complete": true
            }
          }
        ],
        "evidence": [
          {
            "id": "weekend",
            "t": "营业时间",
            "b": "平日九点开门；周末十点开门。",
            "kind": "doc"
          }
        ],
        "complete": true,
        "reply": "还没开门。周六属于周末，10点开门。"
      }
    ]
  },
  {
    "level": 5,
    "title": "排在前面的不一定最有用",
    "skill": "检索候选与排序",
    "learn": [
      "检索先找到候选，重排序（Rerank）帮助把更适合当前问题的资料放到前面。"
    ],
    "note": "没有被找回的资料，不能靠重新排序凭空出现。",
    "source": "teaching-simulation",
    "orders": [
      {
        "question": "珍珠奶茶多少钱？",
        "steps": [
          {
            "action": "take",
            "arg": "",
            "label": "先取默认第一张「新品宣传」，交给写信台",
            "candidates": [
              "新品宣传",
              "活动海报",
              "制作说明",
              "饮品价格表"
            ],
            "item": {
              "id": "promo",
              "t": "新品宣传",
              "b": "新品芋泥波波上市！老顾客买珍珠奶茶，第二杯半价。",
              "kind": "doc",
              "src": "station"
            },
            "judgement": {
              "verdict": "reject",
              "say": "这些资料没有提供价格：这页讲新品上市和第二杯半价，没写珍珠奶茶本身多少钱。"
            }
          },
          {
            "action": "up",
            "arg": "price",
            "label": "回到检索台，把「饮品价格表」上移一位",
            "candidates": [
              "新品宣传",
              "活动海报",
              "饮品价格表",
              "制作说明"
            ]
          },
          {
            "action": "up",
            "arg": "price",
            "label": "回到检索台，把「饮品价格表」上移一位",
            "candidates": [
              "新品宣传",
              "饮品价格表",
              "活动海报",
              "制作说明"
            ]
          },
          {
            "action": "up",
            "arg": "price",
            "label": "回到检索台，把「饮品价格表」上移一位",
            "candidates": [
              "饮品价格表",
              "新品宣传",
              "活动海报",
              "制作说明"
            ]
          },
          {
            "action": "take",
            "arg": "",
            "label": "取新的第一张，交给写信台",
            "candidates": [
              "饮品价格表",
              "新品宣传",
              "活动海报",
              "制作说明"
            ],
            "item": {
              "id": "price5",
              "t": "饮品价格表",
              "b": "珍珠奶茶 18 元 · 杨枝甘露 20 元 · 加料 2 元",
              "kind": "doc",
              "s": "价格表",
              "src": "station"
            },
            "judgement": {
              "verdict": "accept",
              "complete": true
            }
          }
        ],
        "evidence": [
          {
            "id": "price5",
            "t": "饮品价格表",
            "b": "珍珠奶茶 18 元 · 杨枝甘露 20 元 · 加料 2 元",
            "kind": "doc",
            "s": "价格表",
            "src": "station"
          }
        ],
        "complete": true,
        "reply": "珍珠奶茶 18 元。"
      }
    ]
  },
  {
    "level": 6,
    "title": "换一种说法",
    "skill": "改善查询表达",
    "learn": [
      "顾客的说法和资料的写法常常不一样。换一种查询表达（Query Rewrite），检索就能找到资料；回答时仍然照顾客原来的问题来答。",
      "同时用几种说法去查、再把结果合在一起，是 MultiQuery 的思路。"
    ],
    "source": "teaching-simulation",
    "orders": [
      {
        "question": "那个QQ的黑色小丸子，晚上几点就没啦？",
        "steps": [
          {
            "action": "search",
            "arg": "",
            "label": "执行检索，比较候选顺序",
            "candidates": [
              "冬季限定",
              "小料介绍",
              "营业时间"
            ]
          },
          {
            "action": "take",
            "arg": "",
            "label": "先取原查询的第一条，检查证据",
            "candidates": [
              "冬季限定",
              "小料介绍",
              "营业时间"
            ],
            "item": {
              "id": "tangyuan",
              "t": "冬季限定",
              "b": "黑芝麻汤圆：黑色的小丸子，冬季晚上供应。",
              "kind": "doc",
              "src": "station"
            },
            "judgement": {
              "verdict": "reject",
              "say": "这页是黑芝麻汤圆。客人指着的是珍珠奶茶里的珍珠。"
            }
          },
          {
            "action": "rm",
            "arg": "QQ",
            "label": "回到检索台，拿掉查询词「QQ」",
            "candidates": []
          },
          {
            "action": "rm",
            "arg": "黑色小丸子",
            "label": "回到检索台，拿掉查询词「黑色小丸子」",
            "candidates": []
          },
          {
            "action": "rm",
            "arg": "晚上",
            "label": "回到检索台，拿掉查询词「晚上」",
            "candidates": []
          },
          {
            "action": "rm",
            "arg": "几点",
            "label": "回到检索台，拿掉查询词「几点」",
            "candidates": []
          },
          {
            "action": "add",
            "arg": "珍珠",
            "label": "加入查询词「珍珠」",
            "candidates": []
          },
          {
            "action": "add",
            "arg": "停止供应",
            "label": "加入查询词「停止供应」",
            "candidates": []
          },
          {
            "action": "add",
            "arg": "时间",
            "label": "加入查询词「时间」",
            "candidates": []
          },
          {
            "action": "search",
            "arg": "",
            "label": "执行检索，比较候选顺序",
            "candidates": [
              "珍珠 · 供应时间",
              "营业时间",
              "小料介绍"
            ]
          },
          {
            "action": "take",
            "arg": "",
            "label": "取改写后的第一条，交给写信台",
            "candidates": [
              "珍珠 · 供应时间",
              "营业时间",
              "小料介绍"
            ],
            "item": {
              "id": "pearl",
              "t": "珍珠 · 供应时间",
              "b": "珍珠每日19:30停止供应。",
              "kind": "doc",
              "s": "珍珠供应",
              "src": "station"
            },
            "judgement": {
              "verdict": "accept",
              "complete": true
            }
          }
        ],
        "evidence": [
          {
            "id": "pearl",
            "t": "珍珠 · 供应时间",
            "b": "珍珠每日19:30停止供应。",
            "kind": "doc",
            "s": "珍珠供应",
            "src": "station"
          }
        ],
        "complete": true,
        "reply": "您说的是珍珠吧？珍珠每天19:30停止供应，七点半以后就没有了。"
      }
    ]
  },
  {
    "level": 7,
    "title": "这句话说的是哪家店",
    "skill": "找回片段的上下文",
    "learn": [
      "片段一旦离开了它的主体和条件（哪家店），就可能不足以支持回答。展开上下文（Small-to-Big），把主体找回来。"
    ],
    "note": "切分（Chunking）方式也会影响这个问题：切的时候就把店名和时间留在一起，片段本身就够用。重新切分和展开上下文是两种不同的做法。",
    "source": "teaching-simulation",
    "orders": [
      {
        "question": "新宿店周六几点开门？",
        "steps": [
          {
            "action": "take",
            "arg": "frag",
            "label": "先取没有店名的片段，交给写信台",
            "candidates": [
              "片段 · 来源：门店营业时间 第 2 段",
              "门店地址",
              "节假日安排"
            ],
            "item": {
              "id": "frag",
              "t": "片段 · 来源：门店营业时间 第 2 段",
              "b": "……周末10点开门。",
              "kind": "doc",
              "s": "片段",
              "src": "station"
            },
            "judgement": {
              "verdict": "reject",
              "say": "这段没有说明是哪家店。不能直接把 10 点当成新宿店的开门时间。"
            }
          },
          {
            "action": "expand",
            "arg": "",
            "label": "回到检索台，点击「查看来源（展开）」",
            "candidates": []
          },
          {
            "action": "take",
            "arg": "shinjuku",
            "label": "取「新宿店」那一行，交给写信台",
            "candidates": [],
            "item": {
              "id": "shinjuku",
              "t": "门店营业时间 · 新宿店",
              "b": "新宿店：周末9点开门。",
              "kind": "doc",
              "s": "新宿店",
              "src": "station"
            },
            "judgement": {
              "verdict": "accept",
              "complete": true
            }
          }
        ],
        "evidence": [
          {
            "id": "shinjuku",
            "t": "门店营业时间 · 新宿店",
            "b": "新宿店：周末9点开门。",
            "kind": "doc",
            "s": "新宿店",
            "src": "station"
          }
        ],
        "complete": true,
        "reply": "新宿店周六9点开门（周六属于周末）。"
      }
    ]
  },
  {
    "level": 8,
    "title": "两种查找方式",
    "skill": "关键词与语义检索",
    "learn": [
      "按字找（关键词检索）擅长编号、专有名词这类要一字不差的线索；按意思找（语义检索）能找到换了说法的内容。两路合并去重，就是混合检索（Hybrid Search）的思路。"
    ],
    "note": "意思接近不等于事实成立。不管哪一路找到的，都要资料本身明确支持答案。",
    "source": "teaching-simulation",
    "orders": [
      {
        "question": "SKU-MT-042 现在多少钱？",
        "steps": [
          {
            "action": "tool",
            "arg": "sem",
            "label": "先选「按意思找」",
            "candidates": []
          },
          {
            "action": "search",
            "arg": "",
            "label": "执行检索，比较候选顺序",
            "candidates": [
              "价格表 · SKU-MT-024",
              "新品预告",
              "价格表 · SKU-MT-042"
            ]
          },
          {
            "action": "take",
            "arg": "sku024",
            "label": "试取 SKU-MT-024，核对编号",
            "candidates": [
              "价格表 · SKU-MT-024",
              "新品预告",
              "价格表 · SKU-MT-042"
            ],
            "item": {
              "id": "sku024",
              "t": "价格表 · SKU-MT-024",
              "b": "SKU-MT-024 茉香奶绿（大杯）：19 元。",
              "kind": "doc",
              "s": "SKU-024",
              "src": "station"
            },
            "judgement": {
              "verdict": "reject",
              "say": "这页是 SKU-MT-024，客人问的是 042。编号差一位，不是同一件商品。"
            }
          },
          {
            "action": "tool",
            "arg": "kw",
            "label": "回到检索台，也启用「按字找」",
            "candidates": []
          },
          {
            "action": "search",
            "arg": "",
            "label": "执行检索，比较候选顺序",
            "candidates": [
              "价格表 · SKU-MT-042",
              "价格表 · SKU-MT-024",
              "编号说明",
              "新品预告"
            ]
          },
          {
            "action": "take",
            "arg": "sku042",
            "label": "取 SKU-MT-042，交给写信台",
            "candidates": [
              "价格表 · SKU-MT-042",
              "价格表 · SKU-MT-024",
              "编号说明",
              "新品预告"
            ],
            "item": {
              "id": "sku042",
              "t": "价格表 · SKU-MT-042",
              "b": "SKU-MT-042 茉莉奶绿：16 元。",
              "kind": "doc",
              "s": "SKU-042",
              "src": "station"
            },
            "judgement": {
              "verdict": "accept",
              "complete": true
            }
          }
        ],
        "evidence": [
          {
            "id": "sku042",
            "t": "价格表 · SKU-MT-042",
            "b": "SKU-MT-042 茉莉奶绿：16 元。",
            "kind": "doc",
            "s": "SKU-042",
            "src": "station"
          }
        ],
        "complete": true,
        "reply": "SKU-MT-042 是茉莉奶绿，16 元。"
      },
      {
        "question": "有没有口感清爽、带点酸味的饮品？",
        "steps": [
          {
            "action": "tool",
            "arg": "kw",
            "label": "第二单先选「按字找」",
            "candidates": []
          },
          {
            "action": "search",
            "arg": "",
            "label": "执行检索，比较候选顺序",
            "candidates": [
              "焦糖奶茶",
              "加料推荐",
              "柠檬绿茶"
            ]
          },
          {
            "action": "take",
            "arg": "caramel",
            "label": "试取「焦糖奶茶」，核对口味描述",
            "candidates": [
              "焦糖奶茶",
              "加料推荐",
              "柠檬绿茶"
            ],
            "item": {
              "id": "caramel",
              "t": "焦糖奶茶",
              "b": "口感醇厚，香甜浓郁。",
              "kind": "doc",
              "src": "station"
            },
            "judgement": {
              "verdict": "reject",
              "say": "这页只是也用了“口感”两个字。焦糖奶茶香甜浓郁，不清爽也不酸。"
            }
          },
          {
            "action": "tool",
            "arg": "sem",
            "label": "回到检索台，也启用「按意思找」",
            "candidates": []
          },
          {
            "action": "search",
            "arg": "",
            "label": "执行检索，比较候选顺序",
            "candidates": [
              "焦糖奶茶",
              "柠檬绿茶",
              "加料推荐",
              "百香果多多"
            ]
          },
          {
            "action": "take",
            "arg": "lemon",
            "label": "取「柠檬绿茶」，交给写信台",
            "candidates": [
              "焦糖奶茶",
              "柠檬绿茶",
              "加料推荐",
              "百香果多多"
            ],
            "item": {
              "id": "lemon",
              "t": "柠檬绿茶",
              "b": "鲜柠檬配茉莉绿茶，入口爽口，微酸回甘。",
              "kind": "doc",
              "src": "station"
            },
            "judgement": {
              "verdict": "accept",
              "complete": true
            }
          }
        ],
        "evidence": [
          {
            "id": "lemon",
            "t": "柠檬绿茶",
            "b": "鲜柠檬配茉莉绿茶，入口爽口，微酸回甘。",
            "kind": "doc",
            "src": "station"
          }
        ],
        "complete": true,
        "reply": "推荐柠檬绿茶：资料写它入口爽口，微酸回甘。"
      }
    ]
  },
  {
    "level": 9,
    "title": "还缺一条线索",
    "skill": "发现信息缺口，再次检索",
    "learn": [
      "先检查手上的资料够不够回答，发现缺口，再朝缺口的方向发起新的检索。"
    ],
    "note": "这展示了“判断不足—继续检索”的过程，是理解 Agentic RAG 的一个入口；Agentic RAG 不只是重复检索。",
    "source": "teaching-simulation",
    "orders": [
      {
        "question": "今晚8点还能买到珍珠奶茶吗？",
        "steps": [
          {
            "action": "first",
            "arg": "",
            "label": "按顾客问题检索",
            "candidates": [
              "营业时间"
            ]
          },
          {
            "action": "take",
            "arg": "",
            "label": "取营业时间，交给写信台，阅读信息缺口",
            "candidates": [
              "营业时间"
            ],
            "item": {
              "id": "open21",
              "t": "营业时间",
              "b": "门店营业到21:00。",
              "kind": "doc",
              "src": "station"
            },
            "judgement": {
              "verdict": "accept",
              "say": "知道门店开着，但还不知道珍珠供应到几点。"
            }
          },
          {
            "action": "dir",
            "arg": "cost",
            "label": "回到检索台，先试补查价格",
            "candidates": [
              "饮品价格"
            ]
          },
          {
            "action": "take",
            "arg": "",
            "label": "把价格交给写信台，观察缺口仍然存在",
            "candidates": [
              "饮品价格"
            ],
            "item": {
              "id": "cost",
              "t": "饮品价格",
              "b": "珍珠奶茶 18 元。",
              "kind": "doc",
              "src": "station"
            },
            "judgement": {
              "verdict": "reject",
              "say": "这是价格，填不上缺口：还不知道珍珠供应到几点。"
            }
          },
          {
            "action": "dir",
            "arg": "pearl",
            "label": "改查「珍珠 · 供应时间」",
            "candidates": [
              "珍珠 · 供应时间"
            ]
          },
          {
            "action": "take",
            "arg": "",
            "label": "补齐珍珠供应时间，取回信交给顾客",
            "candidates": [
              "珍珠 · 供应时间"
            ],
            "item": {
              "id": "pearl1930",
              "t": "珍珠 · 供应时间",
              "b": "珍珠19:30停止供应。",
              "kind": "doc",
              "s": "珍珠供应",
              "src": "station"
            },
            "judgement": {
              "verdict": "accept",
              "complete": true
            }
          }
        ],
        "evidence": [
          {
            "id": "open21",
            "t": "营业时间",
            "b": "门店营业到21:00。",
            "kind": "doc",
            "src": "station"
          },
          {
            "id": "pearl1930",
            "t": "珍珠 · 供应时间",
            "b": "珍珠19:30停止供应。",
            "kind": "doc",
            "s": "珍珠供应",
            "src": "station"
          }
        ],
        "complete": true,
        "reply": "门店开到21:00，但珍珠19:30就停止供应了，今晚8点买不到珍珠奶茶，可以换别的饮品。"
      }
    ]
  },
  {
    "level": 10,
    "title": "关系里的答案",
    "skill": "利用实体关系回答问题",
    "learn": [
      "资料之间的关系（供应商 → 原料 → 半成品 → 饮品）能把分散的证据连起来，每一步都能追溯到来源。"
    ],
    "note": "物流延误只能说明“可能受影响”；没有库存信息，就不能断言一定停售。完整的 GraphRAG 专题还会探索建图和不同的查询方式，这一关只展示关系能提供关联证据。",
    "source": "teaching-simulation",
    "orders": [
      {
        "question": "A供应商今晚无法按时送货，明天哪些饮品可能受影响？",
        "steps": [
          {
            "action": "node",
            "arg": "A",
            "label": "展开 A 供应商",
            "candidates": []
          },
          {
            "action": "node",
            "arg": "tapioca",
            "label": "展开木薯粉",
            "candidates": []
          },
          {
            "action": "node",
            "arg": "pearl",
            "label": "展开珍珠",
            "candidates": []
          },
          {
            "action": "add",
            "arg": "pearlTea",
            "label": "把珍珠奶茶路径加入证据",
            "candidates": []
          },
          {
            "action": "take",
            "arg": "",
            "label": "先只带一条路径给写信台",
            "candidates": [],
            "item": {
              "id": "graph:pearlTea",
              "t": "关系证据",
              "b": "A 供应商 → 木薯粉 → 珍珠 → 珍珠奶茶（进货单、配方、菜单）",
              "kind": "doc",
              "s": "关系证据",
              "src": "station",
              "data": [
                "pearlTea"
              ]
            },
            "judgement": {
              "verdict": "reject",
              "say": "我只拿到一条受影响的路径。A 供应商还连着别的原料吗？再去关系板看看。"
            }
          },
          {
            "action": "node",
            "arg": "taro",
            "label": "回到关系板，展开 A 供应商的芋头",
            "candidates": []
          },
          {
            "action": "node",
            "arg": "taropaste",
            "label": "展开芋泥",
            "candidates": []
          },
          {
            "action": "add",
            "arg": "taroTea",
            "label": "把芋泥奶茶路径加入证据",
            "candidates": []
          },
          {
            "action": "take",
            "arg": "",
            "label": "带两条路径给写信台；留意回信中的“可能”和库存限制",
            "candidates": [],
            "item": {
              "id": "graph:pearlTea+taroTea",
              "t": "关系证据",
              "b": "A 供应商 → 木薯粉 → 珍珠 → 珍珠奶茶（进货单、配方、菜单）；A 供应商 → 芋头 → 芋泥 → 芋泥奶茶（进货单、配方、菜单）",
              "kind": "doc",
              "s": "关系证据",
              "src": "station",
              "data": [
                "pearlTea",
                "taroTea"
              ]
            },
            "judgement": {
              "verdict": "accept",
              "complete": true
            }
          }
        ],
        "evidence": [
          {
            "id": "graph:pearlTea+taroTea",
            "t": "关系证据",
            "b": "A 供应商 → 木薯粉 → 珍珠 → 珍珠奶茶（进货单、配方、菜单）；A 供应商 → 芋头 → 芋泥 → 芋泥奶茶（进货单、配方、菜单）",
            "kind": "doc",
            "s": "关系证据",
            "src": "station",
            "data": [
              "pearlTea",
              "taroTea"
            ]
          }
        ],
        "complete": true,
        "reply": "可能受影响：珍珠奶茶（木薯粉 → 珍珠）和芋泥奶茶（芋头 → 芋泥），都依赖 A 供应商。资料里没有库存信息，是否停售要先查库存。"
      }
    ]
  }
]

Next: real model experiments

To study actual models, run retrieval and generation separately on each level’s material. Record model versions, candidate scores, full context and raw answers. Prewritten replies here are not model outputs; the experiment README describes the variables to control for further experiments.

The old assistant game has been replaced here. Its historical vector-search experiment remains in experiments/rag/milk-tea-ai-01/ and does not represent these ten levels.

Concepts on this page

Open a concept for Chinese, Japanese and English terms and an explanation.

RetrievalView details
中文
检索
日本語
検索
English
Retrieval

Finding candidate material relevant to a question. Evidence sufficiency still needs to be checked.

Semantic searchView details
中文
语义检索
日本語
意味検索
English
Semantic search

Finding material by semantic relevance. Similarity does not imply factual equivalence. Level 8 uses hand-authored result lists.

Large language model LLMView details
中文
大语言模型
日本語
大規模言語モデル
English
Large language model (LLM)

A model that generates language from input. Replies in this game are prewritten teaching content, not live model calls.

Retrieval-augmented generation RAGView details
中文
检索增强生成
日本語
検索拡張生成
English
Retrieval-augmented generation (RAG)

Retrieve relevant material, then provide it with the question to a model for generation. Retrieval does not guarantee a correct answer.

RerankingView details
中文
重排序
日本語
リランキング
English
Reranking

Reordering retrieved candidates; it cannot add material that was never retrieved. Level 5 uses manual ordering.

Query rewritingView details
中文
查询改写
日本語
クエリ書き換え
English
Query rewriting

Changing the retrieval query while preserving the intent of the original question, which the answer should still address.

Multi-query retrievalView details
中文
多查询检索
日本語
マルチクエリ検索
English
Multi-query retrieval

Retrieving with multiple query formulations and merging distinct candidates. Level 6 demonstrates rewriting, not automatic multi-query execution.

ChunkingView details
中文
文本切分
日本語
チャンク分割
English
Chunking

Splitting documents into retrieval units. Separating subjects or conditions can leave a fragment insufficient for an answer.

Small-to-big retrievalView details
中文
上下文扩展
日本語
コンテキスト拡張
English
Small-to-big retrieval

Retrieving a small fragment and expanding to surrounding or parent content to recover subjects and conditions.

Keyword searchView details
中文
关键词检索
日本語
キーワード検索
English
Keyword search

Finding material through lexical matches, useful for exact identifiers and names.

Hybrid searchView details
中文
混合检索
日本語
ハイブリッド検索
English
Hybrid search

Combining retrieval routes such as keyword and semantic search. Level 8 interleaves lists and removes duplicates.

Agentic RAGView details
中文
智能体式检索增强生成
日本語
エージェント型RAG
English
Agentic RAG

A system choosing subsequent actions such as retrieval based on task state. Level 9 illustrates a fixed evidence-gap loop, not an autonomous agent.

GraphRAGView details
中文
图结构检索增强生成
日本語
グラフ型RAG
English
GraphRAG

Organizing and retrieving evidence using entities and relations. Level 10 demonstrates paths in a hand-built graph, not automated construction or a full GraphRAG system.

Back to the RAG series →