Errata.
spend $——.———— · the homebuyer’s history
Memory that keeps its corrections — every answer cited, every revision kept, refusal when the history never says.

results · Naive top-k RAG (k=10) · rerunC-nothink

Abstention 97.8

88 of 90 rows right — 30 questions × 3 seeds (11/22/33)scored deterministically by exact match — the judge never sees these rows. A row is right iff the arm abstained.same arm, by ability: OverallInformation extractionMulti-sessionTemporalKnowledge updatesame arm, by question_type: single-session-usersingle-session-assistantsingle-session-preferencemulti-sessiontemporal-reasoningknowledge-updatethe same cut, other arms: Errata 93.3Full-context baseline 80.0
  1. INCORRECT6456829e_absmulti-sessionasked 2023/05/30 (Tue) 14:244,789 ctx tok

    How many plants did I initially plant for tomatoes and chili peppers?

    naive
    5other seeds: INSUFFICIENT_INFORMATION: The history mentions planting 5 tomato plants, but does not mention chili peppers.5
    gold
    ABSTAIN — the history never says
    errata 5 tomato plantsfull-context 5
  2. ABSTAINED031748ae_absknowledge-updateasked 2023/06/06 (Tue) 08:193,857 ctx tok

    How many engineers do I lead when I just started my new role as Software Engineer Manager?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context 4
  3. ABSTAINED0862e8bf_abssingle-session-userasked 2023/05/30 (Tue) 21:394,520 ctx tok

    What is the name of my hamster?

    naive
    ABSTAINED — not in the historyother seeds: INSUFFICIENT_INFORMATION: The history mentions a cat named Luna, but does not mention a hamster.INSUFFICIENT_INFORMATION: The history mentions a cat named Luna.
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  4. ABSTAINED09ba9854_absmulti-sessionasked 2023/05/30 (Tue) 22:564,902 ctx tok

    How much will I save by taking the bus from the airport to my hotel instead of a taxi?

    naive
    ABSTAINED — not in the historyother seeds: INSUFFICIENT_INFORMATION: The history mentions taxi fares ranging from $60 to $270 USD and bus fares starting from around $10-$20 or ¥1,000, but does not provide specific hotel locations or exact combined costs to calculate a precise savings amount.INSUFFICIENT_INFORMATION: The history mentions taxi fares ranging from $60 to $270 USD and bus fares starting from around $10 USD, but it does not specify the user's hotel location or provide a direct comparison for a single trip to calculate exact savings.
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context Based on the material provided, taking a taxi from Narita Airport to Tokyo costs approxima
  5. ABSTAINED0ddfec37_absknowledge-updateasked 2023/06/07 (Wed) 08:314,318 ctx tok

    How many autographed football have I added to my collection in the first three months of collection?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  6. ABSTAINED15745da0_abssingle-session-userasked 2023/05/30 (Tue) 23:023,940 ctx tok

    How long have I been collecting vintage films?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  7. ABSTAINED19b5f2b3_abssingle-session-userasked 2023/05/30 (Tue) 17:145,133 ctx tok

    How long was I in Korea for?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  8. ABSTAINED2133c1b5_absknowledge-updateasked 2023/10/15 (Sun) 10:214,761 ctx tok

    How long have I been living in my current apartment in Shinjuku?

    naive
    ABSTAINED — not in the historyother seeds: INSUFFICIENT_INFORMATION: The history mentions living in a studio apartment in Harajuku for a month and then later for 3 months, but does not mention living in Shinjuku.INSUFFICIENT_INFORMATION: The history mentions living in a studio apartment in Harajuku for one month and then three months, but does not mention living in Shinjuku.
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  9. ABSTAINED2311e44b_absmulti-sessionasked 2023/05/30 (Tue) 18:255,254 ctx tok

    How many pages do I have left to read in 'Sapiens'?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  10. ABSTAINED2698e78f_absknowledge-updateasked 2023/06/16 (Fri) 16:185,103 ctx tok

    How often do I see Dr. Johnson?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  11. ABSTAINED29f2956b_abssingle-session-userasked 2023/05/30 (Tue) 20:284,779 ctx tok

    How much time do I dedicate to practicing violin every day?

    naive
    ABSTAINED — not in the historyother seeds: INSUFFICIENT_INFORMATION: The history mentions playing an instrument as a hobby but does not specify violin or daily practice time.INSUFFICIENT_INFORMATION: The chat history mentions playing an instrument as a productive hobby but does not specify violin or daily practice time.
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  12. ABSTAINED60bf93ed_absmulti-sessionasked 2023/01/24 (Tue) 23:584,153 ctx tok

    How many days did it take for my iPad case to arrive after I bought it?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  13. ABSTAINED6aeb4375_absknowledge-updateasked 2023/06/15 (Thu) 13:374,951 ctx tok

    How many Italian restaurants have I tried in my city?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  14. ABSTAINED80ec1f4f_absmulti-sessionasked 2023/03/03 (Fri) 23:444,161 ctx tok

    How many different museums or galleries did I visit in December?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  15. ABSTAINED88432d0a_absmulti-sessionasked 2023/05/30 (Tue) 23:404,996 ctx tok

    How many times did I bake egg tarts in the past two weeks?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  16. ABSTAINED982b5123_abstemporal-reasoningasked 2023/05/27 (Sat) 17:104,968 ctx tok

    When did I book the Airbnb in Sacramento?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  17. ABSTAINEDa96c20ee_absmulti-sessionasked 2023/05/30 (Tue) 12:033,675 ctx tok

    At which university did I present a poster for my undergrad course research project?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata Harvard Universityfull-context Harvard University
  18. ABSTAINEDba358f49_absmulti-sessionasked 2023/05/30 (Tue) 22:294,585 ctx tok

    How old will Rachel be when I get married?

    naive
    ABSTAINED — not in the historyother seeds: INSUFFICIENT_INFORMATION: The history mentions that the user's friend Rachel is getting married next year, but it does not state Rachel's current age or the user's age.INSUFFICIENT_INFORMATION: The history mentions that the user's friend Rachel is getting married next year, but it does not state Rachel's current age or the user's age.
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  19. ABSTAINEDbc8a6e93_abssingle-session-userasked 2023/05/30 (Tue) 15:265,108 ctx tok

    What did I bake for my uncle's birthday party?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  20. ABSTAINEDc8090214_abstemporal-reasoningasked 2023/05/30 (Tue) 05:173,781 ctx tok

    How many days before I bought my iPad did I attend the Holiday Market?

    naive
    ABSTAINED — not in the historyother seeds: INSUFFICIENT_INFORMATION: The history does not mention buying an iPad or attending a Holiday Market.INSUFFICIENT_INFORMATION: The history does not mention buying an iPad or attending a Holiday Market.
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  21. ABSTAINEDe5ba910e_absmulti-sessionasked 2023/05/30 (Tue) 17:194,712 ctx tok

    What is the total cost of my recently purchased headphones and the iPad?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  22. ABSTAINEDedced276_absmulti-sessionasked 2023/05/30 (Tue) 20:595,042 ctx tok

    How many days did I spend in total traveling in Hawaii and in Seattle?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  23. ABSTAINEDeeda8a6d_absmulti-sessionasked 2023/05/30 (Tue) 19:564,997 ctx tok

    How many fish are there in my 30-gallon tank?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  24. ABSTAINEDf4f1d8a4_abssingle-session-userasked 2023/05/30 (Tue) 21:514,613 ctx tok

    What did my dad gave me as a birthday gift?

    naive
    ABSTAINED — not in the historyother seeds: INSUFFICIENT_INFORMATION: The history mentions a silver necklace from a local boutique store that the user bought for their sister, and a dress the user decided to buy for themselves.INSUFFICIENT_INFORMATION: The history mentions a silver necklace bought for the user's sister and a dress the user decided to buy for themselves, but does not mention any gift from their dad.
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  25. ABSTAINEDf685340e_absknowledge-updateasked 2023/06/19 (Mon) 04:294,576 ctx tok

    How often do I play table tennis with my friends at the local park?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context The provided material does not contain information about playing table tennis. It mentions
  26. ABSTAINEDgpt4_372c3eed_absmulti-sessionasked 2023/08/20 (Sun) 23:543,571 ctx tok

    How many years in total did I spend in formal education from high school to the completion of my Master's degree?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  27. ABSTAINEDgpt4_70e84552_abstemporal-reasoningasked 2023/05/21 (Sun) 10:404,448 ctx tok

    Which task did I complete first, fixing the fence or purchasing three cows from Peter?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  28. ABSTAINEDgpt4_93159ced_abstemporal-reasoningasked 2023/05/30 (Tue) 18:544,156 ctx tok

    How long have I been working before I started my current job at Google?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained
  29. ABSTAINEDgpt4_c27434e8_abstemporal-reasoningasked 2023/05/23 (Tue) 11:264,640 ctx tok

    Which project did I start first, the Ferrari model or the Porsche 991 Turbo S model?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context The Ferrari 288 GTO model
  30. ABSTAINEDgpt4_fe651585_abstemporal-reasoningasked 2023/03/17 (Fri) 03:094,078 ctx tok

    Who became a parent first, Tom or Alex?

    naive
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainedfull-context abstained