Errata.
spend $——.———— · the homebuyer’s history
Memory that keeps its corrections — every answer cited, every revision kept, refusal when the history never says.

results · Full-context baseline · rerunB-nothink

Abstention 80.0

72 of 90 rows right — 30 questions × 3 seeds (11/22/33)scored deterministically by exact match — the judge never sees these rows. A row is right iff the arm abstained.same arm, by ability: OverallInformation extractionMulti-sessionTemporalKnowledge updatesame arm, by question_type: single-session-usersingle-session-assistantsingle-session-preferencemulti-sessiontemporal-reasoningknowledge-updatethe same cut, other arms: Errata 93.3Naive top-k RAG (k=10) 97.8
  1. INCORRECT031748ae_absknowledge-updateasked 2023/06/06 (Tue) 08:19112,234 ctx tok

    How many engineers do I lead when I just started my new role as Software Engineer Manager?

    full-context
    4other seeds: 45
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  2. INCORRECT09ba9854_absmulti-sessionasked 2023/05/30 (Tue) 22:56110,721 ctx tok

    How much will I save by taking the bus from the airport to my hotel instead of a taxi?

    full-context
    Based on the material provided, taking a taxi from Narita Airport to Tokyo costs approximately ¥15,000–¥20,000 (daytime) or ¥18,000–¥25,000 (late night), while the Airport Limousine Bus costs approximately ¥2,000–¥3,000. Therefore, you would save between **¥12,000 and ¥23,000** (approximately $110–$210 USD depending on exchange rates and time of day).other seeds: Based on the material provided, specifically in Session `answer_96c743d0_abs_1`, the user mentions a taxi fare of around $60. The assistant suggests that taking the train and subway would likely cost around $20-$30, and the Airport Limousine Bus is around $10-$20. However, the material does not pr…Based on the material provided, taking a taxi from Narita Airport to Tokyo costs approximately ¥15,000–¥20,000 (daytime) or ¥18,000–¥25,000 (late night), while the Airport Limousine Bus costs approximately ¥2,000–¥3,000. Therefore, you would save between **¥12,000 and ¥23,000** (approximately $110–…
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  3. INCORRECT6456829e_absmulti-sessionasked 2023/05/30 (Tue) 14:24109,162 ctx tok

    How many plants did I initially plant for tomatoes and chili peppers?

    full-context
    5
    gold
    ABSTAIN — the history never says
    errata 5 tomato plantsnaive 5
  4. INCORRECTa96c20ee_absmulti-sessionasked 2023/05/30 (Tue) 12:03111,121 ctx tok

    At which university did I present a poster for my undergrad course research project?

    full-context
    Harvard University
    gold
    ABSTAIN — the history never says
    errata Harvard Universitynaive abstained
  5. INCORRECTf685340e_absknowledge-updateasked 2023/06/19 (Mon) 04:29111,088 ctx tok

    How often do I play table tennis with my friends at the local park?

    full-context
    The provided material does not contain information about playing table tennis. It mentions playing **tennis** with friends at the local park every other week (or weekly).
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  6. INCORRECTgpt4_c27434e8_abstemporal-reasoningasked 2023/05/23 (Tue) 11:26110,712 ctx tok

    Which project did I start first, the Ferrari model or the Porsche 991 Turbo S model?

    full-context
    The Ferrari 288 GTO model
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  7. ABSTAINED0862e8bf_abssingle-session-userasked 2023/05/30 (Tue) 21:39109,673 ctx tok

    What is the name of my hamster?

    full-context
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  8. ABSTAINED0ddfec37_absknowledge-updateasked 2023/06/07 (Wed) 08:31109,568 ctx tok

    How many autographed football have I added to my collection in the first three months of collection?

    full-context
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  9. ABSTAINED15745da0_abssingle-session-userasked 2023/05/30 (Tue) 23:02109,037 ctx tok

    How long have I been collecting vintage films?

    full-context
    ABSTAINED — not in the historyother seeds: INSUFFICIENT_INFORMATION: The history mentions collecting vintage cameras, Funko POP! figures, vinyl records, and posters/art prints, but does not mention collecting vintage films.INSUFFICIENT_INFORMATION: The history mentions collecting vintage cameras for three months, but does not mention collecting vintage films.
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  10. ABSTAINED19b5f2b3_abssingle-session-userasked 2023/05/30 (Tue) 17:14110,032 ctx tok

    How long was I in Korea for?

    full-context
    ABSTAINED — not in the historyother seeds: INSUFFICIENT_INFORMATION: The history mentions a trip to Seoul and visiting the Demilitarized Zone (DMZ), but does not specify the duration of that stay.INSUFFICIENT_INFORMATION: The history mentions a trip to Seoul and the Demilitarized Zone (DMZ) in South Korea, but does not specify the duration of that stay.
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  11. ABSTAINED2133c1b5_absknowledge-updateasked 2023/10/15 (Sun) 10:21107,202 ctx tok

    How long have I been living in my current apartment in Shinjuku?

    full-context
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  12. ABSTAINED2311e44b_absmulti-sessionasked 2023/05/30 (Tue) 18:25111,112 ctx tok

    How many pages do I have left to read in 'Sapiens'?

    full-context
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  13. ABSTAINED2698e78f_absknowledge-updateasked 2023/06/16 (Fri) 16:18111,050 ctx tok

    How often do I see Dr. Johnson?

    full-context
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  14. ABSTAINED29f2956b_abssingle-session-userasked 2023/05/30 (Tue) 20:28110,756 ctx tok

    How much time do I dedicate to practicing violin every day?

    full-context
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  15. ABSTAINED60bf93ed_absmulti-sessionasked 2023/01/24 (Tue) 23:58108,939 ctx tok

    How many days did it take for my iPad case to arrive after I bought it?

    full-context
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  16. ABSTAINED6aeb4375_absknowledge-updateasked 2023/06/15 (Thu) 13:37109,451 ctx tok

    How many Italian restaurants have I tried in my city?

    full-context
    ABSTAINED — not in the historyother seeds: INSUFFICIENT_INFORMATION: The history mentions trying Korean restaurants and a new Indian restaurant, but does not state how many Italian restaurants have been tried.INSUFFICIENT_INFORMATION: The history mentions trying Korean restaurants and a new Indian restaurant, but does not state how many Italian restaurants you have tried.
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  17. ABSTAINED80ec1f4f_absmulti-sessionasked 2023/03/03 (Fri) 23:44108,497 ctx tok

    How many different museums or galleries did I visit in December?

    full-context
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  18. ABSTAINED88432d0a_absmulti-sessionasked 2023/05/30 (Tue) 23:40110,949 ctx tok

    How many times did I bake egg tarts in the past two weeks?

    full-context
    ABSTAINED — not in the historyother seeds: INSUFFICIENT_INFORMATION: The history mentions baking a chocolate cake, making a strawberry tart, and preparing various other dishes, but does not mention baking egg tarts.INSUFFICIENT_INFORMATION: The history mentions baking a chocolate cake, making a strawberry tart, and preparing sourdough bread, but does not mention baking egg tarts.
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  19. ABSTAINED982b5123_abstemporal-reasoningasked 2023/05/27 (Sat) 17:10111,528 ctx tok

    When did I book the Airbnb in Sacramento?

    full-context
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  20. ABSTAINEDba358f49_absmulti-sessionasked 2023/05/30 (Tue) 22:29109,136 ctx tok

    How old will Rachel be when I get married?

    full-context
    ABSTAINED — not in the historyother seeds: INSUFFICIENT_INFORMATION: The history mentions that Rachel is getting married next year, but it does not state your age or when you plan to get married.INSUFFICIENT_INFORMATION: The history mentions that Rachel is getting married next year, but it does not state her current age or the user's age.
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  21. ABSTAINEDbc8a6e93_abssingle-session-userasked 2023/05/30 (Tue) 15:26110,914 ctx tok

    What did I bake for my uncle's birthday party?

    full-context
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  22. ABSTAINEDc8090214_abstemporal-reasoningasked 2023/05/30 (Tue) 05:17109,347 ctx tok

    How many days before I bought my iPad did I attend the Holiday Market?

    full-context
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  23. ABSTAINEDe5ba910e_absmulti-sessionasked 2023/05/30 (Tue) 17:19107,506 ctx tok

    What is the total cost of my recently purchased headphones and the iPad?

    full-context
    ABSTAINED — not in the historyother seeds: INSUFFICIENT_INFORMATION: The history mentions the cost of Sony WH-1000XM4 headphones ($378) but does not mention purchasing an iPad.INSUFFICIENT_INFORMATION: The history mentions the cost of the Sony WH-1000XM4 headphones ($378) but does not state the cost of an iPad.
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  24. ABSTAINEDedced276_absmulti-sessionasked 2023/05/30 (Tue) 20:59109,025 ctx tok

    How many days did I spend in total traveling in Hawaii and in Seattle?

    full-context
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  25. ABSTAINEDeeda8a6d_absmulti-sessionasked 2023/05/30 (Tue) 19:56109,058 ctx tok

    How many fish are there in my 30-gallon tank?

    full-context
    ABSTAINED — not in the historyother seeds: INSUFFICIENT_INFORMATION: The history mentions a 10-gallon tank with a betta fish named Bubbles and a 20-gallon tank with neon tetras, golden honey gouramis, a pleco catfish, and potentially lemon tetras or zebra danios, but does not mention a 30-gallon tank.INSUFFICIENT_INFORMATION: The history mentions a 10-gallon tank with a betta fish named Bubbles and a 20-gallon tank with neon tetras, golden honey gouramis, a pleco catfish, and potentially lemon tetras or zebra danios, but does not mention a 30-gallon tank.
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  26. ABSTAINEDf4f1d8a4_abssingle-session-userasked 2023/05/30 (Tue) 21:51111,253 ctx tok

    What did my dad gave me as a birthday gift?

    full-context
    ABSTAINED — not in the historyother seeds: INSUFFICIENT_INFORMATION: The history mentions a stand mixer given by your sister and a silver necklace bought for your sister, but does not mention a gift from your dad.INSUFFICIENT_INFORMATION: The history mentions a sister giving the user a stand mixer as a birthday gift.
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  27. ABSTAINEDgpt4_372c3eed_absmulti-sessionasked 2023/08/20 (Sun) 23:54110,296 ctx tok

    How many years in total did I spend in formal education from high school to the completion of my Master's degree?

    full-context
    ABSTAINED — not in the historyother seeds: INSUFFICIENT_INFORMATION: The history mentions the user attended Arcadia High School from 2010 to 2014 and graduated with a Bachelor's in Computer Science from UCLA in 2020, but it does not state when or if the user completed their Master's degree.INSUFFICIENT_INFORMATION: The history mentions the user attended Arcadia High School from 2010 to 2014 and graduated with a Bachelor's in Computer Science from UCLA in 2020, but it does not state that the user completed a Master's degree.
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  28. ABSTAINEDgpt4_70e84552_abstemporal-reasoningasked 2023/05/21 (Sun) 10:40111,448 ctx tok

    Which task did I complete first, fixing the fence or purchasing three cows from Peter?

    full-context
    ABSTAINED — not in the historyother seeds: INSUFFICIENT_INFORMATION: The history mentions fixing a broken fence on the east side of the property three weeks ago, but it does not mention purchasing cows from Peter.INSUFFICIENT_INFORMATION: The history mentions fixing a broken fence on the east side of the property three weeks ago, but it does not mention purchasing cows from Peter.
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  29. ABSTAINEDgpt4_93159ced_abstemporal-reasoningasked 2023/05/30 (Tue) 18:54106,141 ctx tok

    How long have I been working before I started my current job at Google?

    full-context
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  30. ABSTAINEDgpt4_fe651585_abstemporal-reasoningasked 2023/03/17 (Fri) 03:09109,659 ctx tok

    Who became a parent first, Tom or Alex?

    full-context
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained