Errata.
spend $——.———— · the homebuyer’s history
Memory that keeps its corrections — every answer cited, every revision kept, refusal when the history never says.

results · Full-context baseline · rerunB-nothink

single-session-user 100.0

66 of 66 rows right — 22 questions × 3 seeds (11/22/33)the corpus’s own question_type, over all 450 rows — a gold-abstention row is right iff the arm abstained, every other row iff the judge said CORRECTsame arm, by ability: OverallInformation extractionMulti-sessionTemporalKnowledge updateAbstentionsame arm, by question_type: single-session-assistantsingle-session-preferencemulti-sessiontemporal-reasoningknowledge-updatethe same cut, other arms: Errata 100.0Naive top-k RAG (k=10) 77.3
  1. CORRECT0862e8bfsingle-session-userasked 2023/05/30 (Tue) 12:18105,949 ctx tok

    What is the name of my cat?

    full-context
    Luna
    gold
    Luna
    judge
    Candidate states the same name, Luna.
    errata Lunanaive Luna
  2. ABSTAINED0862e8bf_abssingle-session-userasked 2023/05/30 (Tue) 21:39109,673 ctx tok

    What is the name of my hamster?

    full-context
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  3. ABSTAINED15745da0_abssingle-session-userasked 2023/05/30 (Tue) 23:02109,037 ctx tok

    How long have I been collecting vintage films?

    full-context
    ABSTAINED — not in the historyother seeds: INSUFFICIENT_INFORMATION: The history mentions collecting vintage cameras, Funko POP! figures, vinyl records, and posters/art prints, but does not mention collecting vintage films.INSUFFICIENT_INFORMATION: The history mentions collecting vintage cameras for three months, but does not mention collecting vintage films.
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  4. CORRECT19b5f2b3single-session-userasked 2023/05/30 (Tue) 20:50107,993 ctx tok

    How long was I in Japan for?

    full-context
    Two weeksother seeds: two weekstwo weeks
    gold
    two weeks
    judge
    Same fact: two weeks, just capitalization difference
    errata two weeksnaive two weeks
  5. ABSTAINED19b5f2b3_abssingle-session-userasked 2023/05/30 (Tue) 17:14110,032 ctx tok

    How long was I in Korea for?

    full-context
    ABSTAINED — not in the historyother seeds: INSUFFICIENT_INFORMATION: The history mentions a trip to Seoul and visiting the Demilitarized Zone (DMZ), but does not specify the duration of that stay.INSUFFICIENT_INFORMATION: The history mentions a trip to Seoul and the Demilitarized Zone (DMZ) in South Korea, but does not specify the duration of that stay.
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  6. CORRECT21436231single-session-userasked 2023/05/30 (Tue) 19:25109,269 ctx tok

    How many largemouth bass did I catch on my fishing trip to Lake Michigan?

    full-context
    12
    gold
    12
    judge
    Matches quantity: 12
    errata 12naive 12
  7. CORRECT29f2956bsingle-session-userasked 2023/05/30 (Tue) 20:45107,223 ctx tok

    How much time do I dedicate to practicing guitar every day?

    full-context
    30 minutes
    gold
    30 minutes
    judge
    Matches reference value: 30 minutes
    errata 30 minutes dailynaive 30 minutes daily
  8. ABSTAINED29f2956b_abssingle-session-userasked 2023/05/30 (Tue) 20:28110,756 ctx tok

    How much time do I dedicate to practicing violin every day?

    full-context
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  9. CORRECT36580ce8single-session-userasked 2023/05/30 (Tue) 22:09108,197 ctx tok

    What health issue did I initially think was just a cold?

    full-context
    bronchitisother seeds: Bronchitisbronchitis
    gold
    bronchitis
    judge
    Both state bronchitis
    errata bronchitisnaive bronchitis
  10. CORRECT577d4d32single-session-userasked 2023/05/30 (Tue) 23:54109,671 ctx tok

    What time do I stop checking work emails and messages?

    full-context
    7 pm
    gold
    7 pm
    judge
    Matches reference: 7 pm
    errata 7 pmnaive abstained
  11. CORRECT60d45044single-session-userasked 2023/05/30 (Tue) 20:46111,448 ctx tok

    What type of rice is my favorite?

    full-context
    Japanese short-grain rice
    gold
    Japanese short-grain rice
    judge
    Exact match: Japanese short-grain rice
    errata Japanese short-grain ricenaive abstained
  12. CORRECT6ade9755single-session-userasked 2023/05/30 (Tue) 21:04109,299 ctx tok

    Where do I take yoga classes?

    full-context
    Serenity Yoga
    gold
    Serenity Yoga
    judge
    Matches reference exactly: Serenity Yoga
    errata Serenity Yoganaive Serenity Yoga
  13. CORRECT6b168ec8single-session-userasked 2023/05/30 (Tue) 20:24111,387 ctx tok

    How many bikes do I own?

    full-context
    You own three bikes.
    gold
    three
    judge
    Both state quantity of three bikes.
    errata threenaive 3
  14. CORRECT8550ddaesingle-session-userasked 2023/05/30 (Tue) 23:51111,210 ctx tok

    What type of cocktail recipe did I try last weekend?

    full-context
    Lavender gin fizz
    gold
    lavender gin fizz
    judge
    Same fact: lavender gin fizz, just capitalized differently
    errata lavender gin fizznaive lavender gin fizz
  15. CORRECT8ebdbe50single-session-userasked 2023/05/30 (Tue) 18:44111,515 ctx tok

    What certification did I complete last month?

    full-context
    Data Science
    gold
    Data Science
    judge
    Matches reference fact: Data Science certification
    errata Data Science certificationnaive Data Science
  16. CORRECTa82c026esingle-session-userasked 2023/05/30 (Tue) 21:26111,179 ctx tok

    What game did I finally beat last weekend?

    full-context
    Dark Souls 3 DLC
    gold
    Dark Souls 3 DLC
    judge
    Matches exactly: Dark Souls 3 DLC
    errata Dark Souls 3 DLCnaive abstained
  17. CORRECTad7109d1single-session-userasked 2023/05/30 (Tue) 21:56106,871 ctx tok

    What speed is my new internet plan?

    full-context
    500 Mbps
    gold
    500 Mbps
    judge
    Both state 500 Mbps
    errata 500 Mbpsnaive 500 Mbps
  18. ABSTAINEDbc8a6e93_abssingle-session-userasked 2023/05/30 (Tue) 15:26110,914 ctx tok

    What did I bake for my uncle's birthday party?

    full-context
    ABSTAINED — not in the history
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  19. CORRECTc19f7a0bsingle-session-userasked 2023/05/30 (Tue) 22:19109,889 ctx tok

    What time do I usually get home from work on weeknights?

    full-context
    6:30 pm
    gold
    6:30 pm
    judge
    Matches exact time stated in reference
    errata around 6:30 pmnaive abstained
  20. CORRECTe47becbasingle-session-userasked 2023/05/30 (Tue) 23:40109,436 ctx tok

    What degree did I graduate with?

    full-context
    Business Administration
    gold
    Business Administration
    judge
    Matches reference: Business Administration
    errata Business Administrationnaive abstained
  21. ABSTAINEDf4f1d8a4_abssingle-session-userasked 2023/05/30 (Tue) 21:51111,253 ctx tok

    What did my dad gave me as a birthday gift?

    full-context
    ABSTAINED — not in the historyother seeds: INSUFFICIENT_INFORMATION: The history mentions a stand mixer given by your sister and a silver necklace bought for your sister, but does not mention a gift from your dad.INSUFFICIENT_INFORMATION: The history mentions a sister giving the user a stand mixer as a birthday gift.
    gold
    ABSTAIN — the history never says
    errata abstainednaive abstained
  22. CORRECTf8c5f88bsingle-session-userasked 2023/05/30 (Tue) 23:39109,490 ctx tok

    Where did I buy my new tennis racket from?

    full-context
    a sports store downtown
    gold
    the sports store downtown
    judge
    Same fact: sports store downtown, minor article difference
    errata a sports store downtownnaive a sports store downtown