Errata.
spend $——.———— · the homebuyer’s history
Memory that keeps its corrections — every answer cited, every revision kept, refusal when the history never says.

results · Errata · rerunJ-arith

Information extraction 44.7

51 of 114 rows right — 38 questions × 3 seeds (11/22/33)accuracy over the non-abstention questions of this ability; mean ± sd across seeds 11/22/33same arm, by ability: OverallMulti-sessionTemporalKnowledge updateAbstentionsame arm, by question_type: single-session-usersingle-session-assistantsingle-session-preferencemulti-sessiontemporal-reasoningknowledge-updatethe same cut, other arms: Full-context baseline 80.7Naive top-k RAG (k=10) 63.2
  1. ABSTAINED0a34ad58single-session-preferenceasked 2023/05/30 (Tue) 22:401,892 ctx tokE 0.446

    I’m a bit anxious about getting around Tokyo. Do you have any helpful tips?

    errata
    ABSTAINED — not in the history
    gold
    The user would prefer responses that utilize their existing resources, such as their Suica card and TripIt app, to provide personalized tips for navigating Tokyo's public transportation. They might not prefer general tips or recommendations that do not take into account their prior preparations.
    judge
    Candidate answer is empty, missing required facts
    full-context Use your Suica card to tap in and out at ticket gates when traveling by train or bus. Makenaive * **Get There Early:** Beat the crowds by visiting early in the morning or later in the ev
  2. ABSTAINED1903adedsingle-session-assistantasked 2023/05/30 (Tue) 20:572,915 ctx tokE 0.536

    I think we discussed work from home jobs for seniors earlier. Can you remind me what was the 7th job in the list you provided?

    errata
    ABSTAINED — not in the history
    gold
    Transcriptionist.
    judge
    No candidate answer provided
    full-context Transcriptionistnaive Transcriptionist
  3. ABSTAINED1da05512single-session-preferenceasked 2023/05/30 (Tue) 15:331,705 ctx tokE 0.508

    I'm trying to decide whether to buy a NAS device now or wait. What do you think?

    errata
    ABSTAINED — not in the history
    gold
    The user would prefer responses that take into account their current home network storage capacity issues and recent reliance on external hard drives, highlighting the potential benefits of a NAS device in addressing these specific needs. They might not prefer responses that ignore their current storage challenges or fail to consider their recent tech upgrades and priorities. Preferred responses would utilize the user's previous mentions of storage capacity issues and tech investments to inform their decision.
    judge
    Candidate answer is empty, no facts provided
    full-context The material does not contain information about whether you should buy a NAS device now ornaive The material does not provide advice on whether to buy a NAS device now or wait.
  4. ABSTAINED1de5cff2single-session-assistantasked 2023/05/30 (Tue) 19:493,543 ctx tokE 0.405

    I was going through our previous conversation about high-end fashion brands, and I was wondering if you could remind me of the brand that uses wild rubber sourced from the Amazon rainforest?

    errata
    ABSTAINED — not in the history
    gold
    Veja
    judge
    Candidate answer is empty, no brand named
    full-context Vejanaive Veja
  5. ABSTAINED2bf43736single-session-assistantasked 2023/05/30 (Tue) 21:493,262 ctx tokE 0.376

    I was going through our previous chat and I wanted to clarify something about the prayer of beginners in Tanqueray's Spiritual Life treatise. Can you remind me which chapter of the second part discusses vocal prayer and meditation?

    errata
    ABSTAINED — not in the history
    gold
    Chapter 4 of Book 1, titled 'Vocal Prayer and Meditation'.
    judge
    No answer content provided by candidate
    full-context Chapter 4 of Book 1naive Chapter 4 of Book 1
  6. ABSTAINED352ab8bdsingle-session-assistantasked 2023/05/30 (Tue) 23:313,163 ctx tokE 0.594

    Can you remind me what was the average improvement in framerate when using the Hardware-Aware Modular Training (HAMT) agent in the 'To Adapt or Not to Adapt? Real-Time Adaptation for Semantic Segmentation' submission?

    errata
    ABSTAINED — not in the history
    gold
    The average improvement in framerate was approximately 20% when using the Hardware-Aware Modular Training (HAMT) agent.
    judge
    No answer provided
    full-context approximately 20%naive approximately 20%
  7. INCORRECT38146c39single-session-preferenceasked 2023/05/30 (Tue) 20:021,863 ctx tokE 0.406

    I've been feeling like my chocolate chip cookies need something extra. Any advice?

    errata
    The material does not contain advice for chocolate chip cookies; it mentions a preference for a stronger, more intense frosting flavor to balance carrot cake spices.
    gold
    The user would prefer responses that build upon their previous experimentation with turbinado sugar, suggesting ingredients or techniques that complement its richer flavor. They might not prefer generic cookie-making advice or suggestions that don't take into account their existing use of turbinado sugar.
    judge
    Candidate misses turbinado sugar fact; talks about unrelated frosting preference
    3 citations — session · turn · the span it was read from
    • 55ec6b1f:t2“I like to stay active even when I'm traveling.”
    • 0908a2e6:t6“I've been listening to audiobooks for my daily commute ... and it's been a game-changer for me.”
    • 42234f98:t0“I need something new to watch in my living room.”
    full-context Based on the material provided, there is no specific advice given for enhancing chocolate naive abstained
  8. INCORRECT54026fcesingle-session-preferenceasked 2023/05/30 (Tue) 23:481,946 ctx tokE 0.348

    I've been thinking about ways to stay connected with my colleagues. Any suggestions?

    errata
    The user likes framing ideas as collaborative suggestions and wants everyone to share their thoughts.
    gold
    The user would prefer responses that acknowledge their desire for social interaction and collaboration while working remotely, utilizing their previous experiences with company initiatives and team collaborations. They might prefer suggestions of virtual team-building activities, regular check-ins, or joining interest-based groups within the company. The user may not prefer generic suggestions that do not take into account their specific work situation or previous attempts at staying connected with colleagues.
    judge
    Candidate lacks specific suggestions like virtual team-building or check-ins from reference
    3 citations — session · turn · the span it was read from
    • ef5d4aa7_1:t0“I've already got a personalized coffee mug from Etsy that cost around $25, including shipping”
    • 9733011f_1:t8“I'm planning to stay in Cortina d'Ampezzo for a few days.”
    • ef2e4f61_1:t2“I've already started using a sleep mask to block out any light and I think it's made a big difference.”
    full-context Here are a few suggestions to socialize with your colleagues while working from home: 1. naive Virtual Coffee Breaks, Online Team Activities, Collaborative Projects, Social Channels, Re
  9. ABSTAINED5809eb10single-session-assistantasked 2023/05/30 (Tue) 23:233,120 ctx tokE 0.539

    I'm looking back at our previous conversation about the Bajimaya v Reward Homes Pty Ltd case. Can you remind me what year the construction of the house began?

    errata
    ABSTAINED — not in the history
    gold
    2014.
    judge
    No answer provided
    full-context The construction of the house began in 2014.naive 2014
  10. INCORRECT6ae235besingle-session-assistantasked 2023/05/30 (Tue) 23:412,831 ctx tokE 0.518

    I remember you told me about the refining processes at CITGO's three refineries earlier. Can you remind me what kind of processes are used at the Lake Charles Refinery?

    errata
    The material does not specify which refining processes are used at the Lake Charles Refinery specifically; it only lists two different sets of general refining processes (atmospheric distillation, fluid catalytic cracking, alkylation, and hydrotreating; or atmospheric distillation, delayed coking, hydrocracking, and hydrotreating) and states that CITGO operates refineries in Lake Charles, Louisiana; Lemont, Illinois; and Corpus Christi, Texas.
    gold
    Atmospheric distillation, fluid catalytic cracking (FCC), alkylation, and hydrotreating.
    judge
    Candidate refuses to specify, doesn't confirm Lake Charles' actual processes
    3 citations — session · turn · the span it was read from
    • answer_sharegpt_IUWQYGQ_0:t1“* Atmospheric distillation... * Fluid catalytic cracking (FCC)... * Alkylation... * Hydrotreating”
    • answer_sharegpt_IUWQYGQ_0:t1“* Atmospheric distillation... * Delayed coking... * Hydrocracking... * Hydrotreating”
    • answer_sharegpt_IUWQYGQ_0:t1“CITGO operates three refineries in the United States, located in Lake Charles, Louisiana; Lemont, Illinois; and Corpus Christi, Texas.”
    full-context The Lake Charles Refinery uses the following processes: * Atmospheric distillation * naive The Lake Charles Refinery uses atmospheric distillation, fluid catalytic cracking (FCC), a
  11. INCORRECT6b7dfb22single-session-preferenceasked 2023/05/30 (Tue) 13:571,976 ctx tokE 0.479

    I've been feeling a bit stuck with my paintings lately. Do you have any ideas on how I can find new inspiration?

    errata
    The user gets inspiration from social media.
    gold
    The user would prefer responses that build upon their existing sources of inspiration, such as revisiting Instagram art accounts or exploring new techniques from online tutorials. They might also appreciate suggestions that revisit previous themes they found enjoyable, like painting flowers. The user would not prefer generic or vague suggestions for finding inspiration, and would likely appreciate responses that utilize their recent 30-day painting challenge experience.
    judge
    Missing tutorials, flower theme, and 30-day challenge details
    3 citations — session · turn · the span it was read from
    • 574df152_1:t0“I've been feeling really disconnected from my social circle lately and I'm not sure how to get out of this rut.”
    • answer_f6502d0f:t12“I have been getting inspiration from social media and recently started a 30-day painting challenge.”
    • 009c1517_1:t10“I've been inspired by the guitar work of David Gilmour and Stevie Ray Vaughan.”
    full-context The material mentions that you have been getting inspiration from social media.naive Hang art pieces, prints, or quotes that motivate you. Add plants or a small music system t
  12. ABSTAINED7161e7e2single-session-assistantasked 2023/05/30 (Tue) 20:163,000 ctx tokE 0.358

    I'm checking our previous chat about the shift rotation sheet for GM social media agents. Can you remind me what was the rotation for Admon on a Sunday?

    errata
    ABSTAINED — not in the history
    gold
    Admon was assigned to the 8 am - 4 pm (Day Shift) on Sundays.
    judge
    Candidate answer is empty, no fact provided
    full-context Admon was scheduled for the 8 am - 4 pm (Day Shift) on Sunday.naive Admon works the 8 am - 4 pm (Day Shift) on Sunday.
  13. ABSTAINED71a3fd6bsingle-session-assistantasked 2023/05/30 (Tue) 21:383,180 ctx tokE 0.480

    I'm planning my trip to Speyer again and I wanted to confirm, what's the phone number of the Speyer tourism board that you provided me earlier?

    errata
    ABSTAINED — not in the history
    gold
    +49 (0) 62 32 / 14 23 - 0
    judge
    Candidate provided no answer
    full-context +49 (0) 62 32 / 14 23 - 0naive +49 (0) 62 32 / 14 23 - 0
  14. ABSTAINED75f70248single-session-preferenceasked 2023/05/30 (Tue) 17:511,965 ctx tokE 0.432

    I've been sneezing quite a bit lately. Do you think it might be my living room?

    errata
    ABSTAINED — not in the history
    gold
    The user would prefer responses that consider the potential impact of their cat, Luna, and her shedding on their sneezing, as well as the recent deep clean of the living room and its possible effect on stirring up dust. They might not prefer responses that fail to take into account these specific details previously mentioned, such as generic suggestions or unrelated factors.
    judge
    Candidate answer is empty, missing cat/deep clean details.
    full-context Yes, it could be related to your living room, especially if you have a cat that sheds a lonaive Yes, it might be your living room if you have a cat that sheds a lot, as this can contribu
  15. ABSTAINED8464fc84single-session-assistantasked 2023/05/30 (Tue) 23:011,919 ctx tokE 0.361

    I'm planning to visit the Vatican again and I was wondering if you could remind me of the name of that famous deli near the Vatican that serves the best cured meats and cheeses?

    errata
    ABSTAINED — not in the history
    gold
    Roscioli
    judge
    No answer provided
    full-context Rosciolinaive Roscioli
  16. ABSTAINED8aef76bcsingle-session-assistantasked 2023/05/30 (Tue) 05:063,214 ctx tokE 0.587

    I'm going back to our previous conversation about DIY home decor projects using recycled materials. Can you remind me what sealant you recommended for the newspaper flower vase?

    errata
    ABSTAINED — not in the history
    gold
    Mod Podge or another sealant
    judge
    Candidate answer is empty, no sealant mentioned
    full-context Mod Podge or another sealantnaive Mod Podge or another sealant
  17. UNPARSEABLE95228167single-session-preferenceasked 2023/05/30 (Tue) 21:041,916 ctx tokE 0.401

    I'm getting excited about my visit to the music store this weekend. Any tips on what to look for in a new guitar?

    errata
    The user is considering upgrading from a Fender Stratocaster to a Gibson Les Paul.
    gold
    The user would prefer responses that highlight the differences between Fender Stratocaster and Gibson Les Paul electric guitars, such as the feel of the neck, weight, and sound profile. They might not prefer general tips on buying an electric guitar or suggestions that do not take into account their current guitar and desired upgrade.
    3 citations — session · turn · the span it was read from
    • 57d6e39b_2:t6“I'm thinking of getting a lighter hybrid bike for her”
    • 5d8c99d3_3:t4“I'm actually planning to explore the John Muir Wilderness area during this trip.”
    • f07752be_1:t2“I'll make sure to schedule the appointment for 12:00 PM”
    full-context Based on the chat history provided, there is no mention of a visit to a music store this wnaive abstained
  18. ABSTAINEDb759caeesingle-session-assistantasked 2023/05/30 (Tue) 23:503,226 ctx tokE 0.426

    I was looking back at our previous conversation about buying unique engagement rings directly from designers. Can you remind me of the Instagram handle of the UK-based designer who works with unusual gemstones?

    errata
    ABSTAINED — not in the history
    gold
    @jessica_poole_jewellery
    judge
    Candidate provided no answer
    full-context @jessica_poole_jewellerynaive @jessica_poole_jewellery
  19. ABSTAINEDcc539528single-session-assistantasked 2023/05/30 (Tue) 19:203,324 ctx tokE 0.442

    I wanted to follow up on our previous conversation about front-end and back-end development. Can you remind me of the specific back-end programming languages you recommended I learn?

    errata
    ABSTAINED — not in the history
    gold
    I recommended learning Ruby, Python, or PHP as a back-end programming language.
    judge
    Candidate answer is empty, missing required facts
    full-context Ruby, Python, or PHPnaive Ruby, Python, or PHP
  20. ABSTAINEDceb54acbsingle-session-assistantasked 2023/05/30 (Tue) 22:563,262 ctx tokE 0.377

    In our previous chat, you suggested 'sexual compulsions' and a few other options for alternative terms for certain behaviors. Can you remind me what the other four options were?

    errata
    ABSTAINED — not in the history
    gold
    I suggested 'sexual fixations', 'problematic sexual behaviors', 'sexual impulsivity', and 'compulsive sexuality'.
    judge
    Candidate answer is empty, provides no terms
    full-context Sexual fixations, Problematic sexual behaviors, Sexual impulsivity, Compulsive sexualitynaive Sexual fixations, Problematic sexual behaviors, Sexual impulsivity, Compulsive sexuality
  21. ABSTAINEDd6233ab6single-session-preferenceasked 2023/05/30 (Tue) 01:291,884 ctx tokE 0.443

    I've been feeling nostalgic lately. Do you think it would be a good idea to attend my high school reunion?

    errata
    ABSTAINED — not in the history
    gold
    The user would prefer responses that draw upon their personal experiences and memories, specifically their positive high school experiences such as being part of the debate team and taking advanced placement courses. They would prefer suggestions that highlight the potential benefits of attending the reunion, such as reconnecting with old friends and revisiting favorite subjects like history and economics. The user might not prefer generic or vague responses that do not take into account their individual experiences and interests.
    judge
    No candidate answer provided
    full-context abstainednaive abstained
  22. CORRECT0862e8bfsingle-session-userasked 2023/05/30 (Tue) 12:181,797 ctx tokE 0.670

    What is the name of my cat?

    errata
    Luna
    gold
    Luna
    judge
    Candidate states 'Luna', matching reference answer exactly.
    3 citations — session · turn · the span it was read from
    • answer_c6fd8ebd:t10“my cat's name is Luna”
    • answer_c6fd8ebd:t10“my cat's name is Luna”
    • 6c172949_1:t4“I think the resort was called Oakwood Hills Resort”
    full-context Lunanaive Luna
  23. CORRECT1568498asingle-session-assistantasked 2023/05/30 (Tue) 23:353,347 ctx tokE 0.528

    I'm looking back at our previous chess game and I was wondering, what was the move you made after 27. Kg2 Bd5+?

    errata
    28. Kg3 Be6
    gold
    28. Kg3
    judge
    Candidate includes 28. Kg3, matching reference, with extra move
    3 citations — session · turn · the span it was read from
    • answer_sharegpt_d6JJiqH_76:t22“1. e4 c5 2. Nf3 Nf6 3. d4 cxd4 ... 27. Kg2 Bd5+ 28. Kg3 Be6”
    • answer_sharegpt_d6JJiqH_76:t8“25. g4 hxg4 26. hxg4 Rh8”
    • 6ad46850_2:t0“I'm still undecided which one to get”
    full-context 28. Kg3naive 28. Kg3
  24. CORRECT19b5f2b3single-session-userasked 2023/05/30 (Tue) 20:502,871 ctx tokE 0.511

    How long was I in Japan for?

    errata
    two weeks
    gold
    two weeks
    judge
    Matches reference: two weeks
    3 citations — session · turn · the span it was read from
    • answer_5ff494b9:t2“I spent two weeks traveling solo around the country and it was an incredible experience.”
    • answer_5ff494b9:t4“I stayed in small hostels and guesthouses during my solo trip to Japan”
    • answer_5ff494b9:t2“I spent two weeks traveling solo around the country and it was an incredible experience.”
    full-context Two weeksnaive two weeks
  25. CORRECT21436231single-session-userasked 2023/05/30 (Tue) 19:251,820 ctx tokE 0.541

    How many largemouth bass did I catch on my fishing trip to Lake Michigan?

    errata
    12
    gold
    12
    judge
    Matches quantity 12
    3 citations — session · turn · the span it was read from
    • answer_1e6d4567:t2“I caught 12 largemouth bass on my last trip there”
    • answer_1e6d4567:t2“I caught 12 largemouth bass on my last trip there”
    • answer_1e6d4567:t0“I'm planning a fishing trip to Lake Michigan with my friends”
    full-context 12naive 12
  26. CORRECT29f2956bsingle-session-userasked 2023/05/30 (Tue) 20:453,333 ctx tokE 0.521

    How much time do I dedicate to practicing guitar every day?

    errata
    30 minutes daily
    gold
    30 minutes
    judge
    Both state 30 minutes daily practice
    3 citations — session · turn · the span it was read from
    • answer_7cc5362f:t0“I've been practicing guitar for 30 minutes daily, and it's been helping me progress nicely.”
    • answer_7cc5362f:t0“I've been practicing guitar for 30 minutes daily, and it's been helping me progress nicely.”
    • 00842aad_2:t0“I've been trying to wake up at 7:30 am every weekday”
    full-context 30 minutesnaive 30 minutes daily
  27. CORRECT36580ce8single-session-userasked 2023/05/30 (Tue) 22:092,980 ctx tokE 0.503

    What health issue did I initially think was just a cold?

    errata
    bronchitis
    gold
    bronchitis
    judge
    Matches reference fact exactly: bronchitis
    3 citations — session · turn · the span it was read from
    • answer_93e1bd22:t4“I actually recently had a bad case of bronchitis that I initially thought was just a cold”
    • answer_93e1bd22:t4“I actually recently had a bad case of bronchitis that I initially thought was just a cold”
    • answer_93e1bd22:t0“I'm trying to get back on track with my fitness routine after a few months of dealing with health issues.”
    full-context bronchitisnaive bronchitis
  28. CORRECT577d4d32single-session-userasked 2023/05/30 (Tue) 23:542,087 ctx tokE 0.541

    What time do I stop checking work emails and messages?

    errata
    7 pm
    gold
    7 pm
    judge
    Matches reference fact of 7 pm
    3 citations — session · turn · the span it was read from
    • answer_0dd4d99a:t4“I've been trying to establish a better evening routine, stopping work emails and messages by 7 pm to separate my work and personal life.”
    • answer_0dd4d99a:t4“stopping work emails and messages by 7 pm to separate my work and personal life”
    • b283f9eb_1:t10“I'll make sure to check the data types, handle duplicates, and align the data correctly.”
    full-context 7 pmnaive abstained
  29. CORRECT60d45044single-session-userasked 2023/05/30 (Tue) 20:461,869 ctx tokE 0.628

    What type of rice is my favorite?

    errata
    Japanese short-grain rice
    gold
    Japanese short-grain rice
    judge
    Exact match: Japanese short-grain rice
    3 citations — session · turn · the span it was read from
    • answer_9cddca88:t4“my favorite Japanese short-grain rice”
    • answer_9cddca88:t4“my favorite Japanese short-grain rice”
    • 35a61ccd_1:t0“I'm thinking of renovating an old farmhouse on a plot of land I inherited from my grandfather.”
    full-context Japanese short-grain ricenaive abstained
  30. CORRECT6ade9755single-session-userasked 2023/05/30 (Tue) 21:041,929 ctx tokE 0.696

    Where do I take yoga classes?

    errata
    Serenity Yoga
    gold
    Serenity Yoga
    judge
    Matches reference entity exactly: Serenity Yoga
    3 citations — session · turn · the span it was read from
    • answer_9398da02:t6“I usually stick to the vinyasa flow classes, they're my favorite.”
    • answer_9398da02:t6“I usually stick to the vinyasa flow classes, they're my favorite.”
    • answer_9398da02:t10“We've even started a Sunday morning yoga and brunch routine”
    full-context Serenity Yoganaive Serenity Yoga
  31. CORRECT6b168ec8single-session-userasked 2023/05/30 (Tue) 20:241,769 ctx tokE 0.485

    How many bikes do I own?

    errata
    three
    gold
    three
    judge
    Both state three bikes
    3 citations — session · turn · the span it was read from
    • answer_e623ae87:t10“I think I'll go with the Abus SmartX bike lock with GPS tracking”
    • answer_e623ae87:t4“I've got three of them - a road bike, a mountain bike, and a commuter bike”
    • answer_e623ae87:t2“I'm actually thinking of taking my Trek Emonda for the ride”
    full-context You own three bikes.naive 3
  32. CORRECT8550ddaesingle-session-userasked 2023/05/30 (Tue) 23:512,858 ctx tokE 0.667

    What type of cocktail recipe did I try last weekend?

    errata
    lavender gin fizz
    gold
    lavender gin fizz
    judge
    Matches exactly: lavender gin fizz
    3 citations — session · turn · the span it was read from
    • answer_c8354ae9:t2“I tried a lavender gin fizz recipe last weekend, but it didn't quite turn out as expected”
    • 8ec23b2c:t4“after playing it for 10 hrs last weekend.”
    • 8ec23b2c:t4“after playing it for 10 hrs last weekend”
    full-context Lavender gin fizznaive lavender gin fizz
  33. CORRECT8ebdbe50single-session-userasked 2023/05/30 (Tue) 18:442,922 ctx tokE 0.660

    What certification did I complete last month?

    errata
    Data Science certification
    gold
    Data Science
    judge
    States Data Science certification, matching reference fact
    3 citations — session · turn · the span it was read from
    • answer_8ad8a34f:t2“I need to add my latest certification in Data Science, which I completed last month, to my profile.”
    • answer_8ad8a34f:t2“my latest certification in Data Science, which I completed last month”
    • d4f7a065_2:t0“grilled salmon with roasted vegetables on Saturday”
    full-context Data Sciencenaive Data Science
  34. CORRECTa82c026esingle-session-userasked 2023/05/30 (Tue) 21:262,829 ctx tokE 0.408

    What game did I finally beat last weekend?

    errata
    Dark Souls 3 DLC
    gold
    Dark Souls 3 DLC
    judge
    Matches exactly: Dark Souls 3 DLC
    3 citations — session · turn · the span it was read from
    • answer_787e6a6d:t2“I finally beat that last boss in the Dark Souls 3 DLC last weekend, after weeks of trying.”
    • answer_787e6a6d:t2“I finally beat that last boss in the Dark Souls 3 DLC last weekend, after weeks of trying.”
    • ccaabf0b_1:t0“I've played 8 games of soccer”
    full-context Dark Souls 3 DLCnaive abstained
  35. CORRECTad7109d1single-session-userasked 2023/05/30 (Tue) 21:562,067 ctx tokE 0.651

    What speed is my new internet plan?

    errata
    500 Mbps
    gold
    500 Mbps
    judge
    Matches reference value exactly: 500 Mbps
    3 citations — session · turn · the span it was read from
    • answer_679840f8:t4“I upgraded to 500 Mbps about three weeks ago, and it's made a huge difference.”
    • answer_679840f8:t4“I upgraded to 500 Mbps about three weeks ago”
    • 7ad0bd25_1:t6“I'm thinking of trying out a new energy bar for my upcoming cycling event.”
    full-context 500 Mbpsnaive 500 Mbps
  36. CORRECTc19f7a0bsingle-session-userasked 2023/05/30 (Tue) 22:191,791 ctx tokE 0.706

    What time do I usually get home from work on weeknights?

    errata
    around 6:30 pm
    gold
    6:30 pm
    judge
    Same time, 6:30 pm, just with 'around'
    3 citations — session · turn · the span it was read from
    • answer_f442ccbe:t8“I usually get home from work around 6:30 pm on weekdays”
    • answer_f442ccbe:t8“I usually get home from work around 6:30 pm on weekdays”
    • answer_f442ccbe:t2“I usually have about an hour to spare between getting home from work and starting dinner”
    full-context 6:30 pmnaive abstained
  37. CORRECTe47becbasingle-session-userasked 2023/05/30 (Tue) 23:401,925 ctx tokE 0.468

    What degree did I graduate with?

    errata
    Business Administration
    gold
    Business Administration
    judge
    Matches exactly: Business Administration
    3 citations — session · turn · the span it was read from
    • answer_280352e9:t4“I graduated with a degree in Business Administration”
    • answer_280352e9:t4“I graduated with a degree in Business Administration”
    • 4fd76316:t10“I think I'll call Smokey's to check their wait times.”
    full-context Business Administrationnaive abstained
  38. CORRECTf8c5f88bsingle-session-userasked 2023/05/30 (Tue) 23:391,579 ctx tokE 0.668

    Where did I buy my new tennis racket from?

    errata
    a sports store downtown
    gold
    the sports store downtown
    judge
    Same fact: sports store downtown, article difference only
    3 citations — session · turn · the span it was read from
    • answer_c3567066:t4“I'm really happy with my new tennis racket, which I got from a sports store downtown.”
    • answer_c3567066:t4“It's been performing really well, and I feel like it's improved my game.”
    • answer_c3567066:t4“I'm really happy with my new tennis racket, which I got from a sports store downtown.”
    full-context a sports store downtownnaive a sports store downtown