The real agent setups — goals, failure modes, and live run summaries
Each of these 6 agents runs separately, for a separate purpose — this isn't just the end result, it's each setup's real run breakdown across the sample. Click a status chip on any card for a real restaurant sample at that stage. 3-layer QA methodology: Agent Runs → Human QA (blind relabel) → Adjudication; unresolved spec placeholders are shown honestly, not resolved to a finished number.
Confirm the location has functional online ordering, excluding static menus, before any downstream workstream runs.
A real, interactive ordering page is reachable and distinguishable from a static PDF/image menu. The agent halts here — never proceeding to POS/menu extraction — when either stop condition below is met.
- Initial agent confused static menus with functional ordering (74% initial accuracy).
- Stop condition — third-party listing only: the extracted ordering surface resolves to a marketplace domain (DoorDash/Grubhub/Uber Eats/etc.), not the restaurant's own site — real rate: 3.5% of 5,279 restaurants.
- Stop condition — static PDF menu: the ordering surface is a static PDF/image, not a structured page an agent can parse — real rate: 0.2%.
9,000+ locations (100%) · 100% after full human validation + retraining
Identify the ordering system (Toast, Square, Slice, custom, 50+ platforms), 1st vs 3rd-party.
Vendor, evidence (URL/technical proof), reasoning, and has_online_ordering are all produced.
- Ambiguous checkout DOM signatures across white-labeled platforms.
230 locations (5%) · [CONFIRM: 95%+] — reported, not yet confirmed in source spec
Capture every listed beverage — name, brand, sub-brand, price, image.
Generic listings resolve to a real brand via item variations; hidden drink listings (e.g. under "Sides") are found.
- Generic listings requiring a click-through to reveal brand.
- Hidden drink listings buried in non-beverage sections.
339 locations (7%) · [CONFIRM]% — reported, not yet confirmed in source spec
Add an item to cart and proceed through checkout to detect a beverage upsell prompt.
What/where/which brands the upsell offers, captured per platform's UI (e.g. Toast's rotating carousel).
- Platform-specific UI variance required per-platform prompt engineering.
330 locations (7%) — pilot's broader sample, not this cohort · [CONFIRM: ~90%] — QA found fully trained agents outperformed human annotators here
Identify meal bundles containing beverages and map brands within them.
Correctly distinguishes a required beverage selection from a pre-included one; finds bundles outside a dedicated "Combos" section.
- Combos where beverage is a required selection, not pre-included.
- Bundles listed outside a dedicated section.
230 locations (5%) — pilot's broader sample, not this cohort · [CONFIRM: 97%+] — reported, not yet confirmed in source spec
Evaluate product images for presence, type (professional vs UGC), and content.
Correctly classifies image_source (listing_thumbnail / detail_view / brand_option) where present.
- Most beverage listings have no image at all — ~5% of ~57,800 items pilot-wide.
250 locations — pilot's broader sample, not this cohort · 231 completed, 678 of 3,549 scanned items (19%) had images