Product Manager, Toast IQ Evaluation

About this role

Toast creates technology to help restaurants and local businesses succeed in a digital world, helping business owners operate, increase sales, engage customers, and keep employees happy. At the center of this vision is Toast IQ: a single, unified assistant that orchestrates specialized agents across the Toast ecosystem. The Toast IQ Quality team makes sure Toast IQ delivers useful, reliable help to restaurant operators.

About the Role As Product Manager, Toast IQ Evaluation, you’ll define what good help looks like for a restaurant operator and build the evidence that tells us whether Toast IQ delivers it. You’ll study real usage, work directly with operators, and turn what you learn into datasets, clear quality standards, and priorities for improvement. Your work will help teams across Toast understand where the assistant succeeds, where it falls short, and identify new opportunities for improvement.

This is a hands-on product role. Much of your week will be spent reviewing sessions, labeling examples, investigating failures, and building better ways to do that work. Restaurant judgment, attention to detail, and the ability to turn evidence into action are central to the job. You’ll be the team’s first restaurant domain expert, working alongside engineers from one prioritized backlog and reporting through Product.

A Day in the life (Responsibilities): Understand the operator’s experience. Review real Toast IQ sessions every week. Describe what went wrong in plain operator language. Work directly with operators, including in live sessions, to check that our evaluations reflect what matters to them. Find and prioritize recurring failures. Group observations into failure modes, measure how often they occur, and assess their impact on operators.

Use that evidence to identify the problems most worth solving. Build trustworthy evaluation datasets. Create and maintain labeled examples covering common workflows, edge cases, and known failures. Verify that improvements hold up. Validate fixes in real sessions and turn important production failures into regression cases that catch the same problems in future releases. Help teams across Toast evaluate their work. Lead reviewer calibration sessions and help product teams create and maintain evaluation cases for their capabilities.