Remote
Software Engineer, Agent Optimization & Evaluation
About this role
Reddit is a community of communities. It’s built on shared interests, passion, and trust, and is home to the most open and authentic conversations on the internet. Every day, Reddit users submit, vote, and comment on the topics they care most about. With 100,000+ active communities and approximately 130 million daily active unique visitors, Reddit is one of the internet’s largest sources of information. For more information, visit www.redditinc.com .
We are Software Engineers across multiple levels! The Team We're building Reddit's next-generation AI developer platform: a best-in-class agent stack that helps every Snoo build, test, review, and ship software efficiently while preserving Reddit's control over quality, costs, and long-term AI strategy. Our optimization harness measures and improves agents across the software development lifecycle by benchmarking models, detecting regressions, optimizing routing, incorporating production feedback, and improving developer productivity.
You'll report to Kyle Maxwell and partner with Platform Engineering, Infrastructure, and Developer Productivity to shape AI-assisted software development across Reddit. The Role As a Software Engineer, Agent Optimization & Evaluation , you will lead our agent optimization and evaluation platform. You'll build systems to measure and improve agents' impact on engineering velocity, including Reddit-specific evaluation environments, intelligent model orchestration, and feedback loops that learn from production usage.
This hands-on role involves writing production code and building new systems with ownership and autonomy. Strong platform and infrastructure partners will support your work on agent optimization, evaluation, and orchestration. What You’ll Do: Design and build Reddit's agent optimization harness for evaluating, benchmarking, and improving AI agents across real software engineering workflows. Develop Reddit-specific evaluation environments that accurately measure agent performance against meaningful engineering tasks.
Build systems for offline replay, automated evaluation, synthetic task generation, regression detection, and human-in-the-loop feedback. Design model routing and orchestration strategies that optimize for quality, latency, and cost. Define metrics and experimentation frameworks that measure developer productivity and engineering velocity. Build tooling to analyze agent behavior, investigate failures, and accelerate iteration.