Software Engineer, Agent Optimization & Evaluation

About this role

📋 Description Design and build Reddit's agent optimization harness for evaluating, benchmarking, and improving AI Develop Reddit-specific evaluation environments that accurately measure agent performance against Build systems for offline replay, automated evaluation, synthetic task generation, regression Design model routing and orchestration strategies that optimize for quality, latency, and cost. Define metrics and experimentation frameworks that measure developer productivity and engineering Build tooling to analyze agent behavior, investigate failures, and accelerate iteration.

🎯 Requirements 5-12+ years of engineering experience Extensive experience building and shipping large-scale production software systems. Experience designing evaluation, experimentation, benchmarking, or measurement systems for AI Strong understanding of modern LLMs, AI agents, and the engineering systems required to deploy them Experience building systems that optimize across multiple dimensions, including quality, latency Excellent software engineering fundamentals and a passion for building production-quality systems.

🎁 Benefits Comprehensive Healthcare Benefits and Income Replacement Programs 401k with Employer Match Global Benefit programs that fit your lifestyle, from workspace to professional development to Family Planning Support Gender-Affirming Care Mental Health & Coaching Benefits

Source listing: empllo_remote