Member of Technical Staff (Software Engineer, Enterprise Agent Evals & Efficiency)
Perplexity · San Francisco
About this role
## About the Role The Enterprise Agent Evals & Efficiency team makes Perplexity's agents better at completing real work across enterprise connectors, tools, and data. You’ll help agents choose the right tools, use the right context, and complete complex workflows reliably—with less time and compute. This is a hands-on engineering role with end-to-end ownership: from building realistic enterprise evaluation environments to shipping improvements across the agent harness, tools, and skills. ## Key Responsibilities - Build realistic enterprise evaluation environments spanning connected applications, representative datasets, user roles, and permissions, with complex multi-step workflows and reproducible system state. - Own evaluation infrastructure, task suites, and automated graders that measure agent performance against human judgment and end-to-end outcomes. - Analyze agent trajectories and production failures, identify root causes, and ship improvements across the agent harness, tools, and skills to measurably improve task success and reliability. - Develop reusable skills and improve tool descriptions, schemas, and result handling so agents can select the right tools, execute workflows correctly, and recover from errors with fewer unnecessary calls. - Build and evaluate model-selection and routing strategies that match each task to the right model and effort level, balancing quality, latency, reliability, and cost. - Improve agent efficiency through context management, skill loading, caching, and execution strategies that reduce wasted tokens and cost per successful task without compromising quality or safety. - Build regression gates and launch-readiness signals for model, connector, tool, skill, and harness changes, and validate that evaluation gains translate into better production outcomes. ## Qualifications - 5+ years of software engineering or machine learning engineering experience, with a track record of owning complex production systems from design through deployment and operation. - Strong system design skills, including building reliable, scalable infrastructure for experimentation, evaluation, or data-intensive applications. - Proficiency in a backend language such as Python, Go, or Rust, and the ability to work effectively across a multi-language production stack. - Hands-on experience building, evaluating, or improving LLM-based applications or agents, including tool use and multi-step workflows. - Deep expertise in at least one area: - Evaluation infrastructure, benchmark design, and automated grading - Agent harnesses, tool interfaces, and reusable skills for complex workflows - Realistic test environments or simulations spanning multiple systems, datasets, permissions, and stateful workflows - Model selection, routing, and inference efficiency (context management and caching) - Strong experimental and statistical judgment: choosing representative tasks, defining meaningful success criteria, diagnosing failures, and distinguishing real improvements from noise. - Sound technical judgment balancing quality, latency, reliability, and cost, with a track record of translating evaluation findings into shipped improvements. - Experience leading ambiguous, cross-team initiatives from architecture through production adoption, setting technical direction, and mentoring to raise engineering quality. *If you’re excited about this role, we encourage you to apply even if you don’t meet every qualification. We value demonst
Listing freshness
CronJobs last confirmed this listing 51m ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.