Member of Technical Staff, Enterprise Evals Platform
Mercor · San Francisco
About this role
**Member of Technical Staff, Enterprise Evals Platform** **About Mercor** Mercor organizes human intelligence to power the AI economy. We're a leading AI data company building the layer between human expertise and frontier models. Mercor Enterprise brings infrastructure to Fortune 500 companies, helping them capture how their best people work and translate that expertise into agents. **About the Role** Enterprise agents are complex systems requiring reliable and economically viable evaluation. You'll decompose real work, encode expert standards so agents can't shortcut them, and build the platform infrastructure that scales evaluation across customers, domains, and tasks. **Key Responsibilities** • Define golden sets and encode expert quality bars • Build verifiers over agent trajectories and outputs • Build eval platform for offline environments, task suites, and grading at scale • Run loss analysis and turn failure modes into regression tests • Own rollout gates for agent change decisions • Partner with Enterprise Platform and Applied AI teams **What We're Looking For** • Professional, academic, or research experience in agent engineering and evaluation • Experience building evaluation suites for LLM/agent systems • Judgment about task and rubric design with measurable improvements • Strong software engineering fundamentals • Bonus: Harbor and RL environment experience **Why Mercor** • Impact: Set the bar every enterprise agent must clear • Learning: Eval work spanning frontier models and live deployments • Growth: Research and systems in one role with fast paths to ownership **Benefits** Up to $15k relocation • $10k housing bonus • $1.5k monthly meals • Generous equity • Free Equinox • $200 laundry reimbursement • $200 wellness reimbursement • Health/Dental/Vision **Location:** San Francisco, NYC, or London (in-person 5 days/week)
Listing freshness
CronJobs last confirmed this listing 17h ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.