Machine Learning Research Scientist, Evaluations
Scale AI · San Francisco, CA; Seattle, WA; New York, NY
About this role
**Machine Learning Research Scientist, Evaluations** **About the Role:** Scale is seeking Research Scientists and Research Engineers with expertise in LLM post-training (SFT, RLHF, reward modeling) and evaluation. You'll join the evaluation pod within the GenAI Research Organization, focusing on building benchmarks and diagnosing model failure modes in text and multimodal modalities. **Key Responsibilities:** • Analyze model behavior to identify, characterize, and diagnose failure modes in frontier LLMs and Agents • Design and build benchmarks and evaluation methods for LLM capabilities in text and multimodal modalities • Apply post-training expertise to connect observed failures to data and training interventions • Publish research findings in top-tier AI conferences **Ideal Qualifications:** • Ph.D. or Master's degree in Computer Science, Machine Learning, AI, or related field • Deep understanding of deep learning, reinforcement learning, and large-scale model fine-tuning • Experience with RLHF, preference modeling, instruction tuning, and LLM evaluation/benchmark development • Published research at major conferences (NeurIPS, ICML, ICLR, ACL, EMNLP, CVPR, etc.) • Excellent written and verbal communication skills • Previous customer-facing experience a plus **Compensation:** Base salary range: $180,600 – $225,750 USD (San Francisco, New York, Seattle) Includes equity, comprehensive health/dental/vision coverage, retirement benefits, learning stipend, and generous PTO. **About Scale:** Scale develops reliable AI systems for the world's most important decisions, partnering with industry leaders and government agencies to build, deploy, and oversee impactful AI applications. Equal opportunity employer committed to inclusive workplace and reasonable accommodations.
Listing freshness
CronJobs last confirmed this listing 1d ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.