AI Resident
Ema · San Francisco Bay Area
About this role
**AI Resident — Ema (Agentic AI Residency)** **About Ema** Ema builds **AI Employees** for **HR, IT, and Finance**—handling busy work across the employee experience (recruiting, onboarding, benefits, IT support, invoice processing, and payroll) so people can focus on what needs them. Founded by former executives from **Google, Coinbase, Flipkart, and Okta**, and backed by investors including **Accel** and **Naspers/Prosus**. Headquartered in **Silicon Valley** with offices in **London, Bangalore, and Vancouver**. --- ## The Residency You’ll own **one hard problem end-to-end**: write the proposal, build the system, design the evaluation, ship behind a gate, and finish with a write-up of what was true—including what didn’t work. You’ll work in the **production codebase** with a **senior mentor** and **real production data**. Projects are **scoped with you**, not handed to you. Recent residents have shipped: - Self-improving harnesses - Inference-cost work - Agent memory - Evaluation infrastructure --- ## Problem Space (the loop we care about) **Production traces → data → training & evaluation → better agents → better traces.** Projects may live anywhere in that loop: ### 1) Harness & inference-time work - Context engineering - Tool/skill design - Orchestration - Deciding where extra inference compute pays - Self-improvement loops behind hard fences ### 2) Post-training for agents - SFT on curated trajectories - Preference optimization - RL on real agent tasks - Reward design (verifiable outcomes) - Process vs. outcome supervision - Distilling frontier behavior into cheaper models ### 3) Environments & rewards - Turning enterprise workflows into training/eval environments - Fixture tenants, simulated users (including impatient users) - Verifiable rewards - Defenses against reward hacking - Agents exploiting a lazy grader (and how to prevent it) ### 4) Data engines - Mining production agent-steps into training/eval corpora - Failure mining - Labeling with calibrated judges - Synthetic augmentation that stays useful ### 5) Evaluation - Behavior-level benchmarks from real workflows - LLM judges calibrated against human labels - Reliability statistics for stochastic agents ### 6) Efficiency - Routing, ensembles, caching - Small-model specialization - **Quality per dollar** as a research metric --- ## What We’re Looking For - **No specific degree required**. Strong undergrads, grad students, and self-taught builders welcome—what matters is demonstrated depth in **ML or agent systems**. - **Solid ML fundamentals** and **strong engineering**: Python, PyTorch, and the discipline to ship in a large production codebase. - Real depth in **at least one** area: - Post-training (SFT/DPO/GRPO-family RL) - Reward modeling or LLM judges - Agent/tool-use systems - Retrieval and memory - Eval design - (One area you can teach us beats five you’ve touched.) - **Statistical literacy**: you can size experiments and understand that ~25 samples at one seed is a datapoint, not a result. - **Honest measurement as a habit**: you’d rather kill your own feature with a clean experiment than ship on a hunch. --- ## Nice to Have - Hands-on post-training with open models (TRL, veRL, OpenRLHF, or your own loop) - Debugged a reward-hacked run - Built/trained in interactive agent environments (SWE, web, or tool-use gyms) - Trace analysis, data curation, or synthetic data work at scale - Serving/efficiency experience (vLLM/SGLang, distillation, qu
Listing freshness
CronJobs last confirmed this listing 2h ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.