Software Engineer - RL Environments
Afterquery · San Francisco
About this role
## About AfterQuery AfterQuery (https://www.afterquery.com/) is an applied research lab curating data solutions for foundation model development. We serve frontier AI labs with the mission of delivering the best data to power the best models. We’re based in San Francisco and backed by leading investors including Altos Ventures, BoxGroup, and Y Combinator—plus angels from Google DeepMind, OpenAI, Anthropic, Meta Superintelligence Labs, and Microsoft AI. ## Why Apply - **Massive opportunity**: YC’s fastest unicorn, valued at **$3.2B** - **Founding impact**: Own and architect core infrastructure powering the platform - **Equity & growth**: Competitive salary and meaningful equity - **Strong team**: Founding team experience from Citadel Securities, Meta, Google, Silver Lake, Morgan Stanley, and more ## Role Overview — Software Engineer (RL Environments) Design the **simulations, data, and evaluations** that directly influence how frontier models learn. You’ll work hands-on with research teams at top AI labs to: - Experiment with environment design and novel data creation strategies - Diagnose model failure modes - Develop metrics to determine whether models are truly improving - Move quickly from hypothesis to live experiments that feed into large-scale training runs You’ll design environments and tasks that expose meaningful failure modes across domains like **finance, code, and enterprise workflows**, and build/refine **reward signals** for RL pipelines. ## Responsibilities - Construct simulated worlds and explore data shapes that expose meaningful model failure modes across domains (finance, code, enterprise workflows) - Build and refine **evaluation rubrics** and **reward signals** for **RLHF** and **RLVR** pipelines - Analyze agent-produced trajectories and run experiments to improve model capabilities - Develop quantitative frameworks for measuring **dataset quality, diversity**, and downstream impact on **alignment** and **capability** - Create and manage **real-world & synthetic data pipelines** - Partner with lab research teams to translate training objectives into concrete data and evaluation specifications - Run post-training experiments and help scale training infrastructure ## Required Qualifications - Ability to design lightweight experiments, move fast, and extract actionable insights from messy results - Experience using **Docker** (or similar containerization tools) to design and monitor systems at scale - Strong familiarity with common **reinforcement learning** algorithms and methods, especially for **post-training LLMs** ## Preferred Qualifications - Plus if you’ve worked/interned at RL environment companies or AI safety/benchmarking orgs (e.g., **METR**, **Artificial Analysis**) - Former founders and early engineers at early-stage startups are a plus - Looking for people who work hard, learn fast, and care about getting the details right ## Benefits (for eligible employees) - Health Insurance: Medical, Vision, Dental - 401(k) with Employer Match - Daily Meals: Daily UberEats stipend - Monthly Wellness stipend - Commute covered ## Equal Opportunity AfterQuery is an equal opportunity employer committed to a workplace free from discrimination and harassment. Employment decisions are made without regard to legally protected characteristics. *Note: This job description is intended to describe the general nature and level of work and is not exhaustive. The company may modify it as business needs change.*
Listing freshness
CronJobs last confirmed this listing 2h ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.