ML Inference Engineer â Terminal-Bench Task Designer (vLLM / SGLang / TensorRT-LLM)
Toptal · Anywhere in the World
About this role
## Overview We are building **Terminal-Bench**, a benchmark that tests how well AI agents handle real expert work in a terminal. We’re hiring **ML inference and serving experts** to turn real, paid tasks from your experience into **expert-level Terminal-Bench challenges**. > **Task-based compensation model.** Please check the application questions for more details. ## Responsibilities - Design **expert-level Terminal-Bench tasks** that turn real, paid ML inference work from your experience into challenges an AI agent must solve end-to-end. - Define the **files, systems, and environments** each task runs on (e.g., model serving with streaming) so the challenge is realistic, reproducible, and command-line solvable. - Write a **working reference solution** for every task and verify it runs end-to-end, sized for **~60–75 minutes** of expert effort. - Build **automated checks** that objectively prove correctness, including cases where ML outputs fail silently (e.g., numerical drift or metric bugs). - Package tasks with **Docker and Git** for reproducibility, and document systems/tools/data/rules so reviewers can validate difficulty. - Iterate based on **reviewer feedback** and agent test runs—adjust difficulty, fix ambiguities, and keep tasks **expert-level, fair, and objectively gradable**. ## Skills / Experience - **3+ years** hands-on ML inference/serving work (not only studying/selling/managing), **or** a PhD with active research in the subfield. - Did this work in the **last 3 years** and can name the systems, tools, data, and rules used. - Trained/served/evaluated models at **real scale** (multi-GPU or production serving), not only notebooks or API calls. - Hands-on with **PyTorch or JAX internals** and serving stacks such as **vLLM, SGLang, or TensorRT-LLM**. - Can explain how ML results go wrong without errors (numerical drift, metric bugs) and how to catch them. - Technical fluency in **Python** and the **command line**, including working with files (CSV/JSON), APIs, or databases. - Availability to start by **early November 2026** and complete tasks of **~60–75 minutes** each. ## Nice-to-have - AI lab or LLM infrastructure experience - Contributions to **PyTorch**, **vLLM**, or **SGLang** - GPU profiling experience - **Docker** and **Git** proficiency - Experience writing automated tests ## Process - **Task vetting instead of an interview:** complete a vetting task so the client can assess your task design. - **Paid per task:** each task takes about **60–75 minutes**. - Work is delivered **task-by-task**, with reviewer feedback on submissions. - Expected start: **early November 2026**. ## Application Notes Please mention the word **ASTONISHED** and tag **RMmEwNjo5OGMwOjM2MDA6OjEwMw==** when applying to show you read the job post completely (#RMmEwNjo5OGMwOjM2MDA6OjEwMw==). This is a beta feature to avoid spam applicants—companies can search these words to find applicants that read this and see they’re human.
Listing freshness
CronJobs last confirmed this listing 56m ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.