Software Engineer - ML Infrastructure
Epsilon Health · San Francisco, CA
About this role
## Software Engineer — ML Infrastructure ### About Epsilon Health Epsilon Health is transforming medical imaging and diagnostics by bringing frontier AI into clinical practice. We’re tackling critical healthcare challenges with technology that improves patient outcomes. This year, we’ve served **300,000+ patients** and now serve **3,000+ daily**, working with leading medical institutions across the U.S. Our research team brings experience from **DeepMind** and **Meta**, powered by a comprehensive proprietary medical imaging dataset enriched with real-world clinical data. ### Role Overview We’re looking for an **ML Infrastructure Engineer** to design and build the core systems that enable **scalable, efficient training** of large models for both **deployment and research**. You’ll help make experimentation and training **fast and reliable**, so research teams can focus on science—not system bottlenecks. You’ll sit on the **Engineering team**, working closely with research to own: - **Distributed training** and **reinforcement learning** infrastructure - **Inference and evaluation** systems that move models from experimentation into production ### Key Responsibilities - Partner directly with researchers to understand their workflows and anticipate how needs will evolve - Build distributed training infrastructure for foundation models on large-scale medical imaging, including **long-context parallelism** and **checkpointing** for volumetric CT/MR training - Create high-throughput data loading and preprocessing to keep GPUs saturated on large volumetric and multimodal datasets - Prototype new ideas with researchers and translate them into production-ready code (end-to-end delivery: experimentation → deployment → monitoring) - Contribute to production serving and deployment pipelines (model rollout, canary deployments, monitoring) alongside the backend team - Build the reinforcement learning training stack, including: - high-throughput rollout generation - reward-model serving - experience collection - enabling **online, multi-reward RL at scale** ### Qualifications - **6+ years** designing, building, and operating large-scale distributed systems/infrastructure in production - **2+ years** building ML infrastructure or systems in production - Strong **Python** skills; expertise in **PyTorch or JAX** - Experience with compute/tooling/workflow needs of large-scale ML research - Experience building infrastructure/platforms for research or ML workflows - Deep experience building and operating **Kubernetes** and cloud infrastructure at scale - Experience with distributed training at scale (e.g., **FSDP, DeepSpeed, Megatron-style parallelism**) and keeping large GPU jobs efficient - Prior experience as a technical lead or mentor ### Preferred Qualifications - Experience in a startup/startup-like environment (small, fast-moving team with high autonomy) - Experience building RL training infrastructure (rollout generation, reward-model serving, online/off-policy learning) - Experience with high-performance inference/serving (e.g., **vLLM, SGLang, TensorRT, Triton**) for rollouts and production - Experience optimizing inference/serving for large models (batching, KV/prompt caching, quantization, low-latency/high-throughput sampling) - Experience optimizing training performance (parallelism, distributed communication, mixed/low precision, utilization) - Experience building internal training/experimentation platforms for research teams (A/
Listing freshness
CronJobs last confirmed this listing 3h ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.