HPC Infrastructure Engineer - GPU Clusters
ElevenLabs · United States
About this role
**HPC Infrastructure Engineer - GPU Clusters** **About the Role** Own the GPU cluster infrastructure that powers every model trained at ElevenLabs. Operate NVIDIA GPU clusters across bare metal and rented capacity, ensuring compute is fast, reliable, and seamless for researchers. This is a builder-operator role with real breadth and autonomy on a lean team. **What You'll Be Doing** • Operate and improve GPU fleet end-to-end: provisioning, scheduling, monitoring, upgrades, capacity planning • Build automation for fleet health: node checks, automated remediation, burn-in pipelines • Own the stack beneath training code: OS images, NVIDIA drivers, CUDA, container runtimes, NCCL, high-speed networking • Run and tune job scheduling (Slurm or similar) for fair and fast researcher access • Build and maintain high-performance storage for datasets and checkpoints • Hunt down and fix performance problems: stragglers, degraded links, thermal issues, flaky GPUs • Evaluate rented GPU capacity and hold providers to SLAs • Hands-on hardware work: racking, cabling, diagnostics, vendor coordination • Maintain cluster security: access control, network isolation, secrets management **Requirements** • Production experience running large-scale Linux server or GPU environments • Strong knowledge of NVIDIA stack (drivers, CUDA, NCCL, DCGM) or deep systems expertise • Comfortable with bare-metal environments, server hardware, and high-speed networking • Solid automation skills in Python and/or Bash; IaC tools like Ansible or Terraform • Ability to dig into metrics, logs, and PromQL to diagnose issues • Ownership mindset and willingness to tackle any task **Nice to Have** • ML training workload support experience • GPU cloud provider evaluation experience • Parallel filesystems (WEKA, VAST) or large-scale object storage • BMC/IPMI/Redfish automation and PXE provisioning • Power and cooling knowledge for dense GPU deployments **Location** Remote, global. Optional offices in London, New York, San Francisco, and Warsaw. **About ElevenLabs** AI research and product company with $11B valuation, backed by Andreessen Horowitz, ICONIQ Growth, and Sequoia. Serving millions of users and thousands of businesses globally.
Listing freshness
CronJobs last confirmed this listing 19h ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.