Member of Technical Staff (Software Engineer, GPU Cluster Infrastructure)
Perplexity · San Francisco
About this role
Perplexity serves hundreds of millions of queries per month, each of which fans out into multiple AI inference requests in real time. Behind this is a large GPU fleet spread across several cloud providers. Our inference engineers and researchers build models while also managing networking, securing capacity, and operating the underlying GPU clusters. We’re looking for a dedicated platform team member to own this infrastructure and hide its complexity behind a unified, self-serve platform for running training and inference workloads. **Responsibilities** - **Build a self-serve compute platform:** Design and own systems that let inference engineers and researchers launch training jobs and operate inference services without managing GPU provisioning, cluster configuration, or provider-specific infrastructure. - **Operate the GPU fleet:** Own provisioning, lifecycle management, reliability, and capacity integration across providers—so teams can use compute consistently regardless of where it runs. - **Solve for GPU scarcity:** Build scheduling and placement logic to find available capacity across providers, pack it efficiently, and place workloads onto the right hardware under real constraints. - **Support two very different workloads:** Keep long-running distributed training jobs healthy while guaranteeing availability and latency for production inference services on the same fleet. - **Own Kubernetes for GPU orchestration:** Write operators and CRDs, and manage many clusters across providers so the platform behaves consistently everywhere. - **Make failure boring:** Build fault tolerance, autoscaling, and observability to keep the fleet utilized and allow workloads to survive node loss, provider hiccups, and capacity shifts without human intervention. - **Set technical direction across teams:** Partner with inference and cloud infrastructure engineers to turn operational constraints into a coherent platform architecture and roadmap. **Qualifications** We expect you to have real depth in most of these: - Deep **Kubernetes** experience (custom operators, CRDs, and multi-cluster federation—not just running `kubectl apply`). - Managed **GPU clusters at scale**: NVIDIA hardware, CUDA, and the networking that makes them fast (InfiniBand or RoCE). - Orchestrated compute across **multiple clouds** (CoreWeave, AWS, GCP, or similar) and understand how different they really are. - Strong **distributed systems** fundamentals: scheduling, resource allocation, and fault tolerance under load. - Write infrastructure and systems-level code in **Go, Rust, or C++**. - Experience supporting both **long-running training jobs** and **high-availability inference services**, and understanding why they stress infrastructure differently. - Own problems end-to-end and do well when the path forward isn’t already defined. **Additional experience we value** - Inference serving stacks: **vLLM, SGLang, or TensorRT-LLM**. - **Slurm** or other HPC schedulers. - GPU kernel work in **CUDA or Triton** (not required, but notable). - High-speed interconnects: **InfiniBand, RoCE, or RDMA** in production. - Observability for ML workloads: **Prometheus, Grafana, or Weights & Biases**.
Listing freshness
CronJobs last confirmed this listing 1h ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.