Software Engineer - GPU Networking & Distributed Systems
Baseten · San Francisco
About this role
**Software Engineer - GPU Networking & Distributed Systems** **About Baseten** Baseten powers mission-critical inference for leading AI companies like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer. Recently raised $1.5B Series F led by Altimeter Capital, Conviction Partners, and Spark Capital. **The Role** Lead GPU networking efforts to build the global operating system for distributed, heterogeneous AI hardware. Make RDMA a first-class building block and unlock next-generation distributed inference optimizations. Work at the convergence of networking and compute, architecting the software fabric that unifies thousands of GPUs. **Key Responsibilities** • Integrate RDMA/RoCE/InfiniBand capabilities into inference stack • Optimize distributed inference for Disaggregated KV Cache Offload and WideEP • Enable sub-10-second startup for trillion-parameter models • Characterize and validate networking performance on bleeding-edge clusters (H100/H200, B200/B300, GB200/300 NVL72) • Design observability tools for packet flow, congestion, and bandwidth visualization • Optimize communication kernels (NCCL, NVSHMEM) to overlap compute and data transfer **Requirements** • Deep experience with high-performance networking protocols (InfiniBand, RoCE v2) • Fluency in C++ or Python with understanding of NVIDIA architecture memory hierarchy • Willingness to dive deep into source code and debug complex distributed systems • Judgment on when to build custom solutions vs. use off-the-shelf tools **Nice to Have** • NCCL, NVSHMEM, and UCX expertise • Rust experience for systems-level networking code • GPUDirect Storage (GDS) or high-performance filesystem experience • Familiarity with TensorRT-LLM, vLLM, or SGLang • Hardware qualification and benchmarking experience **Benefits** • Competitive compensation with meaningful equity • 100% medical, dental, vision coverage (U.S.) • Flexible PTO + company-wide Winter Break • Paid parental leave & fertility stipend • 401(k) facilitation (U.S.) **Equal Opportunity Employer**
Listing freshness
CronJobs last confirmed this listing 1h ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.