Engineering Manager, Inference Router
Cerebras · Sunnyvale, CA
About this role
Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. This architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud inference services. This order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real-time iteration and increasing intelligence via additional agentic computation. Cerebras works with the leading model labs, global enterprises, and cutting-edge AI-native startups. OpenAI recently announced a multi-year partnership https://openai.com/index/cerebras-partnership/ with Cerebras, to deploy 750 megawatts of scale, transforming key workloads with ultra high-speed inference. ABOUT THE ROLE The Inference Router team at Cerebras builds the systems that decide where every inference request runs. We route traffic across all of Cerebras’ data centers and third-party integrations, balancing heterogeneous compute, maximizing cache utilization, and delivering the industry’s lowest time-to-first-token (TTFT) at massive scale. As an Engineering Manager, Inference Router, you will lead the team responsible for the intelligent control plane of the fastest generative AI inference platform in the world. Your team owns global request routing, prefill/decode (P/D) disaggregation, prefix-aware routing for high cache hit rates, autoscaling, and the high-availability infrastructure that keeps the platform fast and reliable under real-time, bursty load. This role operates at the intersection of distributed systems, Kubernetes-based infrastructure, routing-algorithm design, and machine learning. It combines technical leadership, people management, and execution ownership, with direct impact on the latency, throughput, and reliability experienced by every Cerebras inference customer. WHAT YOU’LL DO TECHNICAL LEADERSHIP - Own the architecture and evolution of the inference routing layer across all Cerebras data centers and third-party inference integrations. - Guide the design of: - global, multi-region request routing and load balancing; - routing for heterogeneous compute, including prefill/decode (P/D) disaggregated serving; - prefix-aware / cache-aware routing to maximize KV-cache reuse and utilization; - low-TTFT, high-throughput request scheduling under concurrent, bursty workloads. - Drive the routing algorithms that trade off cache locality, queueing, load, and latency in real time. - Invest in simulation: build high-fidelity models of the fleet and its workloads so the team can validate routing policies, failover behavior, and scaling decisions under a wide range of conditions without tying up precious accelerator capacity. - Partner with inference runtime, cloud/platform, hardware, and ML teams to optimize end-to-end performance. TEAM LEADERSHIP - Build, manage, and grow a team of distributed systems, infrastructure, and ML systems engineers. - Provide technical direction, mentorship, and career development. - Foster a culture of ownership, velocity, and operational excellence. - Recruit top talent in distributed systems, Kubernetes/infrastructure, and routing/ML systems. EXECUTION & DELIVERY - Drive execution of complex, cross-functional initiatives spanning inference runtime, cloud/infrastructure, and ML teams. - Own delivery of capabilities such as: - prefix- and cache-aware routing policies; - P/D-disaggregation-aware scheduling and placement; - autoscaling and capacity management...
Listing freshness
CronJobs last confirmed this listing 57m ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.