Senior Software Engineer, ML API – AI Inference
Cerebras · Sunnyvale, CA
About this role
## Senior Software Engineer, ML API – AI Inference ### About the role Cerebras Systems is building a new generation of disaggregated AI inference systems that combine **GPU-accelerated prefill** with **ultra-fast decode** on the **Cerebras Wafer-Scale Engine**. As a **Senior Software Engineer**, you’ll own and evolve the **ML API layer** that makes this heterogeneous serving system accessible, reliable, and easy to use. You’ll deliver consistent behavior across models and accelerator backends, translating evolving model capabilities and customer requirements into clear designs and production-ready implementations. This is a hands-on individual-contributor role with opportunities to lead small project teams, mentor engineers, and improve the quality and reliability of production inference software. ### Responsibilities - **Own features end to end**: design, implement, test, and operate capabilities for chat completions, text generation, streaming, model configuration, tool calling, structured outputs, multimodal inputs, and emerging inference features. - **Lead multi-component delivery**: write design docs, evaluate tradeoffs, break work into milestones, and coordinate implementation with a small group of engineers and partner teams. - **Deliver consistent API behavior** across GPU prefill, Cerebras decode, and other inference backends; maintain compatibility with widely adopted inference interfaces. - **Enable new models and capabilities**: integrate foundation models, tokenizers, chat templates, generation configuration, sampling methods, and model-specific features. - **Integrate serving components** with frameworks/runtimes such as **vLLM, PyTorch, Hugging Face, LLGuidance**, and Cerebras runtime components. - **Support disaggregated inference**: routing, state-transfer integration, error handling, retries, cancellation, and request lifecycle management. - **Deliver measurable performance improvements** (e.g., time to first token, streaming latency, throughput, resource efficiency) via tokenization, serialization, batching, scheduling, and communication improvements. - **Own correctness and release quality** with conformance tests, model-validation suites, numerical comparisons, integration tests, and performance benchmarks. - **Strengthen production reliability**: improve metrics, tracing, logging, alerts, diagnostic tools; investigate incidents and drive resilience improvements. - **Improve customer/developer experience**: partner with product/support/solutions to refine configuration, documentation, examples, and debugging workflows. - **Raise the engineering quality bar** through thoughtful reviews, mentorship, and removing delivery roadblocks. ### Minimum qualifications - **5+ years** of software engineering experience with independent ownership of production services, distributed systems, or substantial software components. - Strong **Python** skills and proficiency in **Go, C++, Rust, or another production-service language**. - Experience designing and delivering features spanning multiple components (design docs, tradeoffs, coordination). - Experience building/evolving **production APIs** (validation, error handling, compatibility, testing). Familiarity with **REST, gRPC, or streaming protocols**. - Experience integrating across service/framework/runtime/infrastructure boundaries. - Experience operating services on **Linux** using **containers** and **Kubernetes (or similar)**, plus CI/CD. - Ability to diagnose correctne
Listing freshness
CronJobs last confirmed this listing 1h ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.