Staff Software Engineer, Inference API
Cerebras · Sunnyvale, CA
About this role
## Staff Software Engineer — Inference API ### About Cerebras Cerebras Systems builds the world’s largest AI chip—an architecture designed to deliver industry-leading training and inference speeds (over 10× faster than GPU-based hyperscale cloud inference services). Cerebras works with leading model labs, global enterprises, and AI-native startups, including a recent multi-year partnership with OpenAI to deploy 750MW of scale for ultra high-speed inference. ### About the Role Cerebras is building disaggregated AI inference systems that combine **GPU-accelerated prefill** with **ultra-fast decode** on the **Cerebras Wafer-Scale Engine**. You’ll help build and evolve the **ML API layer** that makes this heterogeneous serving system accessible, reliable, and easy to use. This is a hands-on role at the intersection of **machine learning systems, API design, model serving, and distributed systems**. You’ll ensure consistent production behavior for features such as **streaming, sampling, tool use, structured outputs, multimodal inputs, and model configuration**. ### Responsibilities - **Build production ML inference APIs** for chat completions, text generation, streaming, model configuration, tool calling, structured outputs, multimodal inputs, and emerging inference capabilities. - **Deliver a unified serving experience** with consistent request/response semantics across GPU prefill, Cerebras decode, and other backends. - **Enable new models and capabilities** by integrating foundation models, tokenizers, prompt formats, sampling methods, attention variants, multimodal inputs, and model-specific features. - **Own API compatibility and evolution**: versioning, deprecation, validation, and backward compatibility. - **Integrate with model-serving runtimes** (e.g., vLLM, PyTorch, Hugging Face, AMD ROCm, and Cerebras runtime components). - **Support disaggregated inference**: coordinate GPU prefill with Cerebras decode (routing, state transfer, error handling, retries, lifecycle management). - **Improve serving performance**: streaming behavior, time-to-first-token, latency, throughput, batching, serialization, tokenization, scheduling, and inter-component communication. - **Ensure functional and numerical correctness** across tokenization, sampling, logits, precision changes, determinism, and model upgrades. - **Strengthen reliability and observability** with end-to-end indicators, structured logging, tracing, metrics, dashboards, health checks, and diagnostics. - **Develop testing and qualification infrastructure** (conformance tests, workload replay, validation suites, benchmarks, integration tests, release gates). - **Improve developer experience**: configuration, SDKs, documentation, examples, debugging tools, and self-service workflows. - **Collaborate across the stack** with compiler, runtime, kernel, cloud, product, and solutions teams. ### Minimum Qualifications - **5+ years** software engineering experience with substantial ownership of production software or distributed systems. - Strong programming in **Python and Go**, plus experience with **C++, Rust, or similar** for performance-sensitive/highly concurrent services. - Experience building stable APIs with validation, error handling, observability, compatibility, and versioning. - Experience integrating across service/framework/runtime/infrastructure boundaries. - Experience with **OpenAI-compatible**, **gRPC**, **REST**, or **streaming** inference APIs. - Experience with **Linu
Listing freshness
CronJobs last confirmed this listing 1h ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.