Software Engineer - Model Products
Baseten · San Francisco
About this role
**Software Engineer - Model Products** **About Baseten** Baseten powers mission-critical inference for leading AI companies like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer. We recently raised our $1.5B Series F and are building the platform engineers use to ship AI products. **The Role** Join our Model Performance team to design and operate Model APIs—the infrastructure powering hosted API endpoints for open-source models. You'll work on distributed systems, model serving, and developer experience at the intersection of product, performance, and infrastructure. **Key Responsibilities** • Design and build Model APIs with advanced inference capabilities (structured outputs, tool calling, multi-modal serving) • Profile and optimize TensorRT-LLM kernels, analyze CUDA performance, and implement custom operators • Productionize performance improvements (speculative decoding, quantization, batching, KV-cache reuse) • Build comprehensive benchmarking frameworks across model architectures and hardware configurations • Implement platform fundamentals: API versioning, validation, metering, quotas, and authentication • Instrument observability and collaborate across teams for robust model serving **Requirements** • 3+ years building and operating distributed systems or large-scale APIs • Proven track record with low-latency, reliable backend services • Strong profiling, tracing, and performance debugging skills • Clear written communication and design documentation abilities • ML/LLM experience not required; strong systems engineering with performance instincts preferred **Nice to Have** • Experience with LLM runtimes (vLLM, SGLang, TensorRT-LLM) • Kubernetes, service meshes, or distributed scheduling knowledge • Developer-facing infrastructure or open-source API background **Benefits** Competitive compensation with equity • Full medical/dental/vision coverage • Flexible PTO + Winter Break • Paid parental leave • Fertility stipend • 401(k) • Networking with ML startups *Equal Opportunity Employer*
Listing freshness
CronJobs last confirmed this listing 1h ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.