Technical Program Manager, Inference
Anthropic · San Francisco, CA | New York City, NY
About this role
## About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems—safe and beneficial for users and society. ## About the role Anthropic’s **Inference** organization serves Claude to millions of users and enterprise customers across Anthropic products and the world’s largest cloud platforms, delivering the speed, reliability, and efficiency frontier AI demands. This role focuses on scaling and improving the tightly coupled inference stack, including accelerator kernels, an in-house inference engine, distributed routing, autoscaling, and capacity management across **GPUs, TPUs, and Trainium**. You’ll be a **Technical Program Manager** driving programs that scale the serving fleet across four closely coupled areas: - **Scaling & infrastructure:** routing, autoscaling, capacity management, and bringing new capacity online - **Velocity:** toolchains, dev environments, and CI/CD to build/test/ship across accelerator platforms - **Inference engine:** software between accelerator kernels and routing that serves every request - **Performance:** improving throughput, latency, cost, and reliability (with correctness treated as part of performance) This is **not** a status-tracking role. You’ll own outcomes end to end—often where no single team is accountable—by partnering across Research, Safeguards, Security, Compute, Infrastructure, API, and cloud/silicon partners. ## Key responsibilities - Own programs that scale Claude’s serving fleet across accelerator and cloud platforms, from early direction through delivery - Drive new capacity online end to end across compute, infrastructure, security, networking, and inference teams - Build visibility into supply, demand, and utilization; drive trade-offs among utilization, cost, latency, and reliability to a clear decision - Lead reliability programs and large migrations without disrupting production traffic or research; ensure incident follow-ups land in team roadmaps - Improve inference engineer productivity by defining the right productivity metrics and removing bottlenecks in build/test/release across platforms - Coordinate roadmaps and priorities across production, research, and safeguards needs that depend on the same teams - Track performance/efficiency work from opportunity to measured impact in production, with correctness as a release criterion - Own inference readiness for model launches alongside launch TPMs ## Minimum qualifications - **7+ years** experience as a TPM (or equivalent engineering/SRE lead role) owning large multi-team infrastructure programs end to end in production distributed systems, ML infrastructure, or cloud platforms - Technical fluency in large-scale serving infrastructure: request routing/load balancing, autoscaling, capacity planning, cluster orchestration, deployment/rollout, and observability - Working understanding of how LLM inference uses hardware (prefill/decode, batching, caching, compute vs memory bandwidth vs interconnect) or a proven ability to ramp quickly in a new technical domain - Track record of building program mechanisms from scratch on ambiguous, fast-moving efforts—and retiring process that no longer helps - Ability to drive outcomes across organizations you don’t manage, with competing priorities and unclear decision ownership - Strong written communication (plans, reviews, executive updates) - Low ego, high ownership (willing to do what the program needs) - Care about Anthropic’s mission and serving fron
Listing freshness
CronJobs last confirmed this listing 1h ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.