Senior Software Engineer, AI Gateway
Docker · United States
About this role
## Senior Software Engineer, AI Gateway ### About Docker Docker is a trusted developer tooling brand used by **20M+ monthly users** and **20B+ container image pulls**. Docker supports developers across products like **Docker Desktop**, **Docker Hub**, and **Docker Scout**. The team is **globally distributed and remote-first**, building tools that define how software is built and delivered. As AI agents reshape software development, Docker is at the center of the shift—providing **sandboxed environments**, **verified images**, and **secure infrastructure** to make autonomous workflows trustworthy by default. ### About Docker AI Gateway Docker AI Gateway helps take AI agents from **prototype to production**. It’s a secure control point that removes toil from **model orchestration**, **tool management**, **observability**, and **governance**, so developers can focus on building AI agents—not gluing infrastructure. **What the Gateway offers:** - **Model and tool routing** with built-in **security** and **cost optimization** - **OpenAI-compatible interface** and an **MCP server** - **Unified observability** and **policy enforcement** - **Auto-RAG**, tool injection, session summarization, and more ### What you’ll do - Design and implement core systems powering the AI Gateway, including the **model router**, **MCP gateway**, and **control plane** - Build infrastructure for **dynamic model selection/routing**, **auto-failover**, **cost-based routing**, and **policy enforcement** - Own critical capabilities such as **secure credential storage**, **session summarization**, **caching**, and **rate limiting** - Develop **APIs** for developers using **OpenAI-compatible interfaces** and the **Model Context Protocol** - Build infrastructure to support **evaluation**, **telemetry**, **replay**, and **backtesting** for agents and LLM workflows - Lead architectural decisions and **mentor engineers** as the team scales - Perform **performance tuning** (caching, session summarization, rate limiting) to keep the gateway fast under real production load - Contribute to **roadmap planning**, **technical strategy**, and cross-functional alignment ### Key problems you’ll help solve - Create a unified abstraction over different model/tool providers without leaking differences into the developer experience - Make **credential and identity handling** for tool access both **airtight** and **fast** - Provide developers **real-time and historical visibility** into what their agents are doing, with **policy enforcement that can’t be bypassed** - Maintain **low latency** and **high reliability** as traffic scales from early adopters to production fleets ### Who you are **Required Qualifications:** - **6+ years** of backend engineering experience with production-grade systems - Deep knowledge of **distributed**, **highly scalable**, cloud-native systems, and **API design** - Experience building **secure, high-throughput services** (e.g., gateways, proxies, load balancers, policy engines) - Fluency in **Go** and/or **Rust** (both preferred) - Familiarity with **AI/ML platforms** or **model serving infrastructure** - Strong **product mindset** and excitement for building developer-facing tools *Note: The provided description appears truncated after “Ownership...”.*
Listing freshness
CronJobs last confirmed this listing 3h ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.