Staff Engineer, Crusoe Managed Data Transfer (Distributed Systems)
Crusoe · San Francisco, CA - US
About this role
**Staff Engineer — Crusoe Managed Data Transfer (Distributed Systems)** Crusoe is building the future of AI infrastructure—accelerating the abundance of energy and intelligence. We own and operate each layer of the stack (from electrons to tokens) to power the world’s most ambitious AI workloads. We’re looking for a **Staff Engineer** to serve as the **technical lead and systems architect** for our **Managed Data Transfer (MDT)** service. --- ## About the Role MDT is a managed service that moves **petabyte-scale training data** between hyperscaler storage and **Crusoe storage**, so AI labs and Physical AI companies can train on Crusoe GPUs **without building their own transfer pipelines**. You will: - Design the **control plane**, **transfer worker fleet**, and **integration** with Crusoe storage and network - Own how MDT fits into **Crusoe Cloud** and set the path for adding **other clouds** and **Crusoe-internal transfers** - Lead architecture through **Crusoe architecture review** - Mentor engineers building the system and report to the **Engineering Manager** for Managed Platform Services MDT starts from a working prototype already serving customers in production—so your early decisions will focus on what to **keep, harden, or replace**, and how to sequence changes **without disrupting customers**. --- ## What You’ll Be Working On - **Architecture Ownership:** End-to-end design of MDT (job API & scheduler, Kubernetes transfer workers, storage integration, network paths) - **Prototype Assessment & Migration:** Refactor/extend/replace parts of the prototype and migrate users without interruption - **Throughput & Correctness:** Data-plane strategy (sharding, concurrency, straggler handling, checksum verification) and benchmarks targeting **line-rate transfers** over dedicated interconnects - **Cross-Team Contracts:** Define interfaces between MDT and Storage, Network, Kubernetes, IAM, Console, Billing, and Monitoring - **Extensibility & Build-vs-Buy:** Pluggable transfer engine + regional worker pools; evaluate open-source/vendor options - **Operational Excellence:** Reliability, observability, and on-call standards - **Product Partnership:** Scope, pricing trade-offs, milestones, and early surfacing of edge cases - **Technical Mentorship:** Design reviews, RFC feedback, and direct coaching to scale the team --- ## What You’ll Bring to the Team - **Distributed Systems Depth:** Hands-on experience designing/operating data-intensive distributed systems at scale - **Professional Experience:** **8+ years** shipping production software; led a service from prototype to GA - **Data Movement & Storage Performance:** Object storage semantics, file systems, TCP throughput; ability to profile and fix slow transfers - **Systems Integration Judgment:** Evolve a live system while keeping risk low for current customers - **Cross-Team Influence:** Define API boundaries and schema contracts; align with partner teams - **Operational Ownership:** Customer-facing on-call experience; build maintainable, observable systems - **Software Engineering Craft:** Proficiency in **Go** (strongly preferred) or another modern compiled language - **Communication:** Explain architectural decisions to engineers, PMs, and EMs --- ## Bonus Points - Bulk data transfer systems at petabyte scale (e.g., AWS DataSync, Google Storage Transfer Service, rclone) - Cloud networking/interconnects (e.g., AWS Direct Connect, GCP Interconnect) - Cross-account cloud IAM + th
Listing freshness
CronJobs last confirmed this listing 58m ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.