Senior Software Engineer, Crusoe Managed Data Transfer
Crusoe · San Francisco, CA - US
About this role
## Senior Software Engineer — Crusoe Managed Data Transfer (MDT) Crusoe is building the future of AI infrastructure with an energy-first approach—accelerating the abundance of energy and intelligence. We own and operate each layer of the stack, from electrons to tokens, to power the world’s most ambitious AI workloads. ### About the Role Crusoe’s Managed Platform Services (MAPS) organization builds managed services that make Crusoe Cloud easy to adopt and hard to leave (including Container Registry, Cloud Monitoring, and Managed Logs). **Managed Data Transfer (MDT)** is its newest service. MDT moves **petabyte-scale training data** from AWS S3 (and later other clouds) into Crusoe storage using a **single API call**. The service shards work across a fleet of transfer workers, verifies every object, and requires the customer to run nothing. You’ll join a new team early and help take MDT from first commit to **preview launch** with design-partner customers—shaping the API and building the **MDT control plane** (job API, scheduler, and job state) that turns customer requests into coordinated, resumable transfers. ### What You’ll Be Working On - **Job API & Lifecycle:** Design and build the data-transfer API (create, list, get, cancel) with a provider-neutral model for sources/destinations. - **Scheduling & Job State:** Split bucket listings into units of work, schedule onto regional worker pools, and track progress, retries, and resume across failures. - **Correctness at Scale:** Make petabyte-scale jobs safe to retry/cancel with end-to-end checksum verification and clear errors. - **Cross-Cloud Identity:** Implement secure access using short-lived, least-privilege credentials (AWS role assumption first; other clouds next), integrated with Crusoe IAM and audit logging. - **Customer Surfaces:** Ship the CLI and, with partner teams, Terraform and Console experiences for creating/monitoring transfers. - **Operational Excellence:** Add metrics/logs/alerts, write runbooks, and participate in on-call. - **Cross-Team Collaboration:** Align interfaces with Storage, Network, IAM, and Cloud Console; partner with Product and Solutions Architects on early onboarding. ### What You’ll Bring to the Team - **Backend engineering craft:** Experience designing/shipping production backend services and APIs (ideally **Go**, with **gRPC or REST**), plus relational databases and Kubernetes. - **Professional experience:** **5+ years** building cloud infrastructure/platform services; experience taking a service from zero to general availability. - **Distributed systems judgment:** Comfort with long-running, failure-prone work (idempotency, retries/backoff, leases, checkpointing, partial failure). - **API design:** Ability to build public APIs customers can automate against without breaking changes. - **Security mindset:** Knowledge of authentication/authorization and least privilege; careful handling of customer credentials. - **Operational ownership:** Production service operation, on-call experience, and building dashboards/alerts. - **Collaboration & writing:** Work well across teams; design docs that settle decisions. - **Education:** Bachelor’s in CS/Engineering (or related) or equivalent experience. ### Bonus Points - Large-scale data movement experience (e.g., S3 multipart uploads, rclone, s5cmd, AWS DataSync, Google Storage Transfer Service) - Cross-cloud identity experience (AWS IAM/STS, GCP impersonation, Azure workload identity) - Storage systems
Listing freshness
CronJobs last confirmed this listing 1h ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.