Staff Software Engineer, Ray Data
Anyscale · San Francisco
About this role
**About Anyscale** Anyscale (https://www.anyscale.com/) is on a mission to democratize distributed computing and make it accessible to software developers of all skill levels. We’re commercializing **Ray** (https://docs.ray.io/en/latest/), an open-source project powering an ecosystem of libraries for scalable machine learning. With Anyscale, we’re building the best place to run Ray—so developers and data scientists can scale ML applications from laptop to cluster without needing to be distributed-systems experts. **About Ray Data Team** **Ray Data** (https://docs.ray.io/en/latest/data/data.html) is a Python-native data processing engine and a one-stop shop for AI data processing needs. The team builds and maintains Ray Data, developing the underlying distributed data processing infrastructure that powers modern AI workloads—especially for increasingly complex tasks like multimodal data processing and large-scale batch inference. Learn more: *Architecting Multimodal Data Pipelines That Scale with Ray* (https://www.anyscale.com/blog/architecting-multimodal-data-pipelines-that-scale-with-ray) --- ## What you’ll do - Design, build, and improve the core systems that power Ray Data, with a focus on **performance, scalability, and reliability**. - Design and optimize **distributed execution** across different stages of data pipelines in **heterogeneous environments**. - Build data loading and processing solutions for **production training and inference** workloads. - Solve challenging problems in **distributed execution, scheduling, resource management, data partitioning, fault tolerance, and performance optimization**. - Make system-level architectural decisions and reason through tradeoffs (e.g., resource allocation, execution models, batch vs. streaming, consistency vs. availability). - Work with customers and AI-native companies to understand and solve challenges in scaling their AI workloads. --- ## What we’re looking for - **6+ years** building production-grade software, infrastructure, or developer-facing systems, with strong **Python** experience. - **6+ years** personally owning core architectural decisions within a distributed data or compute engine (not primarily operating/using someone else’s platform). - Deep experience with distributed systems internals (e.g., scheduling, fault tolerance, data partitioning, distributed execution, performance optimization, or database/query engine internals). - A track record of reasoning through system-level tradeoffs and defending architectural decisions (e.g., batch vs. streaming, static vs. dynamic resource allocation, consistency vs. availability). - Passion for solving unsolved problems in large-scale AI infrastructure and building systems that enable the next generation of AI applications. --- ## Why Anyscale - Mission-driven: make scalable computing effortless. - Ray is in production at companies like OpenAI, Uber, Spotify, Instacart, and Cruise. - Backed by Andreessen Horowitz, NEA, and Addition; **$250M+ raised**. - Partnerships with **Azure, CoreWeave, and Google Cloud** to bring AI-native compute into enterprise environments. - Competitive salary and equity; health/dental/vision coverage (many plans up to 99% employer-covered). - Flexible time off, paid parental leave, and mental health support. --- *Anyscale Inc. is an Equal Opportunity Employer. Candidates are evaluated without regard to age, race, color, religion, sex, disability, national origin, sexual orientation, vete
Listing freshness
CronJobs last confirmed this listing 16h ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.