Senior Data Platform Engineer - Weights & Biases
CoreWeave · Livingston, NJ / New York, NY / San Francisco, CA / Sunnyvale, CA / Bellevue, WA / Remote - US
About this role
## CoreWeave — Senior Data Platform Engineer (Weights & Biases) ### About the role CoreWeave is building an end-to-end AI platform by combining its cloud infrastructure with the best-in-class tools from Weights & Biases. As a **Senior Data Platform Engineer**, you’ll design and operate the **enterprise data platform** that powers the full AI lifecycle. ### What you’ll do - **Design & implement scalable data systems and pipelines (30%)**: Architect, develop, test, and maintain distributed **batch and streaming** pipelines to ingest, transform, validate, and deliver structured/semi-structured data. - **Develop data transformation & orchestration frameworks (20%)**: Build and optimize workflows using tools like **dbt, Dagster, Airflow** (or comparable). Define standards for testing, deployment, dependency management, data quality, and observability. - **Improve data warehouse performance & reliability (15%)**: Design data models/schemas, partitioning, and storage patterns for large-scale analytical workloads. Diagnose and resolve performance, scalability, availability, and cost-efficiency issues in multi-petabyte environments. - **Build product telemetry & data instrumentation (15%)**: Partner with software teams to define event schemas and implement reliable logging/telemetry/instrumentation for web apps, backend services, and distributed systems. - **Develop data catalog & semantic-layer capabilities (10%)**: Create metadata, lineage, governance, and semantic modeling systems for consistent business definitions and secure self-service analytics. - **Support data-powered AI applications (5%)**: Establish data interfaces and platform foundations for internal AI use cases (e.g., automated data discovery, workflow assistance, growth analysis, churn-risk modeling, abuse detection). - **Operate & support the platform (5%)**: Monitor production systems, troubleshoot incidents, document operational procedures, and participate in on-call after onboarding. ### Minimum qualifications - Bachelor’s degree (or foreign equivalent) in a relevant technical/quantitative field. - **5+ years** progressive experience in software engineering, data engineering, data infrastructure, or related work. - Experience designing/implementing **production data pipelines** and **distributed data-processing** systems. - Experience with **dbt, Dagster, Airflow** (or comparable) for transformation/orchestration. - Experience designing **data models/schemas** and **storage strategies** for large-scale analytical databases/warehouses. - Programming experience in **Python, Go, Java, Scala**, or another general-purpose language used for production data systems. - Strong **SQL** skills, including database performance optimization, data-quality testing, and production monitoring. - Knowledge of software engineering practices (version control, automated testing, code review, CI/CD, deployment automation, technical documentation). - Knowledge of web telemetry/event instrumentation, logging systems, APIs, and distributed-service architecture. - Ability to communicate technical designs, risks, and tradeoffs to technical and business stakeholders. ### Preferred qualifications - Experience operating **multi-petabyte** or other high-volume, business-critical data platforms. - Experience with cloud infrastructure/data services on **AWS, GCP, or Azure**. - Experience with data catalogs, metadata management, lineage, governance frameworks, or semantic layers. - Knowledge of ML workflo
Listing freshness
CronJobs last confirmed this listing 1h ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.