Senior Software Engineer - Husky Storage
Datadog · Boston, Massachusetts, USA; New York, New York, USA
About this role
**Senior Software Engineer — Husky Storage (Datadog)** Husky is Datadog’s distributed, petabyte-scale columnar event store at the heart of the Event Platform, powering products such as Logs, RUM, APM, Cloud Network Monitoring, Netflow, and more. Husky is built to store and query massive volumes of event data at low cost, with data fully queryable within seconds of arrival. As a **Senior Software Engineer** on this team, you’ll work hands-on with a distributed storage system at petabyte scale and help evolve the platform to support the next level of scale for the AI era. You’ll move from a custom storage format toward open standards (**Parquet, Arrow, DataFusion**) to enable on-prem solutions, increasingly complex analytical/batch workloads, and AI observability—sustainably into the future. You’ll partner closely with **Husky Ingestion, Metadata, and Platform**, and increasingly with **Lakehouse** on the Iceberg-compatible boundary between Husky and analytical/batch workloads. You’ll also collaborate with the open source community to build reusable software solutions that can be operated at scale. --- ### What You’ll Do - Drive Husky adoption by migrating the storage format to open source **Parquet** and re-imagining **compaction** on a format-agnostic, **DataFusion-based** substrate - Deliver concrete cost-reduction work: block-level inverted index compression, combining the database with external objects at the row level, and storage tiering/caching - Scale the solution to enable customers to manage event versioning at scale using **temporal-database** techniques - Play an influential role in operating and deciding how to best operate the service going forward - Work across the write path (with Ingestion) and toward a cleaner **Iceberg-based** handoff to ADP/Lakehouse for batch/analytical workloads --- ### Who You Are - Enthusiastic and experienced with **large-scale distributed systems** and/or **storage engines** - Comfortable improving and operating large distributed systems that customers rely on for critical business functions - Strong fundamentals in **columnar/file storage formats** (Parquet, ORC, Arrow) or comparable systems **Bonus** - Experience with **Go, Java, or Rust** - Experience with compaction or compression/encoding schemes (RLE, dictionary, delta, gorilla) - Experience with object storage (**S3/GCS/Azure Blob**) at scale, including caching and cost optimization - Experience with **Kafka** and stream processing - Experience with **Kubernetes** --- ### Benefits and Growth - New hire stock equity (**RSUs**) and employee stock purchase plan (**ESPP**) - Continuous professional development, product training, and career pathing - Intradepartmental mentor and buddy program - Inclusive company culture and access to Inclusion Talks - Free, global mental health benefits for employees and dependents age 6+ - Competitive global benefits *Benefits and Growth may vary based on country and employment type.* --- ### Compensation (Estimated) - **$192,000 — $240,000 USD** (yearly) --- ### About Datadog Datadog is the leading observability and security platform for the AI era, providing unified visibility across the technology stack to manage complexity at scale. Learn more about **#DatadogLife** on Instagram and LinkedIn, and the Datadog Learning Center. --- ### Equal Opportunity & Accessibility Datadog is proud to offer equal employment opportunity to everyone regardless of protected characteristics. If you need assi
Listing freshness
CronJobs last confirmed this listing 19h ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.