Senior Applied AI Engineer 2
Drata · Hybrid - San Francisco
About this role
## Senior Applied AI Engineer 2 — Drata Drata is building the trust layer between great companies—automating compliance, managing risk, and helping organizations prove trust continuously as they scale. You’ll help build an agentic AI platform that defines what trust looks like for the next generation of companies. ### Why join Drata - **Problems without a playbook:** Work at the edge of AI and security on agentic governance, continuous compliance, and real-time trust verification. - **Real ownership:** Own outcomes and raise the bar—bring opinions and back them up. - **A seat at the table:** Open debate and diverse viewpoints are built into how decisions are made. - **Growth at rocketship speed:** Fast scaling means fast-growing scope and high ownership. - **A crew, not just coworkers:** A “come as you are” culture with sharp, curious people. ### Job Summary This is a **research-focused** role emphasizing experimentation, evaluation, and applied research over production engineering. You’ll own the science behind how Drata’s AI systems **retrieve, reason, and respond**, partnering closely with AI and Software Engineers to turn validated approaches into production-ready systems. ### What you’ll do - Design and evaluate **information access + reasoning strategies** across **RAG, agents, and classic ML** (e.g., chunking, embedding models, hybrid search, metadata filtering, semantic routing) - Prototype **GenAI workflows** (including agentic systems) mapping and reasoning over compliance objects (**controls ↔ risks ↔ requirements ↔ evidence**) - Explore **ML + probabilistic approaches** where GenAI isn’t the best fit (e.g., classifiers, ranking models, graph/link prediction, calibration, structured prediction) - Build and maintain **evaluation frameworks** (golden datasets, automated quality metrics, regression detection) - Implement and tune **ranking/reranking systems** (cross-encoders, LLM-based rerankers, learning-to-rank, custom scoring) - Run experiments to validate hypotheses and quantify improvements before production rollout - Debug failure modes and build **error taxonomies** across retrieval, reasoning, and generation - Collaborate with AI and Software Engineers to hand off validated approaches for productionization - Stay current on applied research in **RAG, agents, LLM evaluation, and relevance modeling** and bring innovations into the product ### What you’ll bring - **6+ years** experience in applied research, data science, or ML with focus on **NLP, information retrieval, or knowledge systems** - **2+ years** hands-on experience building or contributing to **production AI/ML systems** - Strong foundation in **information retrieval** (dense/sparse retrieval, embedding models, search relevance) - Experience with **RAG systems** (chunking strategies, vector databases, retrieval optimization) - Proficiency in **evaluation methodology** (metrics design, golden dataset creation, A/B testing, statistical significance) - Strong **Python** skills and comfort with notebook-driven research workflows - Ability to communicate research findings and translate insights into actionable engineering improvements - **Bonus:** Experience with compliance/legal or document-heavy domains - **Bonus:** Publications or contributions in IR, NLP, or RAG evaluation ### How Drata supports you - **Shared Success:** Stock equity - **Health & Wellness:** Up to 100% employer-paid medical/dental/vision premiums + wellness benefits - **Financial Well-being
Listing freshness
CronJobs last confirmed this listing 59m ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.