Note: The job is a remote job and is open to candidates in USA. Drata is a leading company in compliance software innovation, dedicated to building trust and security across the internet. They are looking for a Staff Applied AI Engineer to enhance the quality and effectiveness of their AI systems through research, experimentation, and evaluation, focusing on compliance automation.
Responsibilities
- Design and evaluate information access + reasoning strategies across RAG, agents, and classic ML: chunking, embedding models, hybrid search, metadata filtering, structured retrieval, tool use, and multi-step workflows
- Prototype GenAI workflows (including agentic systems) that map and reason over compliance objects (controls ↔ risks ↔ requirements)
- Explore ML + probabilistic approaches where GenAI is not the best fit: classifiers, ranking models, graph/link prediction, calibration, and weak supervision
- Build and maintain evaluation frameworks: golden datasets, automated quality metrics, regression detection
- Implement and tune ranking/reranking systems: cross-encoders, LLM-based rerankers, learning-to-rank, custom scoring functions
- Run experiments to validate hypotheses and quantify improvements before production rollout
- Debug failure modes and build error taxonomies across retrieval, reasoning, and generation
- Collaborate with AI and Software Engineers to hand off validated approaches for productionization
- Stay current on applied research in RAG, agents, LLM evaluation, and relevance modeling; bring innovations into the product
Skills
- 10+ years of experience in applied research, data science, or ML with a focus on NLP, information retrieval, or knowledge systems
- 2+ years of hands-on experience building or contributing to production AI/ML systems
- Strong foundation in information retrieval: dense and sparse retrieval, embedding models, search relevance
- Experience with RAG systems: chunking strategies, vector databases, retrieval optimization
- Proficiency in evaluation methodology: metrics design, golden dataset creation, A/B testing, statistical analysis
- Strong Python skills and comfort with notebook-driven research workflows
- Experience communicating research findings to engineering teams and translating insights into actionable recommendations
- Experience with compliance, legal, or document-heavy domains
- Publications or contributions in IR, NLP, or RAG evaluation
Benefits
- Stock equity to ensure that as the company grows, you share directly in that success
- Up to 100% employer-paid premiums for medical, dental, and vision coverage for employees •and their dependents•
- Comprehensive wellness benefits and healthcare concierge services designed to support your needs beyond traditional insurance
- A comprehensive suite of financial benefits, including a 401(k) plan, company-paid life and disability insurance, tax-advantaged spending accounts, and a range of discounted voluntary offerings to help you customize and strengthen your overall financial position
- Paid Parental Leave policy, after six months of employment
- Access to Kindbody fertility and family-building benefits and dedicated leave specialists who help guide you through the entire process
- Generous annual stipends for both professional and personal development
- Access to a wide range of internal learning opportunities
- Flexible vacation policy, paid holidays, and other perks to recharge
Company Overview
Drata is a Trust Management Platform that automates compliance workflows, streamlines risk management, and continuous security assurance It was founded in 2020, and is headquartered in San Diego, California, USA, with a workforce of 501-1000 employees. Its website is https://drata.com.Company H1B Sponsorship
Drata has a track record of offering H1B sponsorships, with 1 in 2026, 1 in 2025, 2 in 2024, 1 in 2023. Please note that this does not guarantee sponsorship for this specific role.