Note: The job is a remote job and is open to candidates in USA. Grafana Labs, the company behind the open observability cloud, is looking for a Staff AI Engineer to help develop AI-driven features that improve observability tools. The role involves building high-performance AI solutions, collaborating with cross-functional teams, and taking ownership of the AI solutions developed.
Responsibilities
- Build and deliver AI solutions: Take ownership of developing high-performance AI features to help users detect, triage, and resolve incidents using observability data and tools
- Rapid experimentation and iteration: Implement a highly iterative process where you quickly prototype, test, and validate with real users, including shipping and evolving LLM- or agent-powered workflows for incident lifecycle management and automated analysis tasks
- Collaborate cross-functionally: Work with data analysts, product managers, and designers to shape AI-driven product features, including integration of agentic components with internal tools, alerting systems, runbooks, and developer workflows
- Utilize AI tools effectively: Use AI and automation tools to enhance both product functionality and your own development workflows
- Effective communication: You’ll be working in a highly dynamic and collaborative environment, so we need someone who can communicate effectively and contribute across teams
- Ownership and impact: Take full ownership of the AI solutions you develop, ensuring they are not only innovative but also scalable, maintainable, and aligned with real user workflows
Skills
- Experience with LLMs, prompt engineering, and building applications powered by GenAI
- Proven track record of delivering software that made it into production and is actively used by users
- Exposure to working in cloud-native environments (e.g., AWS, GCP, Azure)
- Experience using observability tools to understand and troubleshoot system behavior
- Experience building or working with agent frameworks or multi‑agent workflows
- Experience with infrastructure / devops related tooling: Kubernetes, Docker, Terraform or similar for deployments
- Familiarity with model fine-tuning techniques
- Experience building observability tooling
Benefits
- Equity
- Bonus (if applicable)
- Restricted Stock Units (RSUs)
- 100% Remote, Global Culture
- Career Growth Pathways
- In-Person onboarding
- Global annual leave policy of 30 days per annum
- 3 days of your annual leave entitlement are reserved for Grafana Shutdown Days to allow the team to really disconnect
- We will comply with local legislation where applicable
Company Overview