Note: The job is a remote job and is open to candidates in USA. EPAM Systems is seeking a highly motivated and experienced Generative AI Operations Engineer to join their innovative team. In this role, you will be responsible for building, deploying, and maintaining the operational infrastructure for cutting-edge generative AI models and services.
Responsibilities
- Build and Manage CI/CD Pipelines: Design, implement, and maintain robust, automated CI/CD pipelines for training, evaluating, and deploying large language models (LLMs) and AI agents
- Orchestrate Agentic AI Workflows: Design, deploy, and manage sophisticated, multi-agent systems. Ensure seamless Agent-to-Agent (A2A) communication and collaboration between specialized agents to automate complex business processes
- Manage Tool Integration: Implement and manage secure, scalable integrations between AI agents and external tools/APIs, leveraging open standards like the Model Context Protocol (MCP) to ensure interoperability
- Leverage AI-Powered Development: Utilize AI-powered development tools to accelerate the entire software development lifecycle, from writing infrastructure code and tests to troubleshooting operational issues in cloud environments
- Infrastructure as Code (IaC): Utilize cloud-native IaC services or cloud-agnostic tools like Terraform to define and manage the infrastructure required for GenAI workloads
- Model Monitoring and Observability: Implement comprehensive monitoring and logging solutions to track model and agent performance, resource utilization, and system health. For agentic systems, this includes tracing the agent's actions and logging the multi-step conversational flow
- Scalability and Performance Optimization: Design and implement scalable architectures for model serving and inference. Continuously optimize the performance and cost-effectiveness of our GenAI services
- Security and Compliance: Implement and enforce security best practices for our GenAI infrastructure and data. Ensure compliance with industry standards and regulations
Skills
- 3+ years in a DevOps, SRE, or MLOps role with a focus on cloud infrastructure and a background in cloud services (AWS, GCP, Azure)
- Skills in building and managing CI/CD pipelines (Jenkins, GitLab CI, or cloud-native services) and proficiency in at least one scripting language (e.g., Python, Bash)
- Familiarity with IaC tools (e.g., AWS CDK, CloudFormation, Terraform) and containerization/orchestration (Docker, Kubernetes)
- Track record of deploying and operating LLM inference (e.g., vLLM, Triton, TGI, Ray Serve, KServe/Seldon)
- Hands-on experience with LLM/app tracing and metrics (e.g., OpenTelemetry + Langfuse, Arize Phoenix, WhyLabs) and in building evaluation pipelines (offline/online, regression suites)
- Skills in operating retrieval pipelines: embedding generation, indexing/refresh strategies, vector DBs (Pinecone, Weaviate, Milvus, FAISS), and relevance monitoring
- Experience in running multi-agent workflows (LangGraph, CrewAI, AutoGen-like), including state management, retries, rate limits, tool-failure handling, and step-level auditing
- Experience in implementing guardrails: secrets isolation, tool/API permissions, prompt-injection defenses, data leakage prevention, PII redaction, and policy enforcement
- Background integrating agents with external tools using MCP (or similar tool-calling standards) and operating tool registries is a plus
- Master's degree or PhD in Computer Science, AI, Machine Learning, or a related field
- Experience with cloud-native GenAI services like AWS Bedrock, Azure AI Foundry, or Google Vertex AI
- Familiarity with the architecture and operational challenges of Large Language Models (LLMs)
- Experience designing or managing multi-agent systems or complex, orchestrated workflows
- Knowledge of monitoring and observability tools like Prometheus, Grafana, or Datadog
- Relevant cloud or DevOps certifications
- Strong problem-solving skills and the ability to work effectively in a fast-paced, collaborative environment
Benefits
- Engineering community of industry professionals
- Friendly team and enjoyable working environment
- Flexible schedule and opportunity to work remotely within Poland
- Chance to work abroad for up to 60 days annually
- Business-driven relocation opportunities
- Outstanding career roadmap
- Leadership development, career advising, soft skills, and well-being programs
- Certification (GCP, Azure, AWS)
- Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru
- English classes
- Stable income (Employment Contract or B2B)
- Participation in the Employee Stock Purchase Plan
- Benefits package (health insurance, multisport, shopping vouchers)
- Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more
- Referral bonuses
- Corporate, social and well-being events
Company Overview
Company H1B Sponsorship