- Design, implement, and manage scalable data architectures on distributed platforms (e.g., MapR, HPE Unified Analytics and Data Fabric).
- Develop, optimize, and maintain robust data pipelines using tools such as Spark, Airflow, and EzPresto.
- Configure and maintain Kafka architecture, MapR Streams, and related technologies to support real-time and batch data processing.
- Implement Change Data Capture (CDC) mechanisms and integrate data using APIs and streaming techniques.
- Monitor, tune, and troubleshoot distributed data clusters including MapR and Kubernetes environments.
- Develop and maintain CI/CD pipelines using Jenkins and integrate with GitHub for automated testing and deployment.
- Collaborate with cross-functional teams to ensure data quality, governance, and compliance standards are met.
- Leverage tools such as Iceberg and Superset for data storage optimization and visualization. Required Skills & Experience:
- Strong experience with distributed data platforms, including MapR and Kubernetes.
- Proficient in data pipeline tools and frameworks: Spark, Airflow, EzPresto.
- Solid programming and scripting skills: Python, Bash.
- Expertise in Kafka architecture and operations.
- Experience with CI/CD development workflows using Jenkins and GitHub.
- Knowledge and use of Apache Iceberg for data lake management.
- Familiarity with data architecture best practices, including CDC and API-based integrations. Preferred / Nice to Have:
- Experience with HPE Unified Analytics and Data Fabric.
- Familiarity with MapR Streams and Superset for real-time analytics and dashboarding
Apply for the job now! Apply Now