Note: The job is a remote job and is open to candidates in USA. Quisitive is a global partner of Microsoft specializing in cloud transformation and enterprise data strategy. As a Data Engineer, you will design, build, and maintain scalable data pipelines and platform solutions on Microsoft Azure, ensuring robust data processing and a secure cloud environment for analytics.
Responsibilities
- Design & Develop Data Pipelines: Build end-to-end data ingestion and transformation pipelines using Azure tools (e.g., Azure Data Factory, Synapse Pipelines). Ensure pipelines are efficient, reliable, and ready for continuous integration
- Azure Databricks Development & Administration: Use Azure Databricks (Spark) to develop and optimize data processing jobs (PySpark/Scala) for large datasets. Manage Databricks workspaces and clusters, tuning configurations and troubleshooting jobs to maintain performance and cost-efficiency
- Automate Infrastructure & Deployments: Use Infrastructure-as-Code (Bicep) to provision and manage Azure data platform resources across environments. Implement and maintain Azure DevOps CI/CD pipelines (build & release) for automated deployment of data pipelines and infrastructure changes
- Modern Data Platform Support: Support the day-to-day operations of the Azure Modern Data Platform. Monitor and maintain platform stability (storage, compute), assist in resolving connectivity or networking issues, and ensure proper Azure RBAC permissions and data security measures are in place
- Collaboration & Best Practices: Work closely with data architects, BI developers, cloud engineers, and client stakeholders. Ensure solutions align with Azure best practices, follow Quisitive standards, and meet business requirements
Skills
- 3+ years of hands-on experience in data engineering with Microsoft Azure (data ingestion, ETL/ELT pipeline development, big data processing)
- Proficiency in Azure data services such as Azure Databricks, Azure Data Factory, Azure Synapse Analytics, and related SQL/NoSQL technologies
- Experience using Azure DevOps (Repos & Pipelines) for CI/CD: managing code, building deployment pipelines, and automating releases
- Practical experience with infrastructure-as-code (Bicep, ARM templates or Terraform) for deploying and managing Azure resources and environments
- Strong programming skills in Python/Scala (for Spark jobs) and SQL (for data transformations)
- Familiarity with Microsoft Fabric and Azure fundamentals (cloud networking, security/permissions, monitoring) to support stable operations
- Excellent problem-solving skills and effective communication when working with technical teams and clients
Company Overview