Company: Artech Information Systems
Job description: JOB TITLE : Sr. Databricks EngineerJOB SUMMARY
The Databricks Engineer will be a critical member of our Enterprise Data Team reporting to the Chief Software Architect. This role is responsible for developing and delivering engineering solutions that align with our business requirements. The position relies on expertise in data architecture on Databricks delta lake, streaming technologies, Azure cloud platform, ETL development, data modeling, and data warehousing. This role will contribute to cutting-edge data projects and contribute to the growth of our Digital Transformation organization.ESSENTIAL DUTIES
- Azure Analytics Solutions:
- Develop and implement scalable analytics solutions on the Azure platform
- Utilize Azure's suite of tools to create robust and efficient data systems
- Data Pipeline Management:
- Design, develop, and manage end-to-end data pipelines
- Focus on efficient data collection, storage, and access methods
- Ensure high-quality data engineering, including ETL processes and data modeling
- Databricks Solutions:
- Design and implement data solutions using Databricks
- Work with Delta Lake, data warehouses, and data marts
- Create analytics solutions that support organizational needs
- Cross-functional Collaboration:
- Conduct discovery exercises to understand the data landscape
- Develop and demonstrate proof of concepts to stakeholders
- Work across teams to align data solutions with business objectives
- Best Practices Implementation:
- Apply industry best practices to ensure data and code quality
- Focus on consistency and maintainability in all projects
- Stay updated with the latest trends and technologies in data analytics
Bachelor's or Master's degree in Computer Science, Engineering, Data Science, Information Systems, or a related field or, equivalent of education and experience sufficient to successfully perform the essential functions of the job may be considered.Experience
Extensive Experience in Data Lake and DataMart Development:
- Over 5 years of overall experience in the field
- More than 4 years specifically dedicated to developing data lakes and DataMarts on the Databricks platform
- Proficient in designing and implementing scalable data architectures
- 6+ years of hands-on experience in data engineering
- Strong focus on Azure Databricks and related Azure services
- Demonstrated ability to develop and optimize complex data ingestion processes
- Track record of significantly improving data processing performance
- Demonstrated expertise in programming languages such as Python, Spark, and SQL, with a strong emphasis on writing efficient and maintainable code.
- In-depth experience with Apache Spark and its ecosystem, particularly its integration with Databricks for large-scale data processing.
- Advanced proficiency in SQL, coupled with a solid background in data modeling techniques (e.g., Star Schemas) and practical knowledge of data warehousing concepts.
- Proven track record in developing and maintaining data warehouses and data marts, showcasing an understanding of ETL processes and data pipeline architectures.
- Comprehensive understanding of data security principles and best practices, including data encryption, access control, and compliance with relevant regulations.
- Exceptional communication skills, with the ability to explain complex technical concepts to both technical and non-technical stakeholders effectively.
- Familiarity with software engineering best practices, including version control systems (particularly Git) and experience with CI/CD pipelines for streamlined development and deployment.
- Strong analytical and conceptual thinking skills, paired with excellent collaboration abilities to work effectively in cross-functional teams.
- Robust understanding of system architectures and design patterns, with the capability to design and develop applications adhering to these principles.
- Outstanding problem-solving skills, demonstrating the ability to structure and implement appropriate analytical solutions for complex data challenges.
- A strong sense of teamwork, ownership, and accountability, ensuring project goals are met and high-quality deliverables are produced.
- An equivalent competency level acquired through a variation of these qualifications may be considered.
Azure certifications such as Azure Data Engineer Associate or Databricks Certified Data Engineer Associate preferred.PHYSICAL REQUIREMENTS
Typical Office Environment requirements include: reading, speaking, hearing, close vision, traverse, bending, sitting, and occasional lifting up to 20 pounds.The physical demands described here are representative of those that must be met by an associate to successfully perform the essential duties of this job. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential duties.Additional Physical Requirements
Hybrid onsite/remote work allowed: 3 days a week in our Lakewood offices, 2 from home.
Expected salary:
Location: Lakewood, CO
Apply for the job now! Apply Now