Job Summary:
At Toptal, we measure everything and always rely on data to guide all of our initiatives, including both our long-term strategy and our day-to-day operations. As a Principal Data Engineer, your primary goal is to be one step ahead. You will support Business Analysts, Data Analysts, and Data Scientists by providing infrastructure and tools they can use to deliver end-to-end solutions and business insights. This is more than building and maintaining ETL pipelines–we need innovation, creativity, and solutions that will have a significant impact on our velocity and business impact. We, in turn, will give you autonomy and freedom to turn your ideas into reality.
This is a remote position. We do not offer visa sponsorship or assistance. Resumes and communication must be submitted in English.
Responsibilities:
The following information is intended to describe the general nature and level of work being performed. It is not intended to be an exhaustive list of all duties, responsibilities, or required skills.
- Lead and contribute to architecting and building a modern data stack that is scalable, maintainable, and highly-performant. This includes building frameworks, data pipelines, and other data infrastructure using a variety of raw data sources.
- Collaborate with data source providers both internal and external to set rules, processes, and checks that ensure data availability and integrity.
- Monitor and maintain the data pipelines and ETL processes to proactively remediate issues and preserve data availability.
- Support the migration from legacy orchestration systems and ETL, including migrating pipelines reading from internal databases and APIs, remapping data sources, and translating transformation logic.
- Ensure proper governance practices and effective documentation are implemented throughout the ETL migration process.
- Communicate with team members and convey results efficiently and clearly.
In the first week, expect to:
- Meet mentors that will help you during your onboarding month.
- Meet your team, managers, and other key stakeholders like Business Analysts.
- Start participating in company-wide training sessions.
- Set up your local environment and become familiar with our tech stack.
In the first month, expect to:
- Have a comfortable understanding of our data stack and systems.
- Have a clear understanding of the team’s data strategy and direction for the infrastructure.
- Develop an understanding of Toptal’s business and offerings.
- Develop an understanding of Toptal’s different processes and team structure, and identify and meet key technical and business stakeholders.
- Start contributing to legacy cleanup and migration efforts.
In the first three months, expect to:
- Have strong knowledge of Toptal’s business.
- Actively supply daily support to users, development, migration, and maintenance work.
- Take part in the on-going monitoring and maintenance of data pipelines and processes.
- Be ready to propose and implement improvements to Toptal’s processes and codebase.
- Deliver value in a regular cadence.
- Get comfortable in your daily work within your team.
In the first six months, expect to:
- Own technical initiatives on our team.
- Drive improvements to the codebase and processes.
- Contribute to planning and executing long-term initiatives inside your team.
- Be able to not only solve complex problems, but also consider multiple solutions, weigh them and decide on the best course of action.
- Exercising discretion and independent judgment, proactively identify technical debt and product areas that require attention or improvements and suggest improvements in our technology stack.
In the first year, expect to:
- Have a detailed understanding of Toptal’s business, collaboration rituals, processes, performance, and future work.
- Determine what your career path looks like at Toptal.
- Mentor Toptal’s new team members.
Qualifications and Job Requirements:
- Bachelor’s degree is required.
- 10+ years of experience working with data infrastructure–architecture, cloud data warehouses, data modeling, ETL tools and processes, and data ingestion techniques.
- Extensive experience working with Python, Pandas, and SQL
- Experience with Google Cloud Platform (including Google Cloud Storage and BigQuery), object-oriented programming, CI/CD, and ETL technologies such as Airflow, Luigi, Dagster, and CDC is preferred
- Outstanding English written and verbal communication skills.
- Be excited about collaborating daily with your team and other groups while working via a distributed model.
- Be eager to help your teammates, share your knowledge with them, and learn from them.
- Be open to receiving constructive feedback.
- Ability to work in a fast-paced, rapidly growing company and handle a wide variety of challenges, deadlines, and a diverse array of contacts.
- You must be a world-class individual contributor to thrive at Toptal. You will not be here just to tell other people what to do.
Originally posted on Himalayas
Apply Now