Note: The job is a remote job and is open to candidates in USA. Craft.co is a leader in supplier risk intelligence, enabling enterprises to discover, evaluate, and continuously monitor their suppliers. They are looking for a Senior Data Platform Engineer to build and optimize data pipelines, support data strategies, and leverage machine learning techniques to extract value from diverse datasets.
Responsibilities
- Build and optimize data pipelines (batch and streaming)
- Extracting, analyzing and modeling rich and diverse datasets of structured and unstructured data
- Design software that is easily testable and maintainable
- Support in setting data strategies and our vision
- Keep track of emerging technologies and trends in the Data Engineering world, incorporating modern tooling and best practices at Craft
- Work on extendable data processing systems that allows to add and scale pipelines
- Apply machine learning techniques such as anomaly detection, clustering, regression classification, and summarization to extract value from our data sets
- Leverage AI-powered development tools (e.g. Cursor) to accelerate development, refactoring, and code generation
Skills
- 4+ years of experience in Data Engineering
- 4+ years of experience with Python
- Experience in developing, maintaining, and ensuring the reliability, scalability, fault tolerance, and observability of data pipelines in a production environment
- Have fundamental knowledge of data engineering techniques: ETL/ELT, batch and streaming, DWH, Data Lakes, distributed processing
- Strong knowledge of SDLC and solid software engineering practices
- Familiar with infrastructure-as-code approach
- Demonstrated curiosity through asking questions, digging into new technologies, and always trying to grow
- Strong problem solving and the ability to communicate ideas effectively
- Self-starter, independent, likes to take initiative
- Familiarity with at least some of the technologies in our current tech stack: Python, PySpark, Pandas, SQL (PostgreSQL), ElasticSearch, Airflow, Docker, Databricks, AWS (S3, Batch, Athena, RDS, DynamoDB, Glue, ECS, Amazon Neptune), CircleCI, GitHub, Terraform
- Knowledge surrounding AI-assisted coding and experience with Cursor, Co-Pilot, or Codex
- A strong track record of leveraging AI IDEs like Cursor to: Rapidly scaffold components and APIs, Refactor legacy codebases efficiently, Reduce context-switching and accelerate documentation, Experiment and prototype with near-instant feedback
Benefits
- Equity at a well-funded, fast-growing startup
- Unlimited vacation time so you can take what you need, when you need it
- 99% covered health + dental + vision insurance for employees and dependents
- 401K through Empower with options to invest how you want it
Company Overview
Company H1B Sponsorship