Note: The job is a remote job and is open to candidates in USA. Eclipse Labs is building an AI agent-first marketplace that connects intelligence with real-world tasks. They are seeking a Data Scientist to establish data labeling and processing foundations for next-generation Large Language Models (LLMs). The role involves developing data labeling strategies, optimizing data for LLM consumption, and ensuring data quality through automated processes.
Responsibilities
- Develop Data Labeling Strategies: Design and document a formal data annotation strategy, including clear, scalable, and efficient guidelines for labeling our data. Define and enforce quality metrics, including inter-annotator agreement
- Optimize for LLM Consumption: Research, define, and prototype the optimal data formats, structures, and pre-processing steps required for fine-tuning and training LLMs on our datasets
- Data Quality Analysis: Establish automated processes and metrics to analyze the quality of both raw and labeled data, providing feedback to improve our data collection and labeling workflows
- Collaborate with Engineering: Work closely with the engineering team to guide the implementation of data processing pipelines and ensure the data infrastructure meets the needs of ML applications
Skills
- Proven experience as a Data Scientist or Machine Learning Engineer with a focus on data quality and preparation
- Strong understanding of data labeling methodologies and hands-on experience with data annotation platforms and workflows
- Demonstrated experience preparing datasets for training and fine-tuning Large Language Models (LLMs), including knowledge of techniques like tokenization, embeddings, and NER
- Proficiency in Python and common data science libraries (e.g., Pandas, NumPy, Scikit-learn, spaCy, Hugging Face)
- Experience using APIs/SDKs to automate data annotation and active learning loops
- Excellent communication skills, with an ability to create clear documentation for technical and non-technical audiences
- Experience with audio data processing and relevant libraries
- Familiarity with data annotation platforms and tools
- Knowledge of modern MLOps principles and practices
- Experience with large language model data curation and Reinforcement Learning from Human Feedback (RLHF) pipelines
Benefits
- Competitive salary + equity + benefits package
- Flexibility. We collaborate synchronously and asynchronously, across weekly all-hands meetings, Slack messaging, and quarterly in-person meetups
Company Overview
Company H1B Sponsorship