Job Summary
The role involves designing and sustaining scalable data pipelines essential for fine-tuning Large Language Models (LLMs) and developing autonomous AI agents. It also includes the implementation of telemetry systems to assess model performance and the management of data flows into vector databases to enable retrieval-augmented generation (RAG) capabilities.
Responsibilities
- Design and maintain efficient and scalable data pipelines for LLM fine-tuning.
- Oversee the creation of telemetry systems to monitor and evaluate model performance.
- Manage and optimize data flows into vector databases for RAG functionalities.
Qualifications
- A minimum of 5 years of professional experience in Data Engineering or MLOps.
- Expertise in programming languages, particularly Python and SQL.
- A Bachelor’s or Master’s degree in a technical discipline is required.
- Hands-on familiarity with LLM frameworks and cloud infrastructure is essential.
