Job Summary
The role involves spearheading the technical strategy for extensive model training initiatives. This includes crafting detailed guidelines for pre-training, fine-tuning, and reinforcement learning processes. A significant aspect of the position is to work closely with both researchers and engineers to enhance model performance and ensure scalability on GPU and TPU infrastructures.
Responsibilities
- Provide leadership in the technical direction for large-scale model training.
- Develop comprehensive recipes for pre-training and fine-tuning models.
- Focus on reinforcement learning methodologies.
- Collaborate with team members to optimize model performance on various hardware infrastructures.
- Ensure effective scalability of models on GPU and TPU systems.
Qualifications
- A minimum of 8 years of experience in large-scale model training.
- At least 5 years of experience in production systems.
- Strong proficiency in either PyTorch or JAX.
- Experience with distributed training methodologies.
- Advanced programming skills in Python or C++.
