Job Summary
The position involves creating and sustaining evaluation harnesses designed to evaluate multimodal foundation models across a range of tasks such as text processing, coding, and mathematical computations. The insights gained from your analyses will play a critical role in shaping training methodologies, selecting checkpoints, and detecting regressions in model performance.
Responsibilities
- Develop and maintain evaluation systems for multimodal foundation models.
- Analyze experimental outcomes to guide training strategies and checkpoint choices.
- Identify any regressions in model performance through thorough evaluation.
Qualifications
- Extensive experience in constructing model evaluation frameworks.
- Strong statistical analysis skills for interpreting experimental data.
- Proficiency in Python programming.
- Demonstrated success in creating benchmarks to influence research direction.
Hark provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.
