Job Summary
In this role, you will assess software repositories to understand how well they challenge large language model (LLM) performance. Working alongside researchers, you will investigate GitHub issues, prepare development environments, and examine test coverage to help identify and validate suitable codebases.
Responsibilities
- Analyze and triage issues reported on GitHub.
- Configure development environments for codebase evaluation.
- Review test coverage as part of assessing software repositories.
- Partner with researchers to identify and test codebases that are challenging for LLMs.
- Navigate complex projects and support practical evaluation of their code and testing setup.
Qualifications
- At least three years of professional software engineering experience.
- Strong proficiency in Python.
- Comfort navigating and understanding complex codebases.
- Practical experience using Git and Docker.


