Job Summary
The role involves leading competitive benchmarking efforts across various domains, including search, vector and hybrid retrieval, analytics, and observability. The candidate will be responsible for developing automated and reproducible tests, as well as creating representative datasets to analyze performance and cost trade-offs effectively. Collaboration with engineering and product teams is essential to drive improvements and produce comprehensive reports that clearly present methods, results, and any limitations encountered.
Responsibilities
- Conduct competitive benchmarking for search, retrieval, analytics, and observability tasks.
- Develop automated, reproducible tests and design representative datasets.
- Analyze trade-offs between performance and cost in various workloads.
- Collaborate with engineering and product teams to implement improvements.
- Document methods, findings, and limitations in clear reports.
Qualifications
- Strong software engineering capabilities.
- Experience with performance evaluation in distributed systems, databases, or search infrastructure.
- Expertise in at least one relevant technical field.
- Ability to design effective workloads and conduct repeatable experiments.
- Skill in building reproducible test environments and communicating findings effectively.
- Aptitude for translating evidence into actionable engineering recommendations.
