Job Summary
The incumbent will be responsible for creating and developing an evaluation harness specifically designed for agentic AI, focusing on aspects such as trustworthiness, faithfulness, and overall performance. This role will involve red-teaming activities, ensuring the integrity of data separation, and executing automated testing throughout the platform’s microservices and AI pipelines.
Responsibilities
- Design and develop an evaluation harness for agentic AI systems.
- Conduct red-teaming exercises to assess AI system vulnerabilities.
- Ensure data isolation and validate its effectiveness.
- Implement automated testing processes across microservices and AI pipelines.
- Evaluate and analyze non-deterministic outputs produced by AI systems.
Qualifications
- Strong programming skills in Go or Python.
- Experience with LLM-based or agentic AI systems.
- Proven history in software development and quality engineering.
- Capability to effectively assess and validate AI outputs.
