We're collaborating with a cutting-edge platform that's revolutionising how the world's most innovative companies refine their AI agents through human feedback. This organisation is at the forefront of training Large Language Models (LLMs) to operate as advanced, multi-step agents, focusing on the design, coordination, and optimisation of intricate, real-world architectural workflows.
The Role
- Develop objective, verifiable criteria (rubrics) to evaluate system performance and ensure outputs meet strict functional requirements.
- Review system logs and 'trajectories' to refactor code, improve execution paths, and achieve optimal reliability.
- Test systems for vulnerabilities, including improper data exposure, unauthorized access escalations, and edge-case failures.
- Contribute to training generative AI models as a skilled software expert.
What You'll Need
- 2+ years of experience in backend engineering, AI automation, or complex systems integration.
- Proven ability to build and maintain production-grade software with modular separation.
- Strong command of at least two major languages (e.g., Python, JavaScript, Go, or Java) and experience with SQL databases.
- Practical experience building for live, non-mocked environments and handling multi-turn system interactions.
- Outstanding attention to detail and the ability to provide clear, high-density technical feedback on complex system behaviors.
- Expertise building multi-stage coordination tasks where data acquisition leads to reasoned output is a plus.
What's On Offer
- Fully remote, freelance role with flexible hours.
- Opportunity to shape the future of autonomous agents.
- Competitive project-based compensation, with additional incentives.
- Work directly with leading AI organisations.
Apply via Haystack today!
US
