We are working with a pioneering company at the forefront of AI innovation, focused on enhancing the capabilities of autonomous agents by integrating human intelligence into their development. This organisation partners with leading AI entities to train Large Language Models (LLMs), teaching them to design, coordinate, and optimise complex, real-world architectural workflows.
The Role
- Develop objective and verifiable criteria (rubrics) to evaluate system performance and ensure outputs meet strict functional requirements
- Review system logs and "trajectories" to refactor code, improve execution paths, and achieve optimal reliability
- Test systems for vulnerabilities, including improper data exposure, unauthorized access escalations, and edge-case failures.
- Contribute expertise to train generative AI models.
- This is a remote freelance opportunity with flexible hours.
What You'll Need
- 2+ years of experience in backend engineering, AI automation, or complex systems integration
- Proven ability to build and maintain production-grade software with modular separation
- Strong command of at least two major languages (e.g., Python, JavaScript, Go, or Java)
- Experience working with SQL databases
- Practical experience building for live, non-mocked environments and handling multi-turn system interactions
- Outstanding attention to detail and ability to provide clear, high-density technical feedback on complex system behaviors
What's On Offer
- Opportunity to shape the future of autonomous AI agents
- Flexible freelance work schedule enabling work whenever it fits your schedule
- Competitive hourly rates for project work
- Additional incentives and bonuses tied to performance
Apply via Haystack today!
US
