Empregos de agent evaluation engineer build-time framework em porto

EPAM Systems, Inc. is seeking an Agent Evaluation Engineer to build evaluation frameworks for AI agents, combining deterministic and LLM-based grading with CI/CD deployment gates. You will design multi-layer evaluation suites, simulate multi-turn conversations, and establish reliability metrics, including pass@k metrics and staging validations. The role is...

EPAM Systems in Portugal seeks an Agent Evaluation Engineer to design and maintain a build-time evaluation framework for AI agents. You will implement automated tests, staging validations, and deployment gates integrated with production feedback to ensure robustness. You will develop multi-layer evaluation suites blending deterministic checks with...

We're looking for an Agent Evaluation Engineer — Build-Time Framework & Deployment Gates to join our team in Portugal in a fully remote working mode. In this role, you will design and maintain an evaluation framework for AI agents, ensuring quality and compliance through automated tests and CI/CD deployment gates. You will develop multi-layer evaluation...

EPAM Systems, Inc. is seeking an Agent Evaluation Engineer to build evaluation frameworks for AI agents, combining deterministic and LLM-based grading with CI/CD deployment gates. You will design multi-layer evaluation suites, simulate multi-turn conversations, and establish reliability metrics, including pass@k metrics and staging validations. The role is...

We're looking for an Agent Evaluation Engineer — Build-Time Framework & Deployment Gates to join our team in Portugal in a fully remote working mode.In this role, you will design and maintain an evaluation framework for AI agents, ensuring quality and compliance through automated tests and CI/CD deployment gates.You will develop multi-layer evaluation...

1 week ago Be among the first 25 applicantsGet AI-powered advice on this job and more exclusive features.This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English proficiency.At Mindrift, innovation meets...

Superhuman in Porto is seeking an AI Engineer to help build the next generation of AI-native productivity inside Superhuman, especially the agent platform where multiple AI agents collaborate to solve complex user problems. This high-autonomy, high-impact role is ideal for someone with a deep passion for LLMs, practical experience shipping ML-powered...

Xelerate in Portugal is seeking an experienced engineer to turn AI proofs of concept into reliable, production-grade agent systems for core industrial workflows. You will design and refine intelligent agents for tasks like quality report analysis, CAD reviews, and compliance checks. Join a remote-friendly team applying advanced retrieval strategies,...

Xelerate in Portugal is seeking an experienced engineer to turn AI proofs of concept into reliable, production-grade agent systems for core industrial workflows. You will design and refine intelligent agents for tasks like quality report analysis, CAD reviews, and compliance checks. Join a remote-friendly team applying advanced retrieval strategies,...

An innovative AI company in Lisbon seeks part-time contributors for Quality Assurance roles focused on AI agents. Ideal applicants will have strong analytical thinking, attention to detail, and the ability to communicate findings in English. This freelance opportunity allows you to work flexibly on exciting AI projects while contributing to the future of...

Superhuman offers a dynamic hybrid working model for this role. This flexible approach gives team members the best of both worlds: plenty of focus time along with in-person collaboration that helps foster trust, innovation, and a strong team culture. About Superhuman Superhuman is the AI productivity platform on a mission to unlock the superhuman potential...

We're looking for a Senior ML / Evaluation Engineer to join our team in Portugal in a fully remote working mode. In this role, you will own the design and implementation of advanced evaluation frameworks for an Enterprise Agent Development Platform—a production-grade, cloud-native ecosystem enabling scalable, secure AI agent deployment. You will create...

Seeking an AI Agent Evaluation Analyst for a freelance, part-time role to validate and improve AI agent performance. Responsibilities include reviewing evaluation tasks, identifying inconsistencies, and ensuring AI agents are tested properly. Requires strong analytical thinking and attention to detail.

Circle Internet Financial is hiring a Lead Engineer for the AI Quality engineering team. You will build evaluation frameworks, observability tooling, and diagnostic infrastructure to ensure AI agents perform reliably and cost-efficiently. This is a hands-on, player-coach role with leadership responsibilities. You’ll lead the team, set technical direction,...

Circle is building the world’s leading AI-powered, all-in-one platform for digital businesses, with a fully remote team and a strong focus on high-impact AI quality engineering. We are seeking a Lead Engineer to build evaluation frameworks, observability tooling, and diagnostic infrastructure for AI agents, while growing and leading the AI Quality...

Circle is seeking a Lead Engineer for AI Quality to build evaluation frameworks, observability tooling, and diagnostic infrastructure for its AI agents. This hands-on coach role also leads the AI Quality engineering team and collaborates with AI Core engineering to deliver reliable, scalable AI features. You will design datasets, run experiments across...

EPAM Systems, Inc. is seeking a Senior ML / Evaluation Engineer to design and implement advanced evaluation frameworks for an Enterprise Agent Development Platform. You will build LLM-as-judge evaluators and deterministic validators to enforce quality before release. You will collaborate with platform, orchestration, and DevOps teams to ensure scalable,...

EPAM Systems in Portugal is seeking a Senior ML / Evaluation Engineer to own the design and implementation of advanced evaluation frameworks for a cloud-native AI agent platform. You will craft evaluation strategies, deploy CI/CD gates, and integrate LLM-as-judge and deterministic evaluators into automated pipelines. You will work with AWS AgentCore...

Seeking experienced developers/test automation specialists for part-time AI project work, focusing on creating functional tests and analyzing AI systems using Python.

Seeking experienced developers to create coding tests for AI systems. Requires Python, Full-Stack experience, and English proficiency. Project-based, part-time role.