Empregos de remote agent evaluation engineer em Coimbra

EPAM Systems Inc in Portugal is seeking an Agent Evaluation Engineer to design and maintain an evaluation framework for AI agents, blending deterministic tests with LLM-powered graders. The role focuses on build-time frameworks, CI/CD gates, and deployment validations within a remote setup. You will collaborate with SRE and ML teams to implement staging...

EPAM Systems, Inc. is seeking an Agent Evaluation Engineer to build evaluation frameworks for AI agents, combining deterministic and LLM-based grading with CI/CD deployment gates. You will design multi-layer evaluation suites, simulate multi-turn conversations, and establish reliability metrics, including pass@k metrics and staging validations. The role is...

EPAM Systems in Portugal seeks an Agent Evaluation Engineer to design and maintain a build-time evaluation framework for AI agents. You will implement automated tests, staging validations, and deployment gates integrated with production feedback to ensure robustness. You will develop multi-layer evaluation suites blending deterministic checks with...

An innovative AI company in Lisbon seeks part-time contributors for Quality Assurance roles focused on AI agents. Ideal applicants will have strong analytical thinking, attention to detail, and the ability to communicate findings in English. This freelance opportunity allows you to work flexibly on exciting AI projects while contributing to the future of...

We're looking for an Agent Evaluation Engineer — Build-Time Framework & Deployment Gates to join our team in Portugal in a fully remote working mode. In this role, you will design and maintain an evaluation framework for AI agents, ensuring quality and compliance through automated tests and CI/CD deployment gates. You will develop multi-layer evaluation...

We're looking for an Agent Evaluation Engineer — Build-Time Framework & Deployment Gates to join our team in Portugal in a fully remote working mode. In this role, you will design and maintain an evaluation framework for AI agents, ensuring quality and compliance through automated tests and CI/CD deployment gates. You will develop multi-layer evaluation...

Circle Internet Financial is hiring a Lead Engineer for the AI Quality engineering team. You will build evaluation frameworks, observability tooling, and diagnostic infrastructure to ensure AI agents perform reliably and cost-efficiently. This is a hands-on, player-coach role with leadership responsibilities. You’ll lead the team, set technical direction,...

An AI Data Services company is looking for a remote Kotlin Engineer to develop AI training content and evaluate AI responses for quality. The role requires at least 2 years of Kotlin development experience and strong English writing skills. Responsibilities include creating prompts for AI learning, optimizing AI model performance, and ensuring model...

Innodata Inc. seeks a Freelance Audio Engineer in Portugal to evaluate spoken interactions and assess English communication quality. You will participate in speech-to-speech evaluation tasks, judge pronunciation, fluency, clarity, and naturalness, and report findings according to project guidelines. Qualifications include excellent English listening and...

EPAM Systems in Portugal is seeking a Senior ML / Evaluation Engineer to own the design and implementation of advanced evaluation frameworks for a cloud-native AI agent platform. You will craft evaluation strategies, deploy CI/CD gates, and integrate LLM-as-judge and deterministic evaluators into automated pipelines. You will work with AWS AgentCore...

Jobgether is seeking an AI Evaluation Engineer to design evaluation, testing, and observability infrastructure for production-grade AI agents. You will create scalable evaluation pipelines, automate tests, and implement robust regression and adversarial scenarios, all in a fully remote, globally distributed setting. You will own the evaluation architecture,...

YO AI Labs is seeking a seasoned Full-Stack Software Engineer to support an AI training project focused on software development, debugging, code evaluation, and technical problem-solving. This is a remote, contractor role requiring strong cross-stack proficiency. You will develop front-end and back-end features, work with APIs and databases, review...

EPAM Systems, Inc. is seeking a Senior ML / Evaluation Engineer to design and implement advanced evaluation frameworks for an Enterprise Agent Development Platform. You will build LLM-as-judge evaluators and deterministic validators to enforce quality before release. You will collaborate with platform, orchestration, and DevOps teams to ensure scalable,...

1 week ago Be among the first 25 applicants Get AI-powered advice on this job and more exclusive features. This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English proficiency. At Mindrift, innovation meets...

Wire IT, based in Portugal, seeks an AI Engineer (GenAI & Agentic AI) to join a senior team driving AI-enabled solutions. The position is remote from Portugal, with strong emphasis on practical, production-grade AI systems, APIs, data pipelines, and enterprise integration. The candidate will translate ambiguous business needs into robust technical...

EPAM Systems, Inc. is seeking an Observability Engineer — AgentCore Telemetry to own observability and telemetry for a cloud-native enterprise AI platform. You will design tracing, metrics, and logging solutions using OpenTelemetry and AWS services to ensure health, performance, and enterprise compliance. You will instrument Python-based AI runtimes,...

We're looking for a Senior ML / Evaluation Engineer to join our team in Portugal in a fully remote working mode. In this role, you will own the design and implementation of advanced evaluation frameworks for an Enterprise Agent Development Platform—a production-grade, cloud-native ecosystem enabling scalable, secure AI agent deployment. You will create...

Mercor is seeking a Cloud Engineer / DevOps/SRE for a fully remote contract role. You will work with frontier AI coding agents to design, review, and automate scalable infrastructure across cloud platforms. Requirements include 2+ years in DevOps, SRE or Cloud Engineering, plus hands-on experience with AWS, Azure, or GCP, Kubernetes, Terraform, CI/CD...

Yuno is seeking a Staff AI Engineer to be the senior technical leader for Payments Intelligence—the unit building the AI agents and intelligence products that make payments smarter across the platform. You will architect and build production AI-agent systems like AIDA and NOVA, shape LLM applications for real payment problems, and raise the technical bar...

Receive Talent Seleção e Recrutamento procura um Principal/Staff Software Engineer (AI & Healthcare) para uma startup stealth de Healthcare AI com atuação em remoto na Europa. Procuramos engenheiros excepcionais para construir agentes de IA de raiz, com forte foco em produto e execução. A posição é 100% remota na Europa, com apresentação de...