Llm Evaluator

Odixcity Consulting seeks a detail-oriented LLM Evaluator to assess and improve large language model outputs, focusing on factuality, coherence, safety, and alignment with guidelines. You will judge model responses, provide clear feedback, and help refine evaluation rubrics.Responsibilities include ranking responses, comparing prompts, calibrating scores...

AI Data Evaluator role in Lisbon, Portugal, focused on reviewing and labeling AI-generated responses to ensure accuracy, quality, and safety. Collaboration with international teams and contribution to AI training.

CloserHire is recruiting an Arabic-speaking AI Data Evaluation Specialist for a Lisbon-based project on advanced AI systems. You will review and improve AI-generated responses, ensure language accuracy, safety, and policy compliance, on a 24/7 rotating schedule with two days off. Relocation support is included. Ideal candidates are native Arabic speakers...

French AI Content Evaluator | Lisbon Interested in Generative AI and Large Language Models? Join an international team evaluating AI-generated content, datasets and written responses for quality and accuracy.€930 gross × 14 payments€7.63 daily meal allowanceRelocation support providedOn-site, 24/7 rotating shiftsNative-level written French and B2+...

EPAM Systems in Portugal is seeking a Senior ML / Evaluation Engineer to own the design and implementation of advanced evaluation frameworks for a cloud-native AI agent platform. You will craft evaluation strategies, deploy CI/CD gates, and integrate LLM-as-judge and deterministic evaluators into automated pipelines. You will work with AWS AgentCore modules,...

EPAM Systems, Inc. is seeking a Senior ML / Evaluation Engineer to design and implement advanced evaluation frameworks for an Enterprise Agent Development Platform. You will build LLM-as-judge evaluators and deterministic validators to enforce quality before release. You will collaborate with platform, orchestration, and DevOps teams to ensure scalable,...

We're looking for a Senior ML / Evaluation Engineer to join our team in Portugal in a fully remote working mode. In this role, you will own the design and implementation of advanced evaluation frameworks for an Enterprise Agent Development Platform—a production-grade, cloud-native ecosystem enabling scalable, secure AI agent deployment. You will create...

We're looking for anAgent Evaluation Engineer — Build-Time Framework & Deployment Gatesto join our team in Portugal in a fully remote working mode. In this role, you will design and maintain an evaluation framework for AI agents, ensuring quality and compliance through automated tests and CI/CD deployment gates. You will develop multi-layer evaluation...