evaluation

EPAM Systems, Inc. is seeking an Agent Evaluation Engineer to build evaluation frameworks for AI agents, combining deterministic and LLM-based grading with CI/CD deployment gates. You will design multi-layer evaluation suites, simulate multi-turn conversations, and establish reliability metrics, including pass@k metrics and staging validations. The role is...

Braintrust is seeking a seasoned software engineer to help a top AI lab evaluate and improve large language models through security-focused coding tasks. You will evaluate vulnerabilities, reproduce exploits, verify patches, and create prompts and reference answers; assess model outputs for code generation, debugging and refactoring. Remote contracting with...

Prolific is seeking Fluent Bengali speakers to help train and evaluate AI models. If you have the necessary experience, a quick Hallo skills test will determine your suitability for our Domain Expert participant pool. Successful candidates can join Prolific and earn paid tasks up to $20/hr. Expect to analyze, edit, and write in Bengali, judge AI performance...

EPAM Systems, Inc. is seeking an Agent Evaluation Engineer to build evaluation frameworks for AI agents, combining deterministic and LLM-based grading with CI/CD deployment gates. You will design multi-layer evaluation suites, simulate multi-turn conversations, and establish reliability metrics, including pass@k metrics and staging validations. The role is...

EPAM Systems, Inc. is seeking an Agent Evaluation Engineer to build evaluation frameworks for AI agents, combining deterministic and LLM-based grading with CI/CD deployment gates. You will design multi-layer evaluation suites, simulate multi-turn conversations, and establish reliability metrics, including pass@k metrics and staging validations. The role is...

Help a top AI lab evaluate and improve large language models through security-focused coding tasks. Bring your software-engineering judgment and hands‑on security experience to work involving vulnerabilities, exploit verification and security patches. This is a contracting engagement, with potential for a longer-term engagement. Remote candidates in the...

Mercor in Lisbon is seeking an Analyst / Insights professional with strong analytics, research, or insights experience. You will review, assess, and provide structured feedback on domain documents to ensure quality, accuracy, and relevance. Ideal candidates have a Bachelor’s degree and at least 2 years in a relevant field, with excellent Excel,...

YO AI Labs is seeking experienced Product Managers and Product Owners to contribute to a customer project focused on evaluating and improving AI systems. This remote contractor role emphasizes evaluation and analysis rather than roadmap ownership or delivery management. You will assess AI-generated content, specifications, release notes, and documentation,...

YO AI Labs is seeking a Creative & Design Specialist to support a customer project covering graphic design, multimedia production, UI/UX, motion graphics, and creative workflows. Remote, worldwide with a flexible schedule. You will evaluate AI-generated outputs and provide high-quality human feedback for AI training. Prior paid experience in human-data work...

YO AI Labs is seeking experienced Technical Writers / Editors for a contractor role to evaluate and improve AI-generated content. You will apply strong editorial judgment, assess clarity and tone, and provide actionable feedback to enhance model outputs. No prior AI experience required, but excellent English and analytical skills are essential. Work...

YO AI Labs is seeking experienced Product Managers and Product Owners to contribute to a customer project focused on evaluating and improving AI systems. This remote contractor role emphasizes evaluation and analysis rather than roadmap ownership or delivery management. You will assess AI-generated content, specifications, release notes, and documentation,...

YO AI Labs is seeking experienced Product Managers and Product Owners to contribute to a customer project focused on evaluating and improving AI systems. This remote contractor role emphasizes evaluation and analysis rather than roadmap ownership or delivery management. You will assess AI-generated content, specifications, release notes, and documentation,...

Obsidian is hiring an Analyst / Insights professional with experience in analytics, research, or insights roles. You will review, assess, and provide structured feedback on domain-specific documents to ensure quality, accuracy and relevance.This position suits professionals with strong analytical skills and domain expertise who can contribute to high-impact...

EPAM Systems, Inc. is seeking a Senior ML / Evaluation Engineer to design and implement advanced evaluation frameworks for an Enterprise Agent Development Platform. You will build LLM-as-judge evaluators and deterministic validators to enforce quality before release. You will collaborate with platform, orchestration, and Dev Ops teams to ensure scalable,...

We're looking for an Agent Evaluation Engineer — Build-Time Framework & Deployment Gates to join our team in Portugal in a fully remote working mode. In this role, you will design and maintain an evaluation framework for AI agents, ensuring quality and compliance through automated tests and CI/CD deployment gates. You will develop multi-layer evaluation...

EPAM Systems, Inc. is seeking a Senior ML / Evaluation Engineer to design and implement advanced evaluation frameworks for an Enterprise Agent Development Platform. You will build LLM-as-judge evaluators and deterministic validators to enforce quality before release. You will collaborate with platform, orchestration, and DevOps teams to ensure scalable,...

EPAM Systems, Inc. is seeking a Senior ML / Evaluation Engineer to design and implement advanced evaluation frameworks for an Enterprise Agent Development Platform. You will build LLM-as-judge evaluators and deterministic validators to enforce quality before release. You will collaborate with platform, orchestration, and DevOps teams to ensure scalable,...

EPAM Systems, Inc. is seeking a Senior ML / Evaluation Engineer to design and implement advanced evaluation frameworks for an Enterprise Agent Development Platform. You will build LLM-as-judge evaluators and deterministic validators to enforce quality before release. You will collaborate with platform, orchestration, and DevOps teams to ensure scalable,...

Concentrix in Lisbon, Portugal seeks an AI Labeling & Evaluation Agent (Italian-speaking) to join our on-site team. You will evaluate AI outputs, apply guidelines, and ensure high-quality annotations that improve LLM performance. The role requires fluency in Italian (C1) and strong English (B2). You’ll work with diverse data and contribute to global AI...

EPAM Systems, Inc. is seeking a Senior ML / Evaluation Engineer to design and implement advanced evaluation frameworks for an Enterprise Agent Development Platform. You will build LLM-as-judge evaluators and deterministic validators to enforce quality before release. You will collaborate with platform, orchestration, and DevOps teams to ensure scalable,...