Přejít na obsah

Senior AI Quality/Evaluation Engineer

IDC Central Europe, the Middle East & Africa

  • Místo práce: Malé náměstí 459/11, Staré Město, Praha, Hlavní město Praha
  • Zveřejněno: před 4 dny
  • zdroj: Jobs.cz
  • Zkušený specialista (Senior)

Popis nabídky

IDC is the premier global provider of market intelligence, advisory services, and events for the information technology, telecommunications, and consumer technology markets. IDC helps IT professionals, business executives, and the investment community make fact-based decisions on technology purchases and business strategy.

About the Role & Team

IDC is building the next generation of AI-powered intelligence platforms that transform how technology decisions get made. Our platform re-imagines the way decision-makers discover and interact with trusted research and data, and is foundational to IDC's future.

We are looking for a Senior AI Quality/Evaluation Engineer to establish the evaluation function for the platform's AI systems. This is a solo function initially. You will design and build the evaluation infrastructure that ensures the platform produces accurate, well-sourced, high-quality responses. You will be the first hire in this function and must be able to operate independently, defining your own roadmap and building from scratch.

The platform's credibility depends on the quality of its AI-generated intelligence. You will build the automated test suites, regression detection systems, and evaluation frameworks that catch quality issues before they reach users. You will work closely with the product team to translate quality criteria into measurable, automatable test scenarios, and with the AI engineering team to ensure that every pipeline change is evaluated against rigorous standards.

What You’ll Do

  • Design and build the evaluation infrastructure that ensures the platform's AI systems produce accurate, well-sourced, high-quality responses
  • Build automated test suites that validate answer quality across agent pipeline changes
  • Develop regression detection systems that catch quality degradation before it reaches users
  • Create evaluation frameworks that measure response accuracy, citation correctness, and source quality
  • Work closely with the product team to translate quality criteria into measurable, automatable test scenarios
  • Build cost and latency monitoring that tracks the operational efficiency of AI pipeline execution
  • Define evaluation standards and practices that scale as the platform and team grow

What You Bring

  • 6+ years of software engineering experience, with significant work in testing infrastructure, ML evaluation, or quality systems
  • Experience building evaluation or testing frameworks for LLM-based or ML-based systems
  • Understanding of how to measure response quality for generative AI: accuracy, groundedness, citation correctness, relevance
  • Proficiency in Python
  • Ability to operate independently and define your own roadmap. You will be the first hire in this function
  • Experience working at the intersection of engineering and product, translating qualitative quality criteria into quantitative measurements

Preferred Qualifications

  • Experience with LLM evaluation frameworks (e.g., RAGAS, DeepEval, or custom)
  • Familiarity with LLM observability tools (e.g., Langfuse, LangSmith, Weights & Biases)
  • Background in statistical methods for quality measurement (significance testing, distribution analysis)
  • Experience building A/B testing or experimentation infrastructure
  • Background in search relevance evaluation or information retrieval metrics

What We Offer

  • 5 weeks of holidays + extra corporate days off
  • Full time, permanent position offered in hybrid (Prague or Ostrava office, 2 - 3 days office) or remote model.
  • Flexibility to work from home most of the week
  • Sick days
  • Certain flexibility to schedule your working hours
  • Cafeteria system (use points on Flexipasses, pension/life insurance, or Multisport card)
  • Meal allowance
  • Compensation Transparency

At IDC, we are committed to fair and equitable pay practices. Employees are compensated equitably for their work, aligned with their skills and experience. Salary and incentive structures are determined through a rigorous process that considers experience, education, certifications, role-specific requirements, internal equity, and verified U.S. market data from an independent third-party partner.

Equal Opportunity Employer

IDC is committed to providing equal employment opportunities for all qualified persons. Employment eligibility verification required. We participate in E-Verify.

Požadavky a benefity

Benefity

  • 5 weeks of holidays + extra corporate days off
  • Certain flexibility to schedule your working hours
  • Compensation Transparency
  • Flexibility to work from home most of the week
  • Meal allowance
  • Sick days

Podrobnosti nabídky

Obor
Technika a inženýrství
Úroveň pozice
Zkušený specialista (Senior)

Další informace z inzerátu

Místo pracoviště
Malé náměstí 459/11, Praha – Staré Město

Analýza TXO

Co jednotlivý portál neukáže: trh, historie a další inzeráty této pozice.

Nabídka v číslech

Sazba
neuvedeno
Nábor trvá
od dneška
první zjištění: 16. listopadu 2026
Portály
1
Jobs.cz

Senior AI Quality/Evaluation Engineer (IDC Central Europe, the Middle East & Africa, Praha): inzerát neuvádí mzdu. TXO tuto nabídku poprvé zachytilo dnes.

Časová osa náboru

Nábor začal dnes

  1. 16. listopadu 2026

    Nabídka se objevila na Jobs.cz

    IDC Central Europe, the Middle East & Africa

Na co se zeptat u pohovoru

  1. Jaký je rozpočet na tuto pozici a v jaké formě smlouvy?
  2. Jakou formu smlouvy nabízíte?
  3. Je práce v kanceláři, hybridní, nebo na dálku?
  4. Jaký je pracovní úvazek a v jakých hodinách se pracuje?
  5. Jak dlouhá je zkušební doba a za jakých podmínek?
  6. Od kdy by práce začínala?

Místo práce

Malé náměstí 459/11, Staré Město, Praha, Hlavní město Praha přibližné

O zaměstnavateli

IDC Central Europe, the Middle East & Africa

Praha · 7 nabídek práce na webu

Všechny nabídky zaměstnavatele

ID nabídky: 81718Zobrazení: 0Zveřejněno 12. listopadu 2026Zpět na výsledky

Mohlo by tě zajímat

Podobné nabídky práce

Všechny nabídky
Logo: DX Heroes

Applied AI Engineer (Mid/Senior)

DX Heroes

Praha

400 - 600 Kč / hod.

57% nad trhem

  • Práce na pracovišti
  • Hlavní pracovní poměr
  • Práce na IČO / živnost
  • Zkušený specialista (Senior)
  • Plný úvazek

před 25 dnyStartupJobs.cz

Logo: ManpowerGroup

Network Engineer (Medior)

ManpowerGroup

Praha

Mzda neuvedena
  • Hlavní pracovní poměr
  • Specialista (Medior)
  • Plný úvazek
  • Dobrou znalost TCP/IP, routingu a switchingu
  • Minimálně 3 roky zkušeností v oblasti síťové infrastruktury
  • Praktické zkušenosti s technologiemi Cisco nebo obdobnými enterprise řešeními
  • +2

před 21 dnyManpower CZ

Logo: Daktela s.r.o.

Senior QA Automation Engineer (API & Flutter)

Daktela s.r.o.

Praha

80 000 - 110 000 Kč / měs.

77,6% nad trhem

  • Hybridní práce
  • Hlavní pracovní poměr
  • Práce na IČO / živnost
  • Zkušený specialista (Senior)
  • Plný úvazek
  • Appium
  • BMC Patrol
  • CI/CD
  • +3

před 28 dnyStartupJobs.cz

Elektroprojektant Senior

ICE Industrial Services

Praha

Mzda neuvedena
  • Zkušený specialista (Senior)
  • Plný úvazek

29.01.2026Cocuma

Nenašel jsi, co hledáš? Dáme ti vědět e-mailem, až se objeví nové: nabídky práce pro inženýry a techniky v lokalitě Praha.

Mzda neuvedena

IDC Central Europe, the Middle East & Africa

Odpovědět