The AI QA Engineer is responsible for implementing automated and manual testing for LLM-based applications, detecting biases and hallucinations, collecting / labeling testing data, collaborating with engineers to optimize model performance and ensure high-quality AI outputs.

• Perform manual testing for LLM-based applications, Automate workflows whenever possible
• Build and manage test datasets, ensuring high-quality, diverse, and balanced samples.
• Own dataset labeling and ground truth collection process
• Develop synthetic data pipelines to enhance model evaluation.
• Work closely with AI engineers to debug failures, identify root causes, and optimize model performance.
• Collaborate with Engineers to integrate testing into early-stage development
• Develop and execute AI model evaluation pipelines, ensuring accuracy, consistency, and fairness.
• Develop and maintain test reports, summarizing findings and recommendations.

• Experience with AI/ML testing frameworks and LLM evaluation methodologies.
• Experience with test dataset management and annotation tools.
• Familiarity with synthetic data generation and adversarial testing techniques.
• Strong understanding of LLM behaviors, biases, failure modes, and edge cases.
• Strong problem-solving and debugging skills to analyze AI failures and inconsistencies.
• Proficiency in Python and familiarity with ML testing frameworks (e.g., PyTest, Unittest, custom ML evaluation tools).
• Strong English language proficiency with the ability to evaluate AI-generated text and improve prompts.

All interested candidates are encouraged to apply through this form.
We highly appreciate all applications, however, only shortlisted candidates will be contacted for the next stages.