Job Title: QA Engineer – Generative AI Testing

Location: Hybrid – Hyderabad/Vishakhapatnam

Job Type: Full-time

Job Summary

We are seeking a skilled and detail-oriented QA Engineer with expertise in Generative AI Testing to join our team. In this role, you will be responsible for ensuring the accuracy, safety, and performance of AI-driven applications, particularly in the field of Generative AI. You will design and execute test strategies, develop automation frameworks, and validate AI outputs to maintain high-quality user experiences.

Key Responsibilities

• Design, develop, and execute comprehensive test plans for Generative AI models and applications.
• Validate AI-generated outputs for accuracy, coherence, bias, ethical considerations, and alignment with business requirements.
• Develop and maintain automated testing frameworks for AI-based applications, ensuring scalability and efficiency.
• Perform adversarial testing, edge case analysis, and security testing to assess AI vulnerabilities.
• Collaborate with AI/ML engineers, product managers, and developers to refine model performance and mitigate errors.
• Monitor AI model drift and ensure consistent performance across different inputs and datasets.
• Conduct performance testing to evaluate response times, scalability, and reliability of AI systems.
• Document test cases, test results, and defects while ensuring compliance with AI governance and ethical guidelines.
• Continuously explore new tools and methodologies to improve AI testing processes.
  
  

Requirements

• Bachelors or Master’s degree in Computer Science, Engineering, or a related field.
• 3+ years of experience in Quality Assurance, with at least 1-2 years in AI/ML testing or Generative AI applications.
• Strong understanding of AI/ML concepts, NLP models, LLMs, and deep learning architectures.
• Experience with AI testing tools such as LangTest, DeepChecks, Trulens, or custom AI evaluation frameworks.
• Proficiency in test automation using Python, Selenium, PyTest, or similar frameworks.
• Familiarity with model evaluation metrics such as BLEU, ROUGE, perplexity, and precision-recall for AI-generated content.
• Knowledge of bias detection, adversarial testing, and ethical AI considerations.
• Experience working with APIs, cloud platforms (AWS, Azure, GCP), and MLOps practices.
• Strong analytical and problem-solving skills with attention to detail.
• Excellent communication and collaboration skills to work in cross-functional teams.
  
  

Nice To Have

• Experience in testing AI chatbots, voice assistants, or image/video-generating AI.
• Knowledge of LLM fine-tuning and reinforcement learning from human feedback (RLHF).
• Exposure to regulatory and compliance frameworks for AI governance.
  
  

Why Join Us?

• Opportunity to work on cutting-edge AI products.
• Collaborate with a team of AI and software experts.
• Competitive salary, benefits, and career growth opportunities.
  
  

If you’re interested, Please share your resume with [email protected]