About the Role

We are looking for a rigorous and detail-oriented GenAI & Data Quality QA Engineer to ensure the reliability, accuracy, and safety of our systems. This role is unique because it blends traditional software quality assurance (finding bugs in the application UI, backend, and databases) with cutting-edge AI testing (evaluating Generative AI outputs, handling hallucinations, and testing data pipelines).

You will be the final line of defense, ensuring that our AI models provide trustworthy, safe, and accurate responses, while also ensuring the underlying software platform runs flawlessly.

Key Responsibilities

1. Generative AI & Model Testing

• Evaluate AI Outputs: Validate Generative AI features (chat interfaces, summarization, recommendations) for accuracy, relevance, and consistency.
• Prompt & Response Testing: Design and execute tests to evaluate edge cases, ambiguity handling, and failure scenarios.
• Safety & Compliance: Identify and report issues related to AI hallucinations, incorrect reasoning, bias, fairness, and incomplete responses.
• Adversarial Testing: Conduct "red-teaming" or adversarial testing to identify vulnerabilities, jailbreaks, and edge cases in LLMs and AI workflows.

2. Data Quality & Pipeline Validation

• Data Validation: Analyze and investigate patterns, inconsistencies, and rule violations within datasets used to train or prompt AI models.
• Pipeline Testing: Validate data pipelines to ensure data accuracy, completeness, and consistency from ingestion to the consumption layer.
• RAG Testing: Evaluate Retrieval-Augmented Generation workflows to ensure the AI is pulling the correct context from our databases.

3. Traditional System & API QA

• Functional Testing: Test web and application interfaces to ensure functional correctness, cross-platform compatibility, and proper UI behavior.
• Backend Validation: Test APIs for functionality, validation, error handling, performance, and security.
• Integration: Perform system integration testing, ensuring seamless interaction between AI models, backend systems, and external applications.
• Defect Management: Identify, track, and manage software bugs and data issues through to resolution.

4. Automation & Strategy

• Test Automation: Design, implement, and maintain automated test scripts for APIs, UI components, and AI data validations using standard automation frameworks.
• AI-Assisted QA: Leverage GenAI tools to accelerate test case generation and data creation.
• Continuous Improvement: Collaborate with product managers, developers, and data scientists to refine testing strategies, improve model quality, and optimize delivery pipelines.

Requirements

Experience:

• 2+ years of experience in software Quality Assurance, with at least 1-2 years specifically focused on AI/ML applications, data testing, or NLP systems.
• Proven experience in Agile/Scrum environments.

Technical Skills:

• AI/ML Knowledge: Strong understanding of Generative AI, Large Language Models (LLMs), non-deterministic outputs, and Retrieval-Augmented Generation concepts.
• Automation: Proficiency in modern test automation frameworks and methodologies.
• Programming: Strong scripting and programming skills to support test automation efforts.
• Data & Databases: Solid database querying skills for data validation and analysis. Familiarity with data warehousing and pipeline concepts.
• API Testing: Hands-on experience testing APIs using standard industry tools or automation libraries.

Soft Skills:

• Strong analytical and problem-solving skills, with a high attention to detail (evaluating AI requires nuance beyond simple pass/fail).
• Ability to clearly communicate AI risks, hallucinations, and software defects to both technical and non-technical stakeholders.

Preferred Qualifications:

• Bachelor’s degree in Computer Science, Data Science, or a related field.
• Familiarity with cloud platforms and big data tools.
• Industry-recognized certifications in software testing.