AI-Native Testing

Design and execute test scenarios for AI-enabled applications, AI agents, copilots, chat-based experiences, and AI-assisted workflows. Validate AI outputs for accuracy, relevance, groundedness, consistency, hallucination risk, and responsible AI considerations. Create prompt-based test scenarios, reusable evaluation datasets, and regression packs for AI and GenAI use cases. Perform functional, integration, regression, exploratory, defect validation, and end-to-end business process testing. Design and execute API test cases for REST/SOAP services, including request and response validation, schema validation, error handling, and negative scenarios. Use tools such as Postman, Swagger/OpenAPI, REST Assured, or Playwright API testing for API validation and automation. Validate basic ETL and data flow scenarios including source-to-target mapping, transformation logic, reconciliations, row counts, duplicates, and data quality checks. Write SQL queries to support data validation, reporting checks, and issue analysis across source, staging, and target layers. Develop and maintain automation scripts for UI, API, and data validation using Playwright, Python, TypeScript, or similar tools. Collaborate with product owners, developers, data engineers, AI engineers, and business stakeholders to clarify requirements and drive quality early in the lifecycle. Log, track, and retest defects using tools such as JIRA, Azure DevOps, ALM, or equivalent defect management platforms. Support CI/CD, shift-left testing, and continuous improvement by identifying opportunities to use AI and automation accelerators. Preferred Skills: Exposure to AI evaluation frameworks such as prompt evaluation, RAG evaluation, LLM benchmarking, or human feedback-based evaluation. Basic understanding of RAG, embeddings, vector search, knowledge retrieval, and agentic workflows. Knowledge of cloud platforms such as Azure and DevOps practices. Experience in test data creation, synthetic data generation, or data quality automation. Ability to use AI tools responsibly to accelerate requirement analysis, test design, automation, and defect analysis. Education / Professional Qualifications: Bachelors or Masters degree in Computer Science, Information Technology, Engineering, or a related discipline. Relevant certifications in software testing, automation, cloud, data engineering, or AI/GenAI testing will be an added advantage. .