Data Test Lead

Job Title: Automation Test Lead Data & Platform Engineering **Overview We are seeking a highly hands on Automation Test Lead who can architect and scale enterprise grade test automation frameworks across data platforms, APIs, and cloud native systems. This role is not about managing testing. It is about owning quality engineering as a discipline. You will design frameworks that teams adopt, enforce standards that prevent bad releases, and build systems that proactively catch failures before they hit production. You should be equally comfortable writing Python frameworks, designing data validation strategies, and challenging engineers on weak implementations. Also embrace AI ways of working wherever possible to accelerate automation. Core Expectations You do not wait for bugs, you design systems that make bugs difficult to exist You build frameworks that multiple teams adopt without friction You challenge architecture, not just test outcomes You reduce false positives, flaky tests, and wasted compute You think in terms of data reliability, contracts, and system behaviour under stress Key Responsibilities Test Automation Architecture & Strategy Own the design and evolution of a scalable, modular, metadata driven test automation framework supporting: Data pipelines (batch and streaming) APIs and backend services End to end data product validation Enable plug and play components, parallel execution, environment isolation, and deterministic runs. Data Testing Framework Engineering Build reusable frameworks supporting SQL based assertions and reconciliation Schema validation and evolution checks Data contracts and producer consumer validation Lineage and freshness validation Config driven test definitions (YAML or JSON) Drive adoption by making frameworks easy to integrate into existing pipelines. Break Systems Proactively (Destructive Testing Mindset) Design tests for Schema drift and backward incompatibility Late arriving data and partial failures Duplicate, missing, or out of order events High volume stress and concurrency issues Failure scenarios such as retries, DLQ handling, and backpressure ETL and Streaming Validation at Scale Implement Row, aggregate, and hash based reconciliation Incremental and backfill validation Exactly once or at least once semantics validation Window based and time based correctness checks Data Quality & Observability Integration Integrate and extend tools like: Great Expectations, Soda Build custom validations for: Accuracy, completeness, uniqueness, timeliness Expose quality metrics, SLAs, and test outcomes through dashboards. CI/CD & DataOps Enforcement Embed testing into pipelines Pre merge gates and release blockers Selective and parallel test execution Integration with GitHub Actions, Jenkins, or similar Ensure no pipeline reaches production without passing quality gates. Test Data Management Define strategies for Synthetic data generation Masking and anonymization Deterministic datasets for repeatable testing Edge case and boundary condition simulation Performance & Reliability Testing Design and execute Pipeline and query performance benchmarks Concurrency and stress testing Data skew and partitioning analysis Continuously optimize for cost and execution time. Security & Compliance Validation Automate checks for PII or PHI exposure Encryption and access control Data retention and audit requirements Support compliance frameworks like GxP, SOX, or ISO standards. Cross Functional Quality Leadership Work with data engineers, platform teams, and architects to embed quality early Challenge poor design decisions that impact reliability Mentor engineers to adopt strong testing practices Incident Analysis & Prevention Analyze production data issues and failure patterns Reduce flaky tests and false alarms Drive root cause fixes, not surface level patches Mandatory Skills Programming & Framework Development Strong Python with experience building test frameworks, libraries, and CLI tools Advanced SQL for validation and reconciliation Data Engineering Ecosystem Hands on with DBT, Airflow, and Snowflake or similar platforms Strong understanding of ETL or ELT and data modeling concepts Data Quality & Observability .