01Overview
1. Position Summary
The Data Engineering Lead is responsible for designing, building and operating data platforms and pipelines required for AI, analytics and automation use cases. The role leads a team of Data Engineers to implement the data architecture defined by the Architecture Lead and Solution Architect, ensuring reliable, high-quality and well-governed data for use cases such as Manufacturing Control Tower, Golden Batch, Yield Improvement, OOS prediction and pricing models.
2. Roles Played
Data Engineering team lead for AI and analytics initiatives.
Owner of data pipelines, transformations and data availability SLAs.
Key contributor to data modelling, data quality and governance practices.
3. Key Platforms / Technologies
Azure Data Lake, Data Warehouse, Synapse, Databricks (or equivalent).
ETL/ELT tools and orchestration frameworks.
Streaming and batch data ingestion frameworks.
BI environments (Power BI, Tableau) as data consumers.
Integration with SAP ECC or S/4 HANA , MES, LIMS, Empower, CTMS and other enterprise sources.
4. Overall Job Responsibilities
A. Data Platform & Pipeline Design ( 25%)
Design data ingestion, transformation and storage solutions as per architecture guidelines.
Build data models and structures to support AI, analytics and reporting use cases.
Ensure solutions are modular, reusable and performance optimized.
B. Data Engineering Delivery & Operations ( 25%)
Lead the implementation of data pipelines (batch and streaming) into the data lake/warehouse.
Set up and maintain orchestration, monitoring, alerting and logging for data flows.
Own day-to-day data operations and incident resolution related to pipelines.
C. Data Quality, Governance & Security ( 20%)
Implement data quality rules, validations and reconciliations.
Ensure data lineage and metadata capture for key datasets.
Work with Security and Governance teams to enforce access control, privacy and data integrity requirements.
D. Performance, Optimization & Cost Management ( 10% .
