01Overview
We are looking for a Senior ETL Data Engineer with 8+ years of experience to design, build, and maintain robust data pipelines and scalable data architectures. In this role, you will work closely with cross-functional teams to transform complex raw data into structured, reliable datasets for analytics and business intelligence.
Key Responsibilities
Design, build, and optimize end-to-end data pipelines and ETL workflows using Azure Data Factory (ADF).Develop efficient, scalable data processing scripts and data transformations using PySpark.Build and manage enterprise data warehouse solutions in Azure Synapse Analytics.Implement high-performance data storage and querying patterns using Delta Lake architecture.Write complex, optimized SQL queries, stored procedures, and transformations for data modeling and analysis.Monitor, troubleshoot, and optimize data pipeline performance, ensure data quality, and resolve data bottlenecks.Mandatory Qualifications & Requirements
PySpark: Core proficiency in writing PySpark code for large-scale distributed data processing.Azure Data Factory (ADF): Hands-on experience creating, orchestrating, and monitoring complex ETL/ELT pipelines.Azure Synapse Analytics: Proven experience in designing data warehouses and running distributed analytical queries.SQL: Advanced proficiency in SQL for data manipulation, schema design, and query optimization.Delta Lake: Practical experience implementing ACID transactions, schema enforcement, and time travel features on lakehouse storage.Preferred Skills
Strong understanding of Data Warehousing principles (Dimensional Modeling, Star/Snowflake Schemas).Experience with Git, CI/CD pipelines, and DevOps principles for data workflows.Familiarity with data governance, security, and access control in Azure environments.Compensation: 700,000.00 - 1,500,000.00 per year
Experience:
Pyspark: 6 years (Required)Synapse: 6 years (Required)ADF: 6 years (Required)Delta Lake: 6 years (Required)Work Location: Hybrid remote in Chennai, Tamil Nadu .
