01Overview
Job Title: Azure Data Engineer-Insurance is a Plus
Experience: 710 Years
Location: Delhi/NCR (Hybrid)
Employment Type: Full-Time
About the Role
We are looking for a highly skilled Azure Data Engineer with 710 years of experience in designing, developing, and maintaining scalable data solutions on Microsoft Azure. The ideal candidate should have strong expertise in Azure Data Factory, Azure Databricks, PySpark, Python, SQL, and modern data warehousing solutions. You will be responsible for building robust ETL/ELT pipelines, optimizing data platforms, and enabling enterprise analytics.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines using Microsoft Azure services.
- Build and optimize data ingestion and transformation pipelines using Azure Data Factory (ADF) and Azure Databricks.
- Develop PySpark notebooks for large-scale data processing.
- Design and manage data lakes using Azure Data Lake Storage Gen2 (ADLS Gen2).
- Develop scalable data warehouse solutions using Azure Synapse Analytics.
- Implement data quality, validation, monitoring, and logging frameworks.
- Optimize Spark jobs for performance, scalability, and cost efficiency.
- Collaborate with business stakeholders, architects, and cross-functional teams to gather requirements and deliver high-quality data solutions.
- Implement CI/CD pipelines using Azure DevOps.
- Troubleshoot production issues and ensure high availability of data pipelines.
- Follow best practices for security, governance, and data lifecycle management.
Mandatory Skills
- 710 years of experience in Data Engineering.
- Strong programming skills in Python.
- Excellent knowledge of SQL.
- Hands-on experience with PySpark / Apache Spark.
- Strong expertise in Azure Data Factory (ADF).
- Hands-on experience with Azure Databricks.
- Experience with Azure Data Lake Storage Gen2 (ADLS Gen2).
- Experience with Azure Synapse Analytics.
- Robust understanding of ETL/ELT, Data Warehousing, and Data Lake concepts.
- Experience working with Azure DevOps and Git. .
