01Overview
We are looking for a skilled Data Engineer with robust hands-on experience in Azure and Databricks to build scalable data pipelines and optimize data workflows. The ideal candidate will be responsible for designing ETL/ELT processes, transforming and enriching data, handling structured and unstructured datasets, and supporting business analytics through productive data modeling and performance tuning.
MUST HAVE AZURE CERTIFICATION
Key Responsibilities
Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks
Extract, transform, and load data from multiple data sources (structured & unstructured)
Clean, validate, and enrich data at various processing stages
Develop and optimize data models (dimensional modeling, star schema, etc.)
Monitor, troubleshoot, and improve performance of data pipelines and SQL queries
Ensure data quality, consistency, and governance standards
Collaborate with BI, analytics, and business stakeholders to understand requirements
Implement best practices for performance tuning and cost optimization in Azure
Required Skills & Qualifications
Strong hands-on experience with Azure Databricks
Valuable understanding of Azure Data Lake, Azure Data Factory, and related Azure services
Strong SQL skills (complex joins, aggregations, query optimization)
Experience with PySpark / Apache Spark
Experience building and maintaining ETL/ELT workflows
Knowledge of data modeling concepts
Experience handling unstructured textual data
Strong analytical and problem-solving skills
Valuable to Have
Experience with Delta Lake
CI/CD experience for data pipelines
Knowledge of data governance and security best practices
Exposure to cloud data warehousing solutions .
