01Key Responsibilities
1. Design and Development:
o Design, develop, and optimize large-scale data processing pipelines using
Databricks on Azure.
o Implement ETL processes to ingest and transform data from various sources into
our data platform.
o Develop scalable and efficient data models to support analytics and reporting
needs.
2. Azure Integration:
o Integrate Databricks with Azure services such as Azure Data Lake, Azure SQL
Database, Azure Synapse Analytics, and Azure Machine Learning.
o Utilize Azure DevOps for continuous integration and continuous deployment
(CI/CD) of data solutions.
3. SaaS Product Exposure:
o Collaborate with product teams to understand data requirements and integrate
SaaS product data into the data platform.
o Develop data solutions that support SaaS product analytics and reporting.
4. Performance Optimization:
o Monitor and optimize the performance of data pipelines and queries.
o Implement best practices for data governance, security, and compliance.
5. Collaboration and Leadership:
o Work closely with data scientists, analysts, and other stakeholders to understand
data needs and deliver solutions.
o Mentor and guide junior engineers, promoting best practices and knowledge
sharing within the team.
6. Documentation and Reporting:
o Maintain comprehensive documentation of data engineering processes, designs,
and solutions.
o Generate reports and dashboards to provide insights into data platform
performance and usage.
Qualifications:
Education:
o Bachelors or Masters degree in Computer Science, Engineering, or a related
field.
Experience:
o Minimum of 8-10 years of experience in data engineering or related roles.
o Extensive experience with Databricks and Azure.
o Proven experience with SaaS products and integrating their data into data
platforms.
Technical Skills:
o Proficiency in Spark, SQL, Python, and Scala.
o Strong understanding of Azure services and architecture.
o Experience with data warehousing, ETL/ELT processes, and data modeling.
o Knowledge of CI/CD practices and tools, particularly Azure DevOps.
Soft Skills:
o Excellent problem-solving and analytical skills.
o Strong communication and collaboration abilities.
o Ability to work independently and lead projects.
Preferred Qualifications:
Certifications in Databricks, AWS, Azure, or related Cloud technologies.
Experience with machine learning and AI integration.
Familiarity with data visualization tools like Power BI or Tableau.
Benefits:
Competitive salary and performance-based bonuses.
Comprehensive health insurance including Parents
Complementary Buffet Meal Everyday
Professional development opportunities and continuous learning.
Join our team and contribute to the development of high-quality business applications. .
