01Overview
Job Summary : As a Python Developer with Databricks, you will be responsible for developing and maintaining scalable data pipelines, managing cloud environments on Azure, and ensuring smooth integration with APIs.The ideal candidate will be proficient in Python, Databricks (PySpark), and Azure DevOps, with a strong understanding of cloud services, DevOps practices, and API testing.Key Responsibilities : - Develop and Maintain Data Pipelines : Design, develop, and maintain scalable data pipelines using Python and Databricks (PySpark).- Data Processing : Apply strong proficiency in Python and advanced Python concepts to process and manipulate large datasets effectively.- API Ingestion : Experience in API ingestion, working with data in JSON format to integrate and automate data workflows.- Cloud Management : Use Azure Portal for managing cloud environments and services.- Databricks PySpark : Work with Databricks and PySpark to build distributed data processing applications.- DevOps & Agile Methodology : Implement DevOps best practices and work within a Scrum framework to ensure continuous integration and continuous delivery (CI/CD) pipelines.- API Testing & Automation : Use Postman to test and automate APIs for robust integration and data workflows.- Collaboration : Work closely with cross-functional teams to implement solutions aligned with business objectives and technical requirements.Required Qualifications : - Programming Skills : Strong proficiency in Python with experience in data processing libraries (e.g., Pandas, NumPy).- Databricks Experience : Hands-on experience with Databricks (PySpark) for data processing and analysis.- Cloud Platform : Experience using Azure Portal to manage cloud environments and services.- API Handling : Expertise in working with APIs, specifically with data ingestion and integration in JSON format.- DevOps Methodology : Familiarity with DevOps practices and experience working in Agile/Scrum environments.- API Testing Tools : Proficiency with Postman for API testing and automation. (ref:hirist.tech) .
