Databricks + Pyspark (Indore)

Job Summary Responsibilities - Lead the development and implementation of clinical data systems using Scala and PySpark to ensure effective data processing. - Oversee the setup and maintenance of CI/CD pipelines in Azure DevOps to streamline development workflows. - Provide expertise in Azure Repos for version control and collaboration on code repositories. - Manage and optimize Databricks and Azure Databricks environments for advanced analytics and data engineering tasks. - Implement Azure Stream Analytics to process real-time data streams and integrate them into clinical systems. - Utilize Data Factory and Azure Data Factory for orchestrating data workflows and ensuring seamless data integration. - Leverage Azure Storage solutions to store and manage large volumes of clinical data securely. - Develop and maintain Azure Services to support various clinical data management and processing needs. - Collaborate with cross-functional teams to ensure the successful implementation of clinical systems. - Provide technical guidance and support to junior developers and team members. - Ensure compliance with industry standards and regulations in clinical data management. - Conduct regular code reviews and ensure adherence to best practices in software development. - Monitor system performance and implement optimizations to enhance efficiency and reliability. Qualifications - Possess a strong background in Scala and PySpark for data processing and analytics. - Demonstrate expertise in setting up and managing CI/CD pipelines in Azure DevOps. - Have extensive experience with Azure Repos for version control and collaboration. - Show proficiency in Databricks and Azure Databricks for data engineering and analytics. - Be skilled in implementing Azure Stream Analytics for real-time data processing. - Have hands-on experience with Data Factory and Azure Data Factory for data orchestration. - Be knowledgeable in Azure Storage solutions for secure data management. .