01Overview
Job Title : DataOps Technical Lead Databricks/AzureWork Location : Bangalore / Chennai / Gurgaon/ Pune/ Kolkata / HyderabadDivision/Department : Solution SupportEmployment Type : Full-timeJob Description, Essential Duties and Responsibilities : DataOps Lead is responsible for leading technical team and managing end-to-end project deliverables.Should have strong experience and knowledge in DataOps support deliverables, KPIs and platform administration for Azure-Databricks DataOps project.Data Ops & Support roles : - Pipeline Monitoring & Incident Response- Ticket/Incident Management- Data Anomaly Investigations & Root Cause Analysis- Hotfixes, Enhancements & Release Coordination- Creating Runbooks and Knowledge Base documents- Data Quality TrackingPlatform Administration : - Databricks Platform Administration- Azure Infrastructure Management- Identity, Access & Governance- Data Pipeline Architecture & Platform StandardsTools & Technologies : - Databricks- Azure data services- SQL Server- Azure Synapse- Azure Data Factory (ADF)- Python and PySpark scripting using DLTs- Github- AI based tools (Claude, Co-Pilot etc)- Linear (task management)- ServiceNow- Cosmos- ADOKey Responsibilities : - Data pipelines : Monitor, identify issue, Debugging and Bug fixing of data pipelines. Should understand complex data pipelines and data engineering concepts. Mentoring and guiding junior team members.- Data Quality & Data Observability : Identify and fix the Data quality issues by performing RCA. Should have a good understanding of data observability tools like Unravel etc.- Data orchestration : Orchestration of jobs using Airflow.- DevOps : Should have a good understanding of the Dev Ops process.- Collaboration : Work closely with data engineering, data science, and IT teams to support data pipeline development, troubleshoot issues, and provide guidance on best practices.- Incident Management : Lead troubleshooting and resolution of issues related to Databricks clusters and jobs, ensuring minimal downtime and impact on business operations.- KPI : Responsible for managing MTTA, MTTR, IRR, response & resolution SLA and other key delivery matrix.- Upgrades & Patches : Manage upgrades, patches, and new releases of Databricks, ensuring minimal disruption to ongoing operations.- Documentation : Create and maintain detailed documentation of the Databricks environment, configurations, and procedures.- SOP : Able to build SOPs and set up DataOps best practices.- Continuous improvement and Innovation : Bringing continuous improvement and innovation for cost optimization, effort reduction, incidents reduction, platform stability and discuss with client, Development team for implementation.- Team Leading : Lead the technical team and manage day-to-day technical deliverables.- Escalation : Manage clients day-to-day technical escalation.- Stakeholder management : Able to handle key stakeholders, including client leadership.Skills, Education and Experience : - Having 8+ years of Azure-Databricks DataOps experience.- Team handling experience : Able to handle a team of 5-7 engineers.- Open to work in UK & US Shift.- Working in agile methodology and Sprint framework.- Must have education degree of B. Tech/B.E/MCA (ref:hirist.tech)
