01Responsibilities
Promote DataOps approach to Data science, engineering and analytics delivery processes to automate the provision of data, testing and monitoring and shorten CI/CD.
Collaborate with data & ML leads and create and build optimal data pipeline architecture for the data solutions including data science products
Ensure the data pipelines are scalable and performant as well as creating and maintaining service to connect data products
Create dashboards and other tools required to efficiently monitor our data and ML infrastructure, pipelines, ETL and analytics delivery processes.
Building end-to-end event instrumentation and alerting system to detect and alert any anomaly in the system or in the data
Assist in managing our data and ML infrastructure (upgrading, monitoring, optimising)
Collaborate with IT DevOps engineers and participate in enterprise DevOps activities.
Exchange your knowledge on infra and data standards with other developers and be part of our tech community. Promote the use of engineering best practices.
Contribute to innovative POCs with our data & engineering teams.
Remain flexible towards technology approaches to ensure that the best advantage is being taken by recent technologies.
Required skills & Qualifications:
Strong drive to solve problems, communicate clearly and contribute positively to a DevOps/DataOps culture
Knowledge of the latest DevOps tools and practices.
Experience with data pipelines within AWS (Glue, DataPipeline, Athena, EMR, DMS, Spark)
Experience of Database Replication and databases like Aurora, MySQL, MariaDB, etc.
Efficient in building CI/CD pipelines for containerized Java/Python codestack
Comfortable with Git workflow.
Experience with applications deployed in AWS.
Experience with configuration management and provisioning tools (e.g., Ansible, CloudFormation, Terraform)
Knowledge of one or more scripting languages - Bash/Python/JavaScript
Orchestration/containerisation using Docker and Kubernetes.
Basic knowledge of data science & ML engineering
Bachelor's Degree in computer science or similar degree or Big Data Background from top tier universities.
Experience: 10+ years of experience. .
