01Key Responsibilities
- Create and maintain optimal data pipeline architecture from multiple data sources.
- Assemble large, complex data sets that meet functional / non-functional business requirements.
- Identify, design, and implement internal process improvements: automating manual processes, optimising data delivery, re-designing infrastructure for greater scalability, etc.
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources
- Ensure robust data validation and verification processes to ensure the correctness of data for reporting.
- Develop and maintain data warehousing approaches, including the modelling and pipelining of data for storage and retrieval.
- Keeping data separated and secure as per best practice guidelines
- Ensure that the data taken from the information sources is accurate, liaising with other business functions when necessary
- Proactively keep up to date with the latest cloud data technology
- Provide support, documentation and training to other applicable team members as and when required to ensure that overall objectives are met
Skills, Knowledge, and Experience Required:
- Experience with building and maintaining ETL/ELT pipelines as part of an automated workflow on high-volume, high-dimensionality data from varying sources
- Good understanding on how data management, cleansing and query optimisation influences pipeline and data model design
- Experience using cloud technologies (AWS preferred)
- Experience with scripting languages (e.g. Python)
- Demonstrably strong problem solving skills to help design and implement solutions to data problems.
- Experience of CICD technologies and Source control eg Liquibase, Jenkins
- Understand and apply infrastructure knowledge to develop and maintain an efficient environment.
- Modelling datasets in a way applicable for use in different visualisation tools (e.g. Power BI)
- Strong stakeholder management skills
- Highly numerate and logical
- Ability to clearly communicate and present outputs to stakeholders.
- Experience of working in fast paced environments to strict deadlines.
- Team player and can communicate effectively to technical and non-technical colleagues
- Passionate about continuous improvement, collaboration and knowledge sharing at all levels
- The ability to manage time, prioritise tasks, self-review your work and produce deliverables of a high quality under tight client deadlines in time pressured environments
- It would be useful if youve got experience with some of .
