Senior Data Engineer

Overview of Role

Data Engineer: Ability to manipulate data promptly in response to issues, providing immediate and

effective solutions to data-related challenges, and having a thorough understanding of data

structures, pipelines, and processes. The Data Engineer plays a critical role in supporting data-driven

decision-making and maintaining seamless data operations.

 

Responsibilities:

Data Pipeline Management: Design, develop, and maintain robust data pipelines that

facilitate efficient data processing, transformation, and loading. Optimize these processes

for performance and scalability.

ETL Processes: Architect and implement Extract, Transform, Load (ETL) processes to

integrate and transform raw data from various sources into meaningful, usable formats for

data analytics.

Data Quality Assurance: Implement data quality checks and validation processes to ensure

the integrity and consistency of data. Identify and resolve data anomalies and discrepancies.

Scalability and Performance: Continuously monitor and enhance data processing systems to

ensure they meet the growing needs of the organization. Optimize data architectures for

speed and efficiency.

Innovation and Improvement: Stay updated with the latest industry trends and

technologies. Proactively suggest improvements to data systems and processes to enhance

efficiency and effectiveness. And make sure that do not impact the pipeline and other

technical processes and conflicts should not be occurred.

Documentation and Compliance: Maintain comprehensive documentation of data

processes, architectures, and workflows. Ensure compliance with data governance and

security policies.

Technical Skills

Data Processing Tools: Proficiency in data processing tools such as PySpark and Pandas for

large-scale data manipulation and analysis.

Databricks: Knowledge of Databricks for collaborative data engineering, data processing.

Automation and Templates: Experience with Python for scripting templates and automation

scripts.

Cloud Platforms: Experience with cloud platforms [e.g., AWS, Azure, GCP(Google Cloud Platform)] for data storage and processing.

 

Soft Skills/Business Skills

Problem-Solving: Strong analytical skills with the ability to diagnose issues and develop

effective solutions quickly.

Continuous Learning: Enthusiastic about learning new technologies and staying updated

with industry trends to drive innovation in data engineering practices.

Adaptability: Flexible and adaptable to changing project requirements and priorities.

Capable of handling multiple tasks and projects simultaneously.

Team Collaboration: Ability to work collaboratively in a team environment and contribute to

cross-functional projects.

Communication: Excellent verbal and written communication skills to effectively convey

technical information to non-technical stakeholders.

 

Qualifications & Certifications

Any Graduate with Computer Science Background.

Mandatory certifications in Python and SQL. Additional certifications in cloud technologies

or data engineering are preferred.

Good to have certification on Databricks and Data Engineering Concepts

 

Experience Required

3 to 5 years of experience in data engineering, with a strong focus on data pipeline

development, ETL processes, data warehousing, and templates. Experience in working with

cloud-based data systems.

What you can expect from us

We appreciate that individual growth is important as well and we support you on every aspect of

personal development.

Training in Databricks.

Support on certifications.

Hands-on AWS, Azure, and GCP.

Support you on any cloud certification.

Build leadership skills.

 

For regular job updates kindly join our company LinkedIn group.
https://www.linkedin.com/groups/14581025/

Apply through Google Form:
https://lnkd.in/deEKDswb