01Responsibilities
Must have / Primary Skills / Mandatory
Hands-on experience building data pipelines on Hadoop/Spark ecosystem Strong Spark (Scala and/or PySpark), Hive/Impala SQL, and performance tuning Working knowledge of Kafka for streaming ingestion and NiFi (or StreamSets) for batch/near-real-time flows Experience with Cloudera Manager, YARN/Tez, HDFS, and job orchestration using Oozie/Airflow Good understanding of data warehousing concepts (dimensional modelling, partitioning, bucketing) Proficiency in Linux/Unix, Shell scripting, Git, and CI/CD (Jenkins/GitLab CI) Strong SQL and data modelling for BFSI use cases (lending, liabilities, risk, regulatory reporting) Experience in writing technical design documents (HLD/LLD) and unit/integration testing Exposure to SDLC/Agile and working in onsiteoffshore model Location - Mumbai Develop robust Spark jobs (batch and streaming) with unit tests and observability Implement ingestion patterns (Kafka/NiFi), data quality checks, and job scheduling Analyze and tune SQL/Spark for large-scale datasets Good to have / Secondary Skills / Desired
Experience with Cloudera Data Platform (CDP) Private Cloud Base/Public Cloud Security and governance: Apache Ranger, Atlas; Kerberos; Sentry (legacy) Cloud data services AWS (EMR, Glue, S3), Azure (HDInsight, Synapse, ADLS), or GCP (Dataproc, BigQuery) Databricks experience (Spark, Delta Lake) for select workloads Containerization and orchestration (Docker/Kubernetes) for micro-batch/ML workloads Python for data processing and utilities; familiarity with Scala build tools (sbt/maven) Monitoring/observability Cloudera Manager metrics, Grafana/Prometheus, log aggregation Experience with BI consumption patterns and semantic layers for risk/regulatory dashboards Educational Qualifications
B.E./B.Tech or equivalent
Experience Range
4-6 years
Certifications (Preferred)
Cloudera Certified Associate/Professional (CCA/CCP) or CDP certifications AWS/Azure/GCP data certifications (nice to have) Databricks Lakehouse Fundamentals or Associate (nice to have) .
