Data Software Engineer Spark, Python, Databricks (L2 / L4)

Data Software Engineer Spark, Python, Databricks (L2 / L4) Experience: L2: 45 Years L4: 812 Years Mode: FTE (Full-Time Employment) Job Location: Bangalore Work Mode: Hybrid Notice Period: L2: Immediate to 15 days L4: Immediate to 15 days Drive Type: F2F Drive Location: Bangalore CTC Band: L2: Up to 21 LPA L4: Up to 42 LPA Role Overview We are hiring Data Software Engineers with strong expertise in Apache Spark, Python, and AWS/Azure Databricks. The ideal candidates will have deep Big Data engineering experience, strong distributed systems knowledge, and the ability to work on complex end-to-end data platforms at scale. Key Responsibilities Big Data Engineering Design and build distributed data processing systems using Spark and Hadoop. Develop and optimize Spark applications, ensuring performance and scalability. Create and manage ETL/ELT pipelines for large-scale data ingestion and transformation. Streaming & Event Processing Build and manage real-time streaming systems using Spark Streaming or Storm. Work with Kafka / RabbitMQ for event-driven ingestion and messaging patterns. Cloud & Databricks Engineering Develop & optimize workloads on AWS Databricks or Azure Databricks. Perform cluster management, job scheduling, performance tuning, and automation. Data Integration & Storage Integrate data from diverse sources: RDBMS (Oracle, SQL Server), ERP, file systems. Work with query engines like Hive and Impala. Experience with NoSQL stores: HBase, Cassandra, MongoDB. Programming & Scripting Strong hands-on coding in Python for data transformations and automations. Strong SQL skills for data validation, tuning, and complex queries Team Leadership (L4) Provide technical leadership and mentoring to junior engineers. Drive solution design for Big Data platforms end-to-end Ways of Working Work in Agile teams, participate in sprint ceremonies and planning. Collaborate with engineering, data science, and product teams. Required Skills & Expertise (Both L2 & L4) Apache Spark Expert level (core, SQL, streaming) Python Strong hands-on Distributed computing fundamentals Hadoop ecosystem: Hadoop v2, MapReduce, HDFS, Sqoop Streaming systems: Spark Streaming / Storm Messaging: Kafka or RabbitMQ SQL Advanced (joins, stored procedures, query optimization) NoSQL: HBase, Cassandra, MongoDB ETL frameworks & data pipeline design Hive / Impala querying Performance tuning of Spark jobs AWS or Azure Databricks Experience working in Agile Experience & Level Mapping L2 Mid-Level (45 Yrs) Skills: Spark, Python, AWS Notice Period: Immediate 20 Days CTC Band: Up to 21 LPA L4 Senior-Level (812 Yrs) Skills: Spark, Python, Azure Databricks Notice Period: 15 Days (Nov joiners) OR Jan joiners CTC Band: Up to 42 LPA .