01Responsibilities
- Design, develop, and optimize enterprise-scale batch ETL pipelines supporting LPLs regulatory, supervision, and analytics platforms.
- Lead ETL development using Informatica and PySpark for high-volume and complex data transformations.
- Build and maintain Airflow DAGs to orchestrate batch workflows, dependencies, retries, SLAs, and monitoring.
- Design scalable solutions for large data processing on distributed and cloud-based data platforms.
- Drive performance tuning, resiliency, data quality, and operational stability of batch processing jobs.
- Partner with architecture, upstream/downstream system teams, and business stakeholders to translate requirements into robust, compliant data solutions.
- Enforce LPL standards for security, data governance, auditability, and operational readiness.
- Perform code reviews, mentor Engineer I/II resources, and raise overall engineering maturity of the team.
What are we looking for
Were looking for solid collaborators who deliver exceptional client experiences and thrive in fast-paced, team-oriented environments. Our ideal candidates pursue greatness, act with integrity, and are driven to help our clients succeed. We value those who embrace creativity, continuous improvement, and contribute to a culture where we win together and create and share joy in our work.
02Requirements
- Strong hands-on experience with batch ETL architectures in regulated environments.
- Advanced expertise in Informatica.
- Strong proficiency in PySpark and distributed data processing concepts.
- Hands-on experience with Apache Airflow (authoring DAGs, scheduling, dependency management, monitoring).
- Experience processing large datasets with a focus on performance and reliability.
- Strong SQL and data modeling fundamentals.
- Ability to own complex problem areas end-to-end and operate with minimal supervision.
LPL Global Business Services, LLP - PRIVACY POLICY .
