01Overview
Apache sparkSkillset : Apache Spark Designer Education - BE, MCA, MSc or equivalent( valid degree)8 or more years of professional IT experience, including the majority of the following (specific experiences can be concurrent):BFSI domain experience is preferredJD : "Tech Mahindra Ltd., Mumbai looking for a Apache Spark Designer to create and manage large scale data processing solutions. In this role, Candidate should have a background in data and business analysis. Should have hands on Apache Spark where you have worked on design, development and deployment on cluster of servers.
Also have data related processing problem-solving aptitude.ResponsibilitiesTranslate business needs to technical specificationsArchitecting, Design, build and deploy data processing solutionsHands on tools to process, stage, store data Conduct recon of dataCollaborate with teams to integrate systems Develop and execute Spark Queries, database queries and conduct analysesCreate dataset, RDD, Data Frame for requested projects Develop and update technical documentationSound knowledge about effort estimation methods for data processing solutions requirements Proven experience as Apache Spark Designer.Hands-on coding experience for Apache Spark designingBackground in data processing design (e.g. large scale data processing with API based status) and data curationIn-depth understanding of database management systems, Spark clustering, Spark executor, Driver, real time data processing platform, Java 1.8, and ETL (Extract, transform, load) frameworkKnowledge of SQL queries, Spark Engine and Spark Services, Experience of Spark query tuning and performance optimizationSoft skillsProven abilities to take initiative and be innovativeAnalytical mind with a problem-solving aptitude .
