Position. Spark and Big Data Platform Developer
Primary Skills
-
Big Data Technologies. Extensive experience with Hadoop, HDFS, Hive, Yarn, Jenkins, Ansible, and proficiency in Agile methodologies.
-
Data Processing and Transformation. Proven expertise in data processing, orchestration, parallelization, and transformation techniques within Big Data environments.
-
Spark Development. Proficiency in Spark jobs, joblets, and developing Scala-based custom components. Solid understanding of error handling and performance tuning in Spark applications.
-
CI/CD. Hands-on experience with Continuous Integration and Continuous Deployment (CI/CD) pipelines.
Secondary Skill
-
ETL Development. Familiarity with TALEND ETL tool for efficient data extraction, transformation, and loading processes.
-
Database Management. Knowledge of Postgre for database management and maintenance.
Responsibilities
-
Develop and maintain robust Spark applications for large-scale data processing, ensuring high performance and reliability.
-
Collaborate with cross-functional teams to design, implement, and optimize data workflows and pipelines.
-
Implement best practices for error handling, performance tuning, and monitoring within Spark jobs.
-
Participate in Agile ceremonies and contribute to sprint planning, retrospectives, and daily stand-ups.
-
Utilize CI/CD pipelines to automate build, test, and deployment processes, ensuring rapid and reliable delivery of software solutions.
-
Provide technical guidance and mentorship to junior team members, fostering a culture of knowledge sharing and continuous improvement.
Qualifications
-
Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
-
Strong analytical and problem-solving skills with a keen attention to detail.
-
Excellent communication and interpersonal abilities, with a collaborative mindset.
-
Proven ability to work effectively in a fast-paced, dynamic environment, managing multiple priorities with tight deadlines.
Why Join Us
-
Opportunity to work with cutting-edge technologies and solve complex challenges in the Big Data domain.
-
Dynamic and collaborative work culture that encourages innovation and professional growth.
-
Competitive compensation package with benefits and opportunities for career advancement.
Application Process
To apply for this position, please submit your updated resume along with a cover letter highlighting your relevant experience and why you are interested in joining our team.
Note. Only shortlisted candidates will be contacted for further evaluation.
This job description is intended to convey information essential to understanding the scope of the position and is not an exhaustive list of skills, efforts, duties, responsibilities, or working conditions associated with it.