We are seeking a skilled Big Data Engineer to join our dynamic team. The ideal candidate will have hands-on experience with technologies such as Spark, PySpark, Hive, and HBase. You will play a crucial role in ingesting data from various sources and developing efficient ETL jobs. This is an exciting opportunity to work on innovative projects and contribute to our data-driven initiatives.
Experience. 5 to 10 Years
Key Responsibilities
Data Ingestion and ETL Development
- Ingest data from various sources, including structured, unstructured, and semi-structured data formats.
- Develop robust ETL jobs using Spark, PySpark, Hive, and HBase to ensure efficient data processing and transformation.
Impact Analysis and Estimation
- Conduct impact analysis to evaluate the effects of changes on existing data pipelines and workflows.
- Provide accurate estimates for project timelines and resource requirements.
End-to-End Project Responsibility
- Take ownership of the end-to-end delivery of data solutions, ensuring quality and timely completion.
- Collaborate with cross-functional teams to meet project objectives.
Project Planning and Implementation Strategy
- Create detailed project plans that outline milestones, deliverables, and resource allocation.
- Develop and implement effective strategies for project execution and delivery.
ETL Concepts and Data Transfers
- Maintain a comprehensive understanding of ETL concepts and cross-environment data transfers.
- Ensure best practices are followed for data quality and integrity throughout the ETL process.
Customer Communication and Management Reporting
- Handle customer communications effectively, addressing any queries and providing updates on project progress.
- Prepare management reports that highlight key metrics, project status, and any risks or issues encountered.
Must-Have Skills
- Proficiency in Spark, PySpark, Hive, and HBase.
- Strong understanding of ETL processes and methodologies.
- Excellent problem-solving skills and attention to detail.
- Ability to communicate effectively with technical and non-technical stakeholders.
- Experience with data modeling and database design.
What We Offer
- A collaborative and innovative work environment.
- Opportunities for professional growth and development.
- Competitive salary and benefits package.
If you are passionate about big data technologies and thrive in a fast-paced environment, we would love to hear from you!