We are seeking an experienced Senior Data Engineer to join our dynamic Data Engineering team. In this pivotal role, you will be instrumental in establishing our Cloud-based Data Lake infrastructure and implementing standardized data models that serve as the foundation for enterprise-wide data analytics and reporting. You will play a key part in building scalable data pipelines, driving continuous integration and delivery practices, and enabling data consumption across diverse business functions.
This is a highly collaborative role requiring hands-on development expertise, a strategic mindset, and strong communication skills to align technical delivery with business priorities.
Key Responsibilities
- Data Ingestion & Integration. Design and build robust data pipelines to ingest structured and unstructured data from various internal systems, third-party vendors, public data repositories, and social media platforms. Leverage both batch and real-time processing paradigms.
- Cross-Functional Collaboration. Work closely with data scientists, business analysts, data architects, and application teams to understand data requirements and provide high-quality, clean, and reliable data sets for downstream consumption.
- Solution Design & Implementation. Participate in solution architecture discussions and propose scalable and cost-effective designs for ingestion, storage, transformation, and delivery of data using modern data lakehouse principles.
- Stakeholder Communication. Maintain regular updates with project stakeholders to communicate progress, surface potential risks, and manage expectations. Ensure alignment between business goals and technical solutions.
- Risk & Issue Management. Identify challenges and dependencies early, proactively manage risks, and escalate critical issues to the Head of Data Engineering or initiative sponsors as required.
- Project Delivery Oversight. Track key project milestones, ensure adherence to timelines and budget, and maintain the quality of deliverables. Lead post-implementation reviews to evaluate success and document lessons learned.
- Transition to Operations. Ensure smooth handover of completed projects to operational teams with thorough documentation, training, and knowledge transfer.
What We’re Looking For
Technical Skills & Competencies
- Strong understanding of cloud data architecture, particularly on Microsoft Azure (experience with other cloud providers is a plus).
- Proven hands-on experience in building and maintaining data pipelines using Apache Spark (Spark SQL, Dataframe API) or Hive (HQL).
- Familiarity with Hadoop ecosystems and file formats such as Avro, Parquet, and ORC.
-
Practical experience with data ingestion, transformation, and delivery across multiple data source types including:
-
Relational Databases. SQL Server, Oracle, DB2, Netezza
-
NoSQL / Document Stores
-
Flat Files & APIs
-
- Exposure to ETL tools and big data processing frameworks.
- Basic scripting skills (e.g., Shell, Bash) for automation and system tasks.
- Knowledge of CI/CD pipelines and development tools such as Jenkins, Azure DevOps, Bamboo, Bitbucket, Artifactory, JIRA
- Understanding of DevOps practices including Git-based version control, automated testing, and release management.
- Ability to debug, fine-tune, and optimize data processing jobs to ensure performance and scalability.
Professional Experience & Background
- 10+ years of hands-on experience working within enterprise IT, preferably in cloud-based and big data environments.
- Strong track record of delivering data solutions in fast-paced, large-scale organizations.
- Ability to operate both independently and collaboratively in cross-functional teams.
- Familiarity with agile methodologies and a commitment to continuous improvement.
Educational Qualifications
- Bachelor’s or Master’s degree in STEM disciplines (Science, Technology, Engineering, Mathematics).
- Additional certifications in Data Engineering, Cloud Platforms, or Big Data Technologies are highly desirable and will be considered a strong plus.
Language Proficiency
- Excellent verbal and written communication skills in English.
- Ability to convey complex technical concepts to non-technical stakeholders clearly and effectively.
Why Join Us?
- Be part of a forward-thinking team shaping the future of data-driven decision-making.
- Work on modern cloud technologies and innovative data engineering solutions.
- Engage in impactful work with high visibility across the organization.
- Collaborate with passionate professionals in a culture of learning and growth.
If you're excited by the opportunity to work on transformative data initiatives, solve complex problems, and help build a scalable data platform from the ground up we’d love to hear from you.