As a Data Engineer at Prudential Singapore, you will play a pivotal role in the Data Engineering team, working on cutting-edge technologies to set up and manage a cloud-based Data Lake. You will be responsible for designing and developing data pipelines, implementing standardized data models, and ensuring data transformation, integration, and delivery processes align with business requirements. This position provides the opportunity to work with multiple data sources, cloud platforms, and big data tools, ensuring robust data solutions for critical business operations.
Key Responsibilities
- Data Pipeline Development. Build and manage scalable data pipelines that ingest a wide variety of data from multiple internal sources, as well as external public and social media data, into the Data Lake.
- Cross-Functional Collaboration. Work closely with cross-functional teams to gather data and ensure it’s available for downstream consumption.
- Solution Design. Collaborate with the team to create effective and scalable data solutions tailored to meet business needs.
- Stakeholder Communication. Maintain regular communication with key stakeholders, address concerns, manage risks, and escalate critical issues to the Data Engineering leadership.
- Project Management. Ensure project timelines are adhered to, manage dependencies and challenges, and ensure the initiative is delivered within budget and quality parameters.
- Post-Implementation Review. Conduct reviews post-implementation to ensure the project objectives are met and lessons learned are documented for future improvements.
Who We Are Looking For
Competencies & Personal Traits
- A team player with excellent problem-solving and analytical skills.
- Ability to work efficiently in cloud and big data environments.
- Proactive communicator who can manage stakeholder expectations.
Technical Skills
- Cloud Expertise. Experience with at least one cloud infrastructure provider (Azure, AWS).
- Big Data Processing. Proficient in building data pipelines using batch processing technologies such as Apache Spark (Spark SQL, Dataframe API) or Hive Query Language (HQL).
- ETL Tools. Familiar with big data ETL tools and frameworks.
- File Formats. Knowledge of Hadoop file formats such as Avro, Parquet, ORC.
- Scripting. Basic knowledge of shell/bash scripting.
- Database Experience. Experience working with relational databases (SQL Server, Oracle, DB2, Netezza) and NoSQL/document databases, as well as flat file systems.
- CI/CD & DevOps. Basic understanding of CI/CD tools like Jenkins, JIRA, Bitbucket, Artifactory, Bamboo, and Azure DevOps. Familiarity with DevOps practices using Git for version control.
- Optimization. Skilled in debugging, optimizing, and fine-tuning large-scale data processing jobs.
Professional Experience
- 1-3 years of experience working with enterprise IT applications in cloud platforms and big data environments.
Preferred Qualifications
- Certifications in Data and Analytics are highly desirable.
Education
- Bachelor’s or Master’s degree in Science, Technology, Engineering, or Mathematics (STEM).
Language Requirements
- Fluency in written and spoken English.
Additional Skills
- Primary Skill. Data Engineering.
- Sub-Skills. Data Engineering, Big Data, Databricks.
About Infogain
Infogain is a human-centered digital platform and software engineering company, headquartered in Silicon Valley. We specialize in engineering business outcomes for Fortune 500 companies and digital natives across various industries, including technology, healthcare, insurance, travel, telecom, retail, and CPG. Leveraging cutting-edge technologies such as cloud, microservices, automation, IoT, and AI, we drive transformative, experience-led digital platforms.
As a Microsoft Gold Partner and Azure Expert Managed Services Provider (MSP), Infogain delivers unparalleled services to its clients. With offices in California, Washington, Texas, the UK, the UAE, and Singapore, and delivery centers in Seattle, Houston, Austin, Kraków, Noida, Gurgaon, Mumbai, Pune, and Bengaluru, we’re expanding globally to ensure success for our partners.
Join Infogain to be part of a fast-growing team driving innovative digital transformations for some of the world’s leading companies.