As a Cloud Data Engineer, you will play a pivotal role in designing and implementing robust data engineering solutions.
Infogain is a human-centered digital platform and software engineering company based in Silicon Valley. We specialize in engineering business outcomes for Fortune 500 companies and digital natives across various industries, including technology, healthcare, insurance, travel, telecom, and retail. Our expertise lies in leveraging cutting-edge technologies such as cloud computing, microservices, automation, IoT, and artificial intelligence to drive experience-led transformation in the delivery of digital platforms. As a Microsoft Gold Partner and Azure Expert Managed Services Provider, we are committed to delivering excellence to our clients.
Roles & Responsibilities
- Data Engineering Expertise. Bring 6 to 8 years of overall experience in cloud data engineering, with a focus on Azure Databricks and PySpark for a minimum of 3 to 5 years.
- SQL Proficiency. Demonstrate strong SQL skills, including complex joins, subqueries, functions, and procedures. You should be capable of performing SQL tuning and query optimization independently, ideally with 5 to 6 years of solid SQL experience.
- ETL Pipeline Development. Design, develop, and maintain ETL pipelines using Databricks, PySpark, and Azure Data Factory (ADF) to efficiently extract, transform, and load data from various sources.
- Hosted Solutions Implementation. Experience in implementing large and complex hosted solutions using Azure technologies, including ADF, Azure Data Lake Storage, and Databricks.
- Data Management. Work with Delta tables for deduplication and merging processes, handling terabytes of data sets effectively.
- Optimization. Optimize and fine-tune existing ETL workflows for performance and scalability. A minimum of 2 to 3 years of experience with ADF is desirable.
- Data Processing. Experience in working with large datasets, with knowledge of both batch and streaming data processing being a plus.
- Data Security and Compliance. Develop reliable, secure, and compliant data processing systems.
- Collaboration. Collaborate closely with cross-functional teams to support data analytics, reporting, and business intelligence initiatives.
- Independent Work Ethic. Exhibit a self-driven attitude and the ability to work independently without support.
Experience Required
- 6 to 8 years of experience in data engineering roles, with a strong focus on cloud technologies.
Skills
- Primary Skill. Data Engineering
- Sub Skills. Data Engineering
Additional Skills
- Python
- Databricks
- Azure Data Factory
- PySpark
Why Join Us?
At Infogain, you will be part of a dynamic team that is dedicated to driving digital transformation for our clients. We offer a collaborative work environment, opportunities for professional growth, and the chance to work on exciting projects that leverage the latest technologies.