Senior Data Engineer

We are looking for a highly skilled Senior Data Engineer with extensive experience in AWS data lake architecture and healthcare applications. This role is integral to our product development team, focusing on data lakes, data architecture, and cloud-based solutions within an AWS environment. The ideal candidate will possess a strong background in ETL processing, database management, performance tuning, and AWS cloud services, ensuring robust and scalable data solutions.

Key Responsibilities

1. Data Lake Architecture & Development. Design, implement, and manage a data lake architecture tailored to a healthcare application using AWS services.
2. AWS Data Migration & Troubleshooting. Set up, configure, and troubleshoot AWS data migration services, ensuring smooth and efficient data flow.
3. AWS Glue & DBT Management. Catalog, set up, configure, and troubleshoot AWS Glue or DBT, enabling effective ETL workflows.
4. ETL Processing & Database Management. Perform extensive ETL processing, creating tables and managing Hudi databases to optimize data storage and retrieval.
5. AWS SQS Development. Develop and manage Amazon SQS (Simple Queue Service) for real-time event-driven architectures.
6. Event Bridge Scheduler Utilization. Work extensively with AWS Event Bridge Scheduler to orchestrate and manage scheduled workflows.
7. Redshift Performance Optimization. Design and optimize tables, views, and materialized views (MV's) within Amazon Redshift, ensuring high performance and efficiency.

8. Aurora MySQL Database Management

   • Develop and optimize stored procedures for seamless database operations.

   • Set up and manage read replication for enhanced performance and reliability.

9. Monitoring & Troubleshooting AWS Resources. Continuously troubleshoot and monitor AWS resources, implementing performance improvements where necessary.

Required Skills & Experience

• 7-10+ years of hands-on experience in data engineering and cloud-based architecture.
• Expertise in AWS Data Lake Architecture, including data ingestion, transformation, and storage.
• Strong experience in working with AWS Glue or DBT for data cataloging and ETL workflows.
• Proficiency in Amazon Redshift, with a deep understanding of table/view creation, materialized views, performance tuning, and query optimization.
• Hands-on experience with AWS SQS development, implementing message queues for event-driven applications.
• Strong understanding of AWS Event Bridge Scheduler, ensuring efficient event scheduling and automation.
• Innovative and creative mindset, capable of designing optimized data solutions in a cloud-based environment.
• Experience in Hudi database management and best practices for handling large-scale data.
• Proficiency in Aurora MySQL database management, including stored procedures, replication setups, and performance tuning.
• Ability to troubleshoot and optimize AWS resources for enhanced system performance.

Why Join Us?

• Work on cutting-edge healthcare data solutions within a cloud-based AWS environment.
• Opportunity to architect and implement innovative data lake solutions with the latest AWS technologies.
• Competitive compensation package with a pay scale of up to 36 LPA, based on experience and skill level.
• Collaborative and fast-paced work environment with opportunities for professional growth.

If you are a highly experienced Data Engineer with a passion for cloud computing and innovative data solutions, we encourage you to apply and become part of a team that is revolutionizing healthcare technology!