We are seeking a highly skilled and experienced Senior Azure Data Engineer with over 8+ years of experience in the IT industry. The ideal candidate will have a strong technical background in Azure Data Engineering, excellent problem-solving skills, and the ability to lead a team of data engineers in designing, developing, and implementing robust data solutions.
Responsibilities
- Lead and mentor the data engineering team, sharing best practices, technical expertise, and guidance to ensure high-quality deliverables.
- Architect, design, and develop end-to-end data solutions using Azure Data Engineering Stack, including Data Factory, Databricks, Synapse, Event Hub, Cosmos DB, ADLS Gen2, and Function App.
- Implement and manage data pipelines, ensuring efficient data integration, transformation, and delivery processes.
- Leverage Python and PySpark for data processing, analytics, and automation.
- Collaborate with cross-functional teams to deliver data solutions aligned with business goals.
- Develop and optimize SQL queries and scripts to manage and manipulate large datasets.
- Work with stakeholders to gather requirements, perform data modeling, and design effective Data Warehouse Architectures.
- Utilize Power BI for reporting and visualization, ensuring data-driven decision-making across the organization.
- Integrate SFTP, messaging systems, and APIs into the data architecture for seamless data flows.
- Work in an Agile environment, contributing to sprint planning, retrospectives, and delivering on-time results.
- Maintain best practices in software development, including code reviews, version control, and testing.
- Continuously monitor, troubleshoot, and improve data pipelines to ensure optimal performance and scalability.
Requirements
- 8+ years of experience in the IT industry with a focus on data engineering.
- 5+ years of hands-on experience with the Azure Data Engineering Stack (Data Factory, Databricks, Synapse, Event Hub, Cosmos DB, ADLS Gen2, Function App).
- 3+ years of experience with Python and PySpark for data processing and analytics.
- Strong understanding of other Azure services and their integration into data solutions.
- Proficient in SQL, with the ability to write and optimize complex queries.
- In-depth understanding of Data Warehouse Architecture, Data Modeling, and design concepts.
- Experience with Power BI, SFTP, messaging systems, and APIs.
- Excellent analytical skills and organizational capabilities.
- Ability to work independently and as part of a team in a dynamic, Agile environment.
- Strong communication skills, both written and verbal.
- Knowledge of software development best practices.
- DP200/DP201 certifications are a plus.
Preferred Qualifications
- Certification in Azure data engineering (e.g., DP200, DP201).
- Proven leadership experience in guiding and mentoring data engineering teams.
This is a great opportunity to take on a leadership role and contribute to cutting-edge data solutions while working with a collaborative and innovative team. If you are passionate about data engineering and cloud technologies, we encourage you to apply!