Data Architect

We are seeking an experienced Senior Data Engineer to join our dynamic team. This role requires a deep understanding of data architecture and hands-on expertise with Azure Cloud and PySpark. The ideal candidate will be skilled in architecting and implementing data solutions, with a proven track record in ETL processes and data governance.

Experience. 11+ years

Key Responsibilities

1. Data Architecture. Design and implement robust data architectures using Azure Data Factory (ADF) and Azure Data Lake Storage (ADLS).
2. ETL Development. Develop and maintain ETL pipelines, ensuring data accuracy and efficiency.
3. Azure Cloud. Leverage Azure services such as ADLS, Delta Lake, Delta Live Tables, Delta Sharing, and Unity Catalog for comprehensive data solutions.
4. Data Cataloguing & Governance. Apply best practices in data cataloguing and governance, with a minimum of 6 months hands-on experience.
5. Distributed Data Processing. Utilize Spark and Databricks with Python for distributed data processing tasks.
6. Programming. Write efficient Python and PySpark code for data manipulation and processing.
7. SQL Scripting. Perform CRUD operations on time series databases and optimize SQL queries.
8. Testing. Conduct unit testing and system integration testing using Python frameworks.
9. CI/CD Pipelines. Design and deploy CI/CD pipelines using Jenkins, integrating with DevOps practices.
10. Agile Practices. Collaborate within high-performance agile teams, using Scrum, JIRA, JFrog, and Confluence.

Technical Skills (Must-Have)

1. Distributed Data Processing. Hands-on experience with Spark and Databricks.
2. Programming Languages. Proficiency in Python and PySpark.
3. Azure Services. Expertise in ADLS, Delta Lake, Delta Live Tables, Delta Sharing, and Unity Catalog.
4. SQL. Experience with SQL scripting and CRUD operations on time series databases.
5. Testing. Applied experience in unit testing and system integration testing using Python.
6. DevOps. Experience in designing and deploying CI/CD pipelines using Jenkins.
7. Certifications. AZURE Data Engineering (DP203) or Databricks certification.
8. Agile Tools. Familiarity with Scrum, JIRA, JFrog, and Confluence.

Preferred Qualifications

1. Domain Experience. Experience in the Medical/Healthcare domain is a plus.
2. Communication Skills. Excellent communication skills are essential for collaborating with cross-functional teams and stakeholders.

Why Join Us?

1. Innovative Environment. Work in a cutting-edge environment where you can drive significant impact.
2. Career Growth. Opportunities for professional development and career advancement.
3. Dynamic Team. Collaborate with a high-performance team committed to excellence.

If you are a seasoned Data Engineer with a passion for data architecture and cloud technologies, we would love to hear from you. Apply now to join our team and be part of exciting data-driven projects!