Senior Databricks Data Engineer

As a Senior Data Engineer, you will work alongside business Subject Matter Experts (SMEs) to implement robust data validation rules and support a data-driven approach to decision-making. 

Responsibilities

1. Data Validation Implementation. Collaborate with business SMEs to develop and implement data validation rules, ensuring data quality and reliability.
2. Business Requirements Analysis. Translate business requirements into actionable data workflows, contributing insights from a data engineering perspective.
3. Data Understanding and Flow. Gain a deep understanding of the data and its flow across various systems to optimize data integration.
4. Platform Architecture. Work within our platform’s architecture, leveraging its capabilities to ensure efficient data loading and transformation.
5. Adaptability to Process Changes. Stay up-to-date with evolving processes and incorporate necessary changes into your work.

Mandatory Skills and Experience

1. Experience. 7+ years in data engineering or related fields.
2. Technical Expertise
   1. Azure Databricks. 3+ years of hands-on experience with Azure Databricks.
   2. Python & PySpark. 5+ years of proficiency in Python and PySpark, enabling complex data transformation and processing.
   3. SQL. Advanced SQL skills with 5+ years of practical experience.
3. Data Architecture and Lifecycle Knowledge. Understanding of data lake houses, business intelligence architecture, and the complete data platform lifecycle.
4. DevOps and CI/CD. Familiarity with Azure data/delta lake, Azure DevOps, Git, and CI/CD practices.
5. Documentation and Code Quality. Ability to create detailed design documents, unit test plans, and code review reports.
6. Analytical and Communication Skills. Strong analytical skills, with the capability to understand complex data. Excellent communication skills to engage with stakeholders across various levels.
7. Education. Bachelor’s degree in IT, Computer Science, or a related field.

Nice-to-Have Skills

1. Financial Markets Knowledge. Familiarity with financial markets, portfolio theory, and risk management is a plus.
2. Agile Work Environment. Experience with Agile methodologies (Scrum, Agile).

Join us in this exciting journey to transform data into impactful insights and be a part of a team that values innovation, collaboration, and quality. Apply today if you’re ready to bring your expertise and make a difference in a rapidly evolving tech landscape!