We are seeking an experienced Data Engineer with a robust background in Data Bricks, Python, and Spark. This role is critical in ensuring seamless data access, preparing data for model consumption, and collaborating with global teams to create scalable, model-ready datasets.
Key Responsibilities
- Data Integration & Preparation. Collaborate with data engineers to guarantee that data is accessible for discovery and is properly prepared for model consumption.
- Model-Ready Data. Work closely with ML engineers and data scientists to design and implement scalable data pipelines and ensure data versioning for effective model training.
- Solution Design. Develop and design solutions based on project requirements, ensuring alignment with best practices and organizational standards.
- Data Analysis & Profiling. Conduct comprehensive data analysis and profiling to ensure data quality and integrity.
- SQL Expertise. Write and optimize complex SQL queries to extract and manipulate data effectively.
Technical Skills
- Programming. Proficient in Python and PySpark for data processing and transformation.
- Data Platforms. Extensive experience with Data Bricks and the Azure platform, including Azure Data Factory (ADF), Azure Data Lake, and Blob Storage.
- Data Warehousing. Knowledge of Hive for data warehousing and querying.
- SQL. Strong SQL skills with experience in writing and optimizing queries.
- RDBMS. Experience with Teradata or other relational database management systems is a plus.
Desired Qualifications
- Proven experience in designing and implementing data solutions in a cloud environment.
- Ability to work effectively with global teams and communicate technical concepts clearly.
- Strong analytical skills with a focus on data quality and performance.
- Familiarity with additional data engineering tools and technologies is advantageous.
Why Join Us
- Work with cutting-edge technologies in a dynamic and collaborative environment.
- Opportunity to contribute to impactful data-driven projects and solutions.
- Engage with a global team of experts in data engineering, machine learning, and analytics.
How to Apply
If you are passionate about data engineering and possess the skills outlined above, we encourage you to apply. Please send your resume and a cover letter detailing your experience and qualifications.