We are seeking a highly skilled Senior Data Engineer to join our team. This role is pivotal in shaping our data collection procedures and ensuring data integrity for machine learning models. You will be responsible for extracting and analyzing data from various databases, performing data cleansing, and ensuring compliance with data quality standards. Your expertise will be crucial in solving complex business issues and recommending solutions to enhance data systems.
Responsibilities
- Design and Development. Create secure and efficient data architectures, including updates and programming changes for data pipelines and repositories that handle structured and unstructured data.
- Analysis and Integration. Assess design requirements, code, program, and integrate data systems based on architectural objectives.
- Testing and Debugging. Develop and execute comprehensive testing plans. Identify and resolve issues related to code and integration within data systems.
- Project Leadership. Lead a team of data engineers to develop high-quality, cost-effective data solutions. Provide guidance and mentorship to junior team members.
- Collaboration and Communication. Work closely with project teams to track progress, resolve issues, and ensure successful project delivery.
- Data Accuracy and Automation. Identify data inaccuracies and opportunities for automation to improve data quality. Analyze complex datasets to determine trends and patterns.
- Cross-Functional Work. Collaborate with various departments to define and develop reporting deliverables.
- Representation and Mentorship. Act as the representative for the data engineering team in larger, complex development projects and mentor less experienced staff.
Education & Experience
- Required. Bachelor’s or Master’s Degree in Computer Science, Information Technology, Software Engineering, Statistics/Mathematics, or a related field. Typically requires 4-7 years of relevant work experience, or 3-5 years with an advanced degree.
- Preferred. Experience in data analytics, data engineering, or data modeling is highly desirable.
Preferred Certifications
- Certifications in programming languages such as SQL, Python, or similar.
Knowledge & Skills
- Technical Expertise. Proficiency in Agile methodologies, Amazon Web Services (AWS), Apache Hadoop, Apache Kafka, Apache Spark, Big Data technologies, and Extract Transform Load (ETL) processes. Experience with data warehousing, data pipelines, and data modeling.
- Programming Languages. Strong skills in Python, SQL, Java, Scala, and familiarity with machine learning.
- Cloud Platforms. Experience with Microsoft Azure.
Cross-Org Skills
- Effective Communication
- Results Orientation
- Learning Agility
- Digital Fluency
- Customer Centricity
Impact & Scope
This role impacts multiple teams and may involve leading or guiding projects, validating information, and facilitating team decision-making.
Complexity
Handle moderately complex issues within established guidelines, providing solutions and recommendations.
Disclaimer
This job description outlines the general nature and level of work performed in this role and is not intended to be an exhaustive list of all duties, skills, or responsibilities. Additional functions may be assigned as needed.