As a Senior Engineer in the Data Engineering & Analytics team, you will play a pivotal role in developing advanced data solutions for consumer-focused industries such as retail, restaurants, and banking. The primary focus will be on leveraging vast datasets to create high-performance algorithms, cutting-edge analytical techniques (including machine learning and AI), and intuitive workflows. Your contributions will empower users to derive actionable insights from big data, driving significant business outcomes.
Responsibilities
Software Development
- Develop high-quality, secure, and modular code that meets functional and testability standards.
- Drive the evolution of data & services platforms with a strong emphasis on data science and engineering.
Data Architecture and Pipelines
- Design and implement scalable data architecture and efficient data pipelines.
- Optimize existing machine learning libraries and frameworks for enhanced performance.
- Provide support for deployed data applications and analytical models, acting as a trusted advisor to Data Scientists.
- Ensure compliance with data governance policies, implementing and validating data lineage, quality checks, and classification.
Innovation and Integration
- Discover, ingest, and integrate new sources of data (real-time, streaming, batch, API-based) to enhance analytical capabilities.
- Experiment with emerging tools and technologies to streamline development, testing, deployment, and operationalization of data pipelines.
Collaboration and Communication
- Collaborate with cross-functional teams including consultants, engineers, and sales to identify and prioritize business problems.
- Evangelize releases, gather user feedback, and iterate to improve product features and functionalities.
Continuous Learning and Improvement
- Stay abreast of technical trends through self-learning, training, and hands-on experience.
- Participate in the development of data and analytic infrastructure, continuously innovating to solve complex business challenges.
Skills Required
Technical Expertise
- Proficiency in Python/Scala, Spark (job tuning), SQL, and Hadoop platforms for building robust Big Data products.
- Strong programming skills in Java, familiarity with Spring Boot, and experience with unit testing frameworks like Junit.
- Knowledge of software development test approaches, CI/CD pipelines, RESTful APIs, and micro-services architectures.
- Hands-on experience with SQL databases (e.g., Postgres, Oracle) and Hadoop ecosystem tools (Hive, Impala, Spark).
- Comfortable developing shell scripts for automation and possessing strong troubleshooting and debugging skills.
Analytical and Problem-Solving Skills
- Ability to solve complex problems using multi-layered datasets, applying statistical analytical techniques and data engineering principles.
- Capability to innovate and adopt new approaches, tools, and technologies to drive business insights and recommendations.
Soft Skills
- Strong attention to detail and the ability to manage multiple tasks effectively.
- Excellent communication skills (verbal and written) with a knack for building relationships and collaborating in diverse, geographically distributed teams.
Preferred Qualifications
Experience with Performance Tuning
- Tuning database schemas, SQL queries, ETL jobs, and related scripts to optimize performance.
Cloud and Agile Experience
- Familiarity with Cloud APIs (e.g., Azure, AWS) and participation in complex engineering projects within Agile methodologies (e.g., Scrum).
- Benefits
- Opportunity to work on cutting-edge data solutions that impact businesses across diverse sectors.
- Career growth and development through continuous learning and exposure to emerging technologies.
- Collaborative work environment with a focus on innovation and cross-functional teamwork.