As a Senior Engineer in the Data Engineering & Analytics team, you'll spearhead the development of data and analytics solutions leveraging vast datasets from retail stores, restaurants, banks, and other consumer-focused industries. Your primary challenge will be crafting high-performance algorithms, employing cutting-edge analytical techniques, including machine learning and artificial intelligence, and designing intuitive workflows to empower users in deriving actionable insights from big data, thereby driving business growth. You'll have the opportunity to work with datasets spanning billions of transactions and create front-end visualizations to unlock the value of big data. Your role will involve developing innovative analytical solutions, building machine learning models, designing automated data pipelines, and optimizing data architecture to support business and client needs effectively.
Responsibilities
- Develop high-quality, modular, and testable code as a hands-on developer.
- Drive the evolution of Data & Services products/platforms with a focus on data science and engineering.
- Design and implement scalable data architecture and data pipelines.
- Solve complex problems using multi-layered data sets and optimize existing machine learning libraries and frameworks.
- Provide support for deployed data applications and analytical models, acting as a trusted advisor to Data Scientists and other stakeholders.
- Ensure adherence to proper data governance policies, including implementing Data Lineage, Quality checks, and classification.
- Explore and integrate new sources of data into our platform, including real-time, streaming, batch, and API-based data.
- Experiment with new tools to enhance the development, testing, deployment, and running of data pipelines.
- Participate in the development of data and analytic infrastructure for product development.
- Continuously innovate and explore new approaches, tools, techniques, and technologies to solve business problems and generate insights.
- Collaborate with cross-functional teams to define vision, establish team culture and processes.
- Prioritize operational activities based on key drivers of organizational value.
- Escalate technical errors or bugs detected in project work.
- Stay updated on relevant technical and product trends through self-learning, training classes, and job shadowing.
- Support the building of scaled machine learning production systems by designing pipelines and engineering infrastructure.
Skills
Must Have
- Proficiency in Python/Scala, Spark, SQL, and Hadoop platforms for building Big Data products & platforms.
- Strong programming skills in Java, Spring Boot, and Junit.
- Knowledge of software development test approaches & frameworks.
- Familiarity with RESTful APIs and micro-services architectures.
- Experience with CI/CD and SQL databases like Postgres and Oracle.
- Hands-on experience with Hadoop big data tools such as Hive, Impala, and Spark.
- Familiarity with data pipeline and workflow management tools like NIFI and Airflow.
- Ability to develop shell scripts for automation.
- Strong troubleshooting and debugging skills.
- Proficiency in standard software development practices, including version control, testing, and deployment.
- Basic knowledge of statistical analytical techniques, coding, and data engineering.
- Quick learning ability and adaptability to new technologies.
- Strong problem-solving skills with multi-layered data sets.
- Innovative mindset to explore new approaches and technologies.
- Ability to multitask with attention to detail.
- Excellent communication, collaboration, and organizational skills.
Nice to Have
- Experience in performance tuning of databases, SQL, ETL jobs, etc.
- Familiarity with Cloud APIs (e.g., Azure, AWS).
- Participation in complex engineering projects in an Agile setting (e.g., Scrum).
This role offers an exciting opportunity to lead impactful projects at the intersection of data science and engineering, driving business growth through innovative data-driven solutions.