We are seeking a Senior Data Engineer with strong expertise in modern data architecture, data pipeline development, and cloud-native technologies to join our high-performing team. In this role, you will play a critical part in designing, building, and maintaining scalable, secure, and high-performance data solutions that power insights, analytics, and innovation across the company.
You’ll work on complex data integration projects involving batch, real-time streaming, and CDC pipelines, supporting diverse use cases such as advanced analytics, business intelligence, and machine learning. You’ll collaborate with cross-functional teams, Data Scientists, Product Managers, Cloud Architects, and Business Stakeholders to transform data into actionable knowledge.
What You'll Do
- Design, develop, and maintain end-to-end ETL/ELT pipelines using Java, Scala, Python, PySpark, and SparkSQL, ensuring robustness, performance, and scalability.
- Develop solutions for real-time and batch data ingestion using modern data integration patterns such as event streaming (Kafka/Kinesis), replication/CDC, and traditional batch processing.
- Architect and implement solutions in AWS, leveraging services such as Kinesis, Glue, Athena, S3, RDS, and more.
- Build and optimize data models in Snowflake Cloud Data Warehouse, enabling efficient querying and reporting across large datasets.
- Work on large-scale initiatives involving the rationalization and modernization of complex data environments, integrating data from multiple sources, both internal and external.
- Collaborate with business units to understand data requirements and deliver solutions that are reliable, performant, and easy to maintain.
- Apply best practices in software engineering and agile development, contributing to a culture of continuous improvement, code quality, and team excellence.
- Write high-quality, reusable code with an emphasis on modular design, testing, and maintainability.
- Support and mentor junior engineers, contributing to their technical development and fostering a culture of learning and growth.
- Experiment with new tools, frameworks, and technologies to continually improve how we work with data and deliver value to the business.
What You Need to Bring
- Bachelor’s degree in Computer Science, Engineering, Data Science, or a related field or equivalent experience.
- 7+ years of hands-on experience in software/data engineering with strong exposure to data-focused roles.
- Proficiency in Java, Scala, or Python, particularly in the context of data integration and transformation tasks.
- Strong experience with big data processing tools and frameworks like PySpark and SparkSQL.
- In-depth understanding of cloud-based data engineering using AWS services (e.g., Kinesis, Glue, RDS, Athena, S3, etc.).
- Experience working with Snowflake, including writing efficient SQL, optimizing queries, and managing schemas.
- Solid grasp of SQL and experience working with various relational databases (PostgreSQL, MySQL, etc.) for analytics and reporting.
- Experience working with Agile methodologies and DevOps tools for CI/CD and version control (e.g., Git, Jenkins).
- A self-motivated, curious mindset and a willingness to explore new technologies and data paradigms.
- Excellent communication skills, both written and verbal, with the ability to explain complex data concepts to non-technical stakeholders.
What Makes You Stand Out (Bonus)
- Experience participating in or leading large-scale data modernization initiatives.
- Familiarity with data governance concepts, metadata management, and data cataloging tools.
- Experience with event-driven architectures and microservices.
- Working knowledge of data security and compliance in cloud-based data environments.
Why Brightly?
At Brightly, you’ll find more than just a job, you’ll find a career path that aligns with your values. We believe in creating a workplace where everyone can thrive, feel included, and bring their whole self to work.
- Be part of a company that is shaping the future of smart cities and sustainable infrastructure.
- Work in a collaborative, inclusive, and growth-oriented environment that values diversity of thought and experience.
- Enjoy opportunities for professional development, leadership training, and career advancement.
- Join a team where your voice matters, your impact is visible, and your passion is celebrated.
Let’s build something meaningful together. Apply now and become part of the Brightly journey.