We are looking for a highly skilled and motivated Data Engineer to join our team and play a key role in designing and building robust, scalable, and high-performance data solutions on the Microsoft Azure cloud platform. The ideal candidate will bring deep expertise in data engineering, strong SQL skills, and experience working with Azure-native tools and services to support the entire lifecycle of enterprise data management, from ingestion to analytics. If you thrive in a fast-paced, collaborative environment and are passionate about turning data into actionable insights, we’d love to hear from you.
Key Responsibilities
End-to-End Data Pipeline Development
- Design, develop, and maintain scalable and secure data pipelines using modern Azure services such as Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.
- Build robust ETL/ELT workflows that can process large volumes of structured and unstructured data from diverse sources.
Data Architecture and Modeling
- Create and maintain efficient data architectures to support analytics, reporting, and machine learning use cases.
- Develop logical and physical data models, star/snowflake schemas, and data marts to facilitate business intelligence initiatives.
Data Storage and Management
- Leverage Azure Data Lake Storage (Gen2), Azure SQL Database, and other relevant storage solutions to manage data at scale.
- Ensure proper partitioning, indexing, and lifecycle policies for optimized performance and cost-efficiency.
Data Integration and Orchestration
- Integrate data across multiple cloud and on-premises systems through seamless orchestration using Azure-native tools.
- Schedule and automate data workflows while managing dependencies and error handling.
Performance Optimization and Tuning
- Write clean, maintainable, and high-performance SQL queries for data extraction, transformation, and aggregation.
- Tune and optimize query performance, stored procedures, and data transformation logic to minimize latency and resource usage.
Collaboration and Solution Delivery
- Work closely with Data Scientists, Business Analysts, Software Engineers, and Stakeholders to deliver end-to-end data solutions.
- Translate complex data requirements into scalable, reusable components and services that support business goals.
Monitoring and Maintenance
- Monitor data pipelines and job execution using Azure Monitor, Log Analytics, and custom alerting strategies.
- Implement and enforce data quality checks to ensure accuracy, completeness, and consistency of ingested data.
Security and Compliance
- Implement best practices for data security, including encryption, role-based access control, and data masking.
- Ensure all solutions comply with data governance policies and regulatory standards like GDPR, HIPAA, or CCPA.
Troubleshooting and Issue Resolution
- Proactively identify and resolve issues across the data stack, from ingestion to analytics layers.
- Provide root cause analysis and implement preventive measures to avoid recurring problems.
Technical Leadership and Mentoring
- Act as a technical mentor and thought leader within the data engineering team.
- Share knowledge, conduct code reviews, and guide junior engineers on best practices, tools, and techniques.
Continuous Learning and Innovation
- Stay updated with the latest trends, tools, and best practices in cloud data engineering and Azure technologies.
- Recommend and implement improvements to architecture, tools, and processes to ensure scalability, performance, and maintainability.
Preferred Tools and Technologies
- Azure Data Factory, Azure Databricks, Azure Synapse Analytics
- Azure Data Lake Storage (Gen2), Azure SQL Database, Cosmos DB
- Python / PySpark / Scala for data processing
- T-SQL / U-SQL
- CI/CD pipelines (Azure DevOps, GitHub Actions)
- Monitoring tools. Azure Monitor, Application Insights
- Data Governance tools. Purview (optional)
This role offers a unique opportunity to work with cutting-edge cloud technologies, drive innovation, and shape the data strategy for a forward-thinking organization. You’ll be at the forefront of transforming raw data into strategic assets, supporting decision-making and advanced analytics at scale.