As a Data Engineer, you will play a critical role in designing, developing, and maintaining data solutions to support the organization's data needs. Your responsibilities will include creating robust data pipelines, ensuring data quality, and implementing ETL (Extract, Transform, Load) processes for seamless data migration and deployment. Your work will ensure the integrity, performance, and scalability of data infrastructure.
Key Responsibilities
- Data Pipeline Development. Design and develop efficient data pipelines to extract, transform, and load data across various systems.
- Data Quality Assurance. Implement data validation and cleansing processes to ensure the accuracy and integrity of data.
- Collaborative Solution Design. Work closely with cross-functional teams to understand data requirements and design effective data solutions.
- Performance Optimization. Optimize and tune data infrastructure to enhance performance and scalability.
- Issue Resolution. Troubleshoot and resolve data-related issues promptly.
- Continuous Learning. Stay current with industry trends and emerging technologies in data engineering.
- Mentorship. Provide guidance and support to junior data engineers, sharing expertise and best practices.
Must-Have Skills
- Proficiency in Python. Strong programming skills in Python to build and maintain data solutions.
- Data Engineering Fundamentals. Solid understanding of data engineering principles and best practices.
- Database Skills. Experience with data modeling, database design, and SQL.
- ETL Tools Experience. Hands-on experience with ETL frameworks such as Apache Spark or Apache Airflow.
- Cloud Platforms. Familiarity with cloud services like AWS or Azure for data storage and processing.
Good-to-Have Skills
- Big Data Technologies. Experience with Hadoop, Apache Kafka, or similar technologies.
- Distributed Computing. Knowledge of distributed computing and parallel processing concepts.
- Data Warehousing. Experience with data warehousing and integration tools.
- Data Security. Understanding of data security and privacy principles.
Educational Qualification
- Bachelor of Engineering (BE) in a relevant field.
Additional Information
- Experience. Minimum of 3 years of experience in Python programming and data engineering.
Summary
In this role, you will be integral to managing and optimizing data infrastructure to meet organizational data needs. You'll be expected to work independently, contribute to team discussions, and provide solutions to complex data challenges. If you're passionate about data engineering and eager to make a significant impact, we encourage you to apply.