We are seeking a Lead Data Engineer who will be responsible for building and maintaining the VDC platform and ensuring its alignment with the evolving needs of the organization. This is an exciting opportunity to work with cutting-edge technologies, contribute to open-source projects, and drive Visa’s data architecture into the future. As a Lead Data Engineer, you will collaborate with internal teams and external stakeholders, playing a key role in developing a centralized repository for all data-related information.
Key Responsibilities
- Platform Development and Maintenance. Lead the design, development, and continuous improvement of the Visa Data Catalog (VDC) platform. Ensure the platform evolves to meet Visa's dynamic needs while maintaining high availability and performance.
- Metadata Management. Oversee the metadata ingestion process, ensuring the catalog remains up-to-date with new data definitions, business context, and evolving business requirements. Collaborate with data stewards and other stakeholders to enhance metadata with business definitions, data lineage, and usage context.
- Search and Discovery. Enhance the platform’s search and discovery capabilities to ensure users can intuitively find the data they need, empowering business users to unlock the full potential of the data assets.
- Integration with Enterprise Systems. Integrate the VDC with other enterprise systems and tools that support Visa’s data governance, data quality, and analytics initiatives. Ensure smooth interoperability with Visa’s broader data architecture.
- Performance Optimization. Monitor and optimize the platform’s performance for scalability, ensuring that the VDC can handle increasing data volumes and user traffic.
- Training and Support. Provide training and support to internal users on how to use the VDC effectively for their data needs, ensuring that the organization maximizes the value of its data assets.
- Open-Source Contribution. Actively contribute to the open-source Datahub project, leveraging community-driven improvements and keeping the VDC aligned with industry best practices.
- Technology Advancement. Stay updated on the latest developments in the metadata management and data cataloging space. Leverage these advancements to keep Visa’s data platform cutting-edge and scalable.
- Code Development. Dedicate 70% of your time to writing code in various languages and frameworks, including Java, Python, and SQL/NoSQL technologies. You will also engage in designing and developing end-to-end solutions.
Qualifications
Basic Qualifications
- A Bachelor’s degree in Computer Science, Information Technology, or a related technical discipline. Advanced degrees (e.g., Master’s, MBA, PhD) are a plus.
- 10+ years of relevant work experience with proven hands-on expertise in designing and developing large-scale data management platforms.
Preferred Qualifications
- 12-15 years of experience in building and scaling data management platforms, including backend development and architecture design.
- Proficiency in programming languages. Deep hands-on experience in Java/Python for backend development, with an understanding of various frameworks.
- Database Expertise. Advanced knowledge in working with both SQL (Oracle, DB2, SQL Server) and NoSQL technologies.
- Web Services. Extensive experience with RESTful APIs, SOAP, and gRPC for building data-centric applications and services.
- Frontend Skills. Knowledge of React or Angular for front-end development (nice to have for building user-friendly interfaces).
- Big Data and Cloud Technologies. Solid experience in leveraging big data platforms such as Hadoop, Hive, Spark, and Kafka. Understanding cloud-based data infrastructure is essential.
- Service Architecture Expertise. Proven experience in designing high-performance, scalable service architectures for large-scale data applications.
- Data Management Experience. Deep understanding of key data management principles, including data acquisition, ingestion, processing, and distributed processing at scale.
- Leadership Skills. Strong leadership and mentoring capabilities, with experience growing junior engineers and fostering team collaboration.
- Problem-Solving and Innovation. Ability to solve complex data challenges and continuously seek innovative solutions, staying ahead of emerging industry trends.
- Industry Experience. Payment industry experience is a plus, but not mandatory.
Additional Information
Visa is an Equal Opportunity Employer. We embrace diversity and encourage applications from individuals of all backgrounds. We consider applicants for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability, or protected veteran status. Visa is committed to providing reasonable accommodations for applicants with disabilities.
Why Visa
Visa offers an exciting work environment where you can collaborate with talented teams, contribute to innovative projects, and grow your career in the world’s leading payments technology company. If you're ready to work at the intersection of cutting-edge technology and business, Visa provides you with the tools and support to build your future.
How to Apply
If you are a skilled data engineer looking for a rewarding career at Visa, apply today to join a dynamic, global company where you will make a significant impact on the world of payments.