As a Senior Test Automation Engineer specializing in Data Engineering, you will play a critical role in building scalable, robust, and intelligent test automation systems that ensure the reliability and integrity of Visa's data-centric applications. You’ll work at the heart of our distributed data systems, shaping how test frameworks are designed and implemented for Spark and Hadoop-based batch applications.
This role is ideal for someone with a deep technical background in big data systems, a sharp eye for detail, and a strong passion for building best-in-class automation solutions that drive quality and performance at scale.
What You’ll Do
Test Architecture & Automation
- Architect and develop high-performance, maintainable, and reusable automation frameworks tailored for big data applications running on Apache Spark and Hadoop.
- Design intelligent test strategies that encompass data validation, system reliability, and performance assurance across complex data pipelines.
- Automate end-to-end testing of batch processing systems, ensuring data transformations are accurate, reliable, and compliant with business rules.
- Develop and integrate REST API automation test suites for data services using tools such as RestAssured, Postman, or equivalent frameworks.
Data Quality Engineering
- Implement comprehensive data validation mechanisms to verify transformations, consistency, and integrity across distributed systems.
- Design and implement data quality frameworks to detect anomalies, mismatches, or regressions in large-scale datasets.
- Perform root cause analysis for data processing issues and partner closely with data engineering teams to proactively mitigate quality risks.
Performance & Scalability Testing
- Create and execute performance testing plans for data processing workloads to simulate real-world traffic and usage scenarios.
- Identify bottlenecks in data pipelines and recommend improvements to ensure scalability and system robustness.
Tooling & CI/CD Integration
- Integrate test automation suites into CI/CD pipelines using Jenkins, GitLab CI, or GitHub Actions.
- Continuously enhance automation tooling, drive adoption of best practices, and reduce time-to-detection for defects across data products.
Collaboration & Leadership
- Collaborate with data engineers, architects, DevOps, and product owners to gather requirements and influence product design for testability.
- Provide technical leadership across multiple test automation initiatives and mentor junior engineers in best practices and tooling.
- Stay informed about evolving industry trends in big data, automation testing, and distributed systems and evaluate new tools or approaches accordingly.
Qualifications
Basic Requirements
Preferred Qualifications
- 7+ years of experience in building and maintaining test automation solutions in a data engineering environment.
- Expert-level proficiency in Apache Spark, Hadoop, and distributed data processing systems.
- Advanced scripting and programming skills in Python, Scala, or Java.
- Proven experience with REST API testing tools and frameworks (e.g., Postman, RestAssured).
- Strong command of SQL and deep understanding of relational and NoSQL databases.
- Hands-on experience with CI/CD systems and automation in modern development pipelines.
- Knowledge of mocking frameworks, synthetic test data generation, and test metrics reporting.
- Experience developing custom tools when commercial/open-source solutions fall short.
- Track record of mentoring, code reviews, and leading testing strategies in agile environments.
- Excellent communication and cross-team collaboration skills.
- A growth mindset with the ability to learn and adapt quickly in a fast-paced, evolving technical landscape.
Why Join Visa?
- Be part of a transformative journey in shaping the future of business payments.
- Work with some of the brightest minds in the industry and on impactful global projects.
- Thrive in a culture that values inclusion, innovation, and continuous learning.
- Enjoy competitive compensation, flexible work arrangements, and comprehensive benefits.
Visa is committed to creating a diverse and inclusive workplace. We are proud to be an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender identity or expression, sexual orientation, national origin, disability, age, or veteran status. We also consider qualified applicants with criminal histories, consistent with applicable laws.