Senior Data Engineer- Java/Python/SQL/Spring-boot (4yrs+)

Company Description

Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose – to uplift everyone, everywhere by being the best way to pay and be paid.

Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa.

Job Description

The Natural Language Query (NLQ) platform is an innovative initiative designed to transform the way users interact with data. Our platform leverages advanced natural language processing (NLP) to convert user queries in plain language into executable SQL queries, enabling seamless data retrieval and analysis without the need for SQL expertise. The NLQ platform will be powered by the Enterprise Data Catalog, ensuring comprehensive and accurate metadata definitions for all table and column information. This platform empowers different business units to build external-facing conversational BI chatbots to handle customer requests, while also significantly reducing data exploration efforts for internal data analysts by more than 90%. 

Key Responsibilities:

• Provide Platform-as-a-Service offerings that are easy to consume, scalable, secure, and reliable using open source-based solutions.

• Develop and enhance the NLQ Copilot platform, ensuring it meets the evolving needs of multi-tenant environments.

• Implement Context builder algorithms by leveraging different prompt engineering techniques to generate 100% accurate SQL as per the customer needs.

• Collaborate with downstream clients to integrate business requirements, adding robust guardrails to prevent unauthorized query generation.

• Work closely with data scientists, engineers, and product managers to optimize the performance of the NLQ platform.

• Utilize cutting-edge NLP / LLM and machine learning techniques to improve the accuracy and efficiency of query transformations.

• Ensure the platform's scalability and reliability through rigorous testing and continuous improvement.

• Champion the adoption of open infrastructure solutions that are fit for purpose while keeping technology relevant.

• Spend 80% of the time writing code in different languages, frameworks, and technology stacks.

This is a hybrid position. Expectation of days in office will be confirmed by your hiring manager.

Qualifications

Basic Qualification
•4 yrs+ of experience in architecture design and development of large-scale
data management platforms and data application with simple solutions
•Bachelor's or master’s degree in Computer Science or related technical
discipline required
•Must have extensive hands-on coding and designing skills on
•Java/Python for backend
•MVC (model-view-controller) for end-to-end development
•SQL/NoSQL technology. Familiar with Databases like Oracle, DB2, SQL Server,
etc.
•Web Services (REST/ SOAP/gRPC)
•React/Angular for front-end (UI front-end nice to have)
•Expertise in design and management of complex data structures and data
processes
•Expertise in efficiently leveraging the power of distributed big data systems,
including but not limited to Hadoop Hive, Spark, Kafka streaming, etc.
•Strong service architecture and development experience with high
performance and scalability
•Strong on driving for results and self-motivated, strong learning mindset, with
good understanding of related advanced/new technology. Keep up with the
technology development in the related areas in the industry, which could be
leveraged to enhance current architectures and build durable new ones.
•Strong leadership and team player.
•Strong skills on mentoring/growing junior people
•Payment industry experience is a plus.
Preferred Qualification
•Deep knowledge and hands on experience on big data and cloud computing
technologies.
•Experience with LLM / GenAI tools / applications and Prompt Engineering
•Experience with ETL/ELT tools / applications
•Experience with Apache NiFi and Apache Spark for processing large data sets
•Experience on Elastic Search
•Knowledge on Data Catalog tools
•Experience in building Data Pipeline development tools
•Experience with Data Governance and Data Quality tools

Additional Information

Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.