Lead the quality strategy for AI-powered products, including LLM outputs, agent workflows, APIs, cloud services, and enterprise deployments. Manage and mentor QA teams, build automation frameworks, develop AI-specific evaluation and regression testing, oversee performance and API testing, integrate tests into CI/CD, manage release sign-offs, and monitor model reliability and production behavior. Partner with Product, ML, Platform, DevOps, and leadership teams to drive quality initiatives.
About JazzX AI:
Vision: Enterprises operating on institutional intelligence—governed, self-improving, and scalable beyond individual expertise.
JazzX AI is defining the future of enterprise work—by building AI-native digital workers that actually get the job done.
We believe enterprises don't scale expertise—they lose it. Knowledge stays trapped in individuals, judgment gets applied inconsistently, and the best talent spends time on work that should run itself. We're changing that.
JazzX AI transforms messy enterprise reality into institutional intelligence: governed digital workers that capture expert judgment, make every decision explainable, and continuously improve through real-world execution. The result is faster decisions, higher-quality outcomes, and reliable execution at scale—in domains where getting it wrong isn't an option.
We're starting with lending and due-diligence workflows—complex, regulated, high-stakes. From here, we're building the backbone for enterprise intelligence across industries.
This is early-stage, hard, and consequential work. If you want to be part of bringing AI systems to market that actually run in production, handle real complexity, and deliver real outcomes—not demos, not chatbots—JazzX AI is the place.
About SAIGroup :
SAIGroup a private investment firm that has committed $1B to build and scale next-generation, AI-powered enterprise software companies. SAIGroup’s portfolio serves 2,000+ global enterprise customers, generates nearly $800M in annual revenue, and employs 4,000+ people worldwide — providing JazzX AI with long-term capital, deep operating expertise, and access to real-world enterprise scale from day one.
Role Overview
We are looking for a QA Engineer who has the overall quality strategy for JazzX AI’s platforms and AI-powered products. This role requires a blend of traditional QA expertise, AI/ML awareness, and strong leadership capabilities.
You will lead the quality roadmap for agent workflows, LLM outputs, APIs, and cloud services while managing a high-performing QA team.
Key Responsibilities
- Quality Strategy & Leadership
- Build and own JazzX AI’s end-to-end QA strategy across AI copilots, workflows, and enterprise deployments.
- Lead, mentor, and scale a team of QA engineers and automation specialists.
- Partner closely with Product, ML Engineering, and Platform Engineering to embed quality in early design phases.
- Define and enforce quality gates and release-readiness standards.
- AI/LLM-Specific Quality Assurance
- Develop evaluation systems for LLM accuracy, hallucination detection, factuality, safety, and bias mitigation.
- Design scalable tests for agent reasoning, tool-use correctness, and multi-step workflow validation.
- Create synthetic datasets and scenario-based tests for edge-case prediction and regression testing.
- Validate model performance across fine-tuning, RAG, knowledge-grounding, and inference scenarios.
- Automation Engineering
- Architect and manage automation frameworks using Python/Java/JS, Playwright, Selenium, PyTest, Postman, etc.
- Ensure seamless integration of automated suites into CI/CD pipelines (GitHub Actions/GitLab/Jenkins).
- Automated API, microservices, and backend testing for JazzX AI’s cloud architecture running on AWS/GCP.
- Performance Testing
- Load Testing, Stress Testing, Spike Testing, Endurance/Soak Testing using industry-standard tools like JMeter, LoadRunner, Gatling, k6, or Locust.
- Release, Monitoring & Reliability
- Own the release sign-off process for all AI software components and platform features.
- Conduct RCA, maintain issues of dashboards, and drive continuous improvement in QA lifecycle.
- Collaborate with DevOps/ML Ops to monitor model drift, performance anomalies, and post-production behaviors.
- Documentation & Communication
- Prepare detailed QA documents: strategies, test plans, evaluation reports, and execution metrics.
- Provide clear communication of risks, quality insights, and impact analysis to leadership.
Required Qualifications
- 3+ years of experience in Software QA.
- Strong hands-on experience with test automation frameworks and scripting (Python/Java/JavaScript).
- Experienced in working with microservice-based architecture
- Working knowledge of AI/ML systems, including datasets, model evaluation, vector stores, RAG, and LLM operations.
- Architect and manage automation frameworks using Python/Java/JS, Playwright, Selenium, PyTest, Postman, etc.
- Automated API, microservices, and backend testing for JazzX AI’s cloud architecture running on Azure /AWS.
- Experience testing distributed cloud systems, APIs, and high-performance backend services.
- Strong analytical and problem-solving skills with the ability to drive organization-wide quality initiatives.
Preferred Skills
- Experience testing LLMs, RAG pipelines, or autonomous AI agents.
- Familiarity with AI observability tools, logging/monitoring stacks (ELK, Prometheus, Datadog).
- Exposure security testing, performance/load testing, and compliance (SOC2, GDPR).
- Ability to operate in a fast-paced, zero-to-one startup environment.
Why Join JazzX AI:
What draws people to JazzX AI is the opportunity to build something truly foundational.
We are at the beginning of a new era — one where enterprise systems will no longer be static tools, but dynamic collaborators that learn, reason, and evolve alongside the people who use them. At JazzX AI, you’ll help build a platform that redefines how work happens in the AGI era.
JazzX AI brings together vision, deep technology, and purpose. This is not about experimenting with AI features; it’s about turning the promise of AGI into practical, auditable, and human-aligned systems that enterprises can trust and rely on at scale. The work here directly impacts productivity, efficiency, and decision-making in complex, real-world environments.
If you’re motivated by building systems that matter — systems that combine intelligence with responsibility — JazzX AI offers the chance to do that work at a deeper level, with real ownership and lasting impact.
Headquartered in Los Altos, CA. Backed by SAIGroup.
Learn more about JazzX AI:
Website: https://jazzx.ai
LinkedIn: https://www.linkedin.com/company/jazzx-ai
Learn more about SAIGroup:
Website: https://saigroup.ai
Similar Jobs
Aerospace • Information Technology • Software • Cybersecurity • Design • Defense • Manufacturing
Leads software quality engineering and test automation activities, including designing automated frameworks, test plans, and scripts for UI, API, microservice, integration, regression, and performance testing. Integrates testing into CI/CD pipelines, manages defects, analyzes failures, validates software requirements, and collaborates with developers and stakeholders in an Agile environment. The role also supports test strategy, code reviews, static analysis, Azure-based applications, and continuous improvement of quality practices.
Top Skills:
Application InsightsArtifactoryAzureAzure Blob StorageAzure MonitorAzure PaasAzure Redis CacheBlazemeterCi/CdCucumberDockerEntity Framework CoreGitlabJasmineJavaJavaScriptJenkinsJmeterJunitMavenMicroservicesMicrosoft Azure DevopsMssqlPostmanRabbitMQRest ApisSeleniumSonarqubeSynopsys Coverity
Information Technology
Develop and execute automated and manual test suites for cybersecurity features across control-plane and data-plane software. Collaborate with product lifecycle management, development, and cross-functional teams to plan testing and releases. Automate repetitive testing, optimize system scalability and performance, validate cloud deployments, and support reliable feature delivery for a SASE security platform.
Top Skills:
CypressDockerGitJavaJIRAJunitKubernetesLinuxOpenapiPlaywrightPostmanPytestPythonRest ApisRest AssuredSeleniumTestngTypescriptWireshark
Healthtech • Software
Conduct manual functional, regression, integration, web application, API, and mobile testing. Create and maintain test plans, cases, procedures, and regression libraries; identify and track defects; report outcomes; review requirements and documentation; collaborate cross-functionally; estimate testing work; improve QA processes; and participate in on-call support.
Top Skills:
Api Testing ToolsAPIsMobile ApplicationsWeb Applications
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
