Headquartered in Los Altos, CA. Backed by SAIGroup.
Learn more about JazzX AI:
Website: https://jazzx.ai
LinkedIn: https://www.linkedin.com/company/jazzx-ai
Learn more about SAIGroup:
Website: https://saigroup.ai
Role Overview
We are looking for a QA Lead who can architect the overall quality strategy for JazzX AI’s platforms and AI-powered products. This role requires a blend of traditional QA expertise, AI/ML awareness, and strong leadership capabilities.
You will lead the quality roadmap for agent workflows, LLM outputs, APIs, and cloud services while managing a high-performing QA team.
Key Responsibilities
- Quality Strategy & Leadership
- Build and own JazzX AI’s end-to-end QA strategy across AI copilots, workflows, and enterprise deployments.
- Lead, mentor, and scale a team of QA engineers and automation specialists.
- Partner closely with Product, ML Engineering, and Platform Engineering to embed quality in early design phases.
- Define and enforce quality gates and release-readiness standards.
- AI/LLM-Specific Quality Assurance
- Develop evaluation systems for LLM accuracy, hallucination detection, factuality, safety, and bias mitigation.
- Design scalable tests for agent reasoning, tool-use correctness, and multi-step workflow validation.
- Create synthetic datasets and scenario-based tests for edge-case prediction and regression testing.
- Validate model performance across fine-tuning, RAG, knowledge-grounding, and inference scenarios.
- Automation Engineering
- Architect and manage automation frameworks using Python/Java/JS, Playwright, Selenium, PyTest, Postman, etc.
- Ensure seamless integration of automated suites into CI/CD pipelines (GitHub Actions/GitLab/Jenkins).
- Automated API, microservices, and backend testing for JazzX AI’s cloud architecture running on AWS/GCP.
- Performance Testing
- Load Testing, Stress Testing, Spike Testing, Endurance/Soak Testing using industry-standard tools like JMeter, LoadRunner, Gatling, k6, or Locust.
- Release, Monitoring & Reliability
- Own the release sign-off process for all AI software components and platform features.
- Conduct RCA, maintain issues of dashboards, and drive continuous improvement in QA lifecycle.
- Collaborate with DevOps/ML Ops to monitor model drift, performance anomalies, and post-production behaviors.
- Documentation & Communication
- Prepare detailed QA documents: strategies, test plans, evaluation reports, and execution metrics.
- Provide clear communication of risks, quality insights, and impact analysis to leadership.
Required Qualifications
- 5+ years of experience in Software QA.
- Strong hands-on experience with test automation frameworks and scripting (Python/Java/JavaScript).
- Experienced in working with microservice-based architecture
- Working knowledge of AI/ML systems, including datasets, model evaluation, vector stores, RAG, and LLM operations.
- Architect and manage automation frameworks using Python/Java/JS, Playwright, Selenium, PyTest, Postman, etc.
- Automated API, microservices, and backend testing for JazzX AI’s cloud architecture running on Azure /AWS.
- Experience testing distributed cloud systems, APIs, and high-performance backend services.
- Strong analytical and problem-solving skills with the ability to drive organization-wide quality initiatives.
Preferred Skills
- Experience testing LLMs, RAG pipelines, or autonomous AI agents.
- Familiarity with AI observability tools, logging/monitoring stacks (ELK, Prometheus, Datadog).
- Exposure security testing, performance/load testing, and compliance (SOC2, GDPR).
- Ability to operate in a fast-paced, zero-to-one startup environment.
Why Join JazzX AI -
What draws people to JazzX AI is the opportunity to build something truly foundational.
We are at the beginning of a new era — one where enterprise systems will no longer be static tools, but dynamic collaborators that learn, reason, and evolve alongside the people who use them. At JazzX AI, you’ll help build a platform that redefines how work happens in the AGI era.
JazzX AI brings together vision, deep technology, and purpose. This is not about experimenting with AI features; it’s about turning the promise of AGI into practical, auditable, and human-aligned systems that enterprises can trust and rely on at scale. The work here directly impacts productivity, efficiency, and decision-making in complex, real-world environments.
If you’re motivated by building systems that matter — systems that combine intelligence with responsibility — JazzX AI offers the chance to do that work at a deeper level, with real ownership and lasting impact.
