About the role
Blue Machines AI builds a production AI agent platform used by large enterprise customers to automate customer interactions at scale — across phone calls, web chat, WhatsApp, and email. The same agent has to hold up whether it's handling a live call, a chat thread, or an email, each with its own failure modes: a voice call breaks on latency and STT/TTS accuracy, a chat session breaks on state and context handling, WhatsApp breaks on message formatting and delivery, email breaks on threading and turnaround. A regression in any of these isn't a broken UI — it's a customer's live support channel going down.
We're hiring a Senior QA Engineer to own quality across both the agents themselves and the platform that builds and runs them: the agent builder/configuration flows, workflow orchestration, integrations, and the admin surfaces customers use to manage their agents. You'll build automated coverage for our APIs, services, and agent flows, and go deep on the channel-specific and platform-specific failure modes that a generic QA playbook won't catch. You'll work directly with engineering, not behind a ticket queue, and your calls on release-readiness will carry real weight.
What you'll do
- Design and maintain automated test suites (API, integration, and regression) and wire them into CI/CD so nothing ships without passing them.
- Build and run test coverage across every channel the agent operates in: voice (call flow, STT/TTS accuracy, latency and timing, degraded network conditions), web chat (state and context handling across turns), WhatsApp (message formatting, delivery, session handling), and email (threading, formatting, response turnaround) — with the same conversation logic and business rules holding up consistently across all of them.
- Test the platform layer itself, beyond individual agent conversations: agent builder/configuration flows, workflow orchestration, third-party integrations, and the admin/dashboard surfaces customers use to manage and monitor their agents.
- Drive manual and exploratory testing for new features, new channels, and new platform capabilities before automation coverage exists, with a bias toward finding the edge cases automation won't catch.
- Write and maintain test plans, test cases, and bug reports with enough evidence and reproducibility that engineering can act on them without back-and-forth.
- Partner with engineering on release sign-off — you're expected to have and defend an opinion on whether a release is safe to ship.
- Set up and improve monitoring/alerting signals that catch quality regressions in production, not just pre-release.
- Mentor and raise the testing bar across the team as we grow QA from an individual function into a practice.
What we're looking for
- 4–6 years in QA/SDET roles, with real ownership of both manual and automated testing — not just executing a fixed test suite.
- Hands-on experience with test automation frameworks (e.g., Selenium, Playwright, or Cypress) and API testing tools (Postman, REST-assured, or similar).
- Working knowledge of CI/CD pipelines and integrating automated tests into them (Jenkins, GitHub Actions, GitLab CI, or similar).
- Experience testing conversational or multi-channel systems — any combination of voice/IVR, chat/messaging platforms, or email-based workflows — and the judgment to reason about failure modes in channels or platform surfaces (builders, rules engines, integrations) you haven't tested before.
- Comfort reading logs/traces to root-cause a failure rather than just filing "doesn't work."
- Basic scripting ability (Python or JavaScript) to build test tooling, not just consume it.
- Clear written communication — your bug reports and test plans need to hold up without you in the room.
Nice to have
- Direct experience testing STT/TTS pipelines, WhatsApp Business API integrations, or conversational AI/LLM-based products.
- Familiarity with SIP, telephony carrier integrations, or call quality metrics (jitter, packet loss, MOS scores).
- Experience with performance/load testing tools (k6, JMeter, Locust).
- Prior experience setting up a QA function or automation practice from scratch.
Why Blue Machines
You'll help shape how quality gets built into a fast-moving, multi-channel AI agent platform — with real ownership over what ships, working directly alongside the engineers building it.
