ML Engineer, Retrieval & Grounded Generation
Redcellpartners · Remote, USA
About this role
**About Red Cell Partners / DEFCON AI** Red Cell Partners is an incubation firm building and investing in rapidly scalable technology-led companies across healthcare, cyber, and national security. DEFCON AI is an insights company leveraging AI, mathematical optimization, data analytics, and software engineering to enable resilient optimization of complex systems—helping customers anticipate, assess, and mitigate disruption impacts. --- **About the Role (QA Engineer — AI-Enabled Decision Support)** Own quality for an AI-enabled decision-support system: the test plan, automated coverage, and the evidence that what was delivered matches what was specified. This role sits on the **product side** of the organization (not engineering). You’ll report through the Product Lead, translate product acceptance criteria into executable tests, and hold engineering/platform/partners accountable to verifiable requirements. You’ll also write code and build real automation (not manage test cases), with enough fluency in the application code to test it well. As the first dedicated quality hire, you’ll establish the testing approach and standards. **Location / Travel**: Fully remote with occasional travel (up to **25%**) to DEFCON AI HQ, customer sites, and vendor facilities as required. --- **Key Responsibilities** **Test Strategy & Acceptance Criteria** - Own the system test plan (functional, integration, performance, and load testing) as a formal deliverable - Translate product acceptance criteria into executable tests - Review acceptance criteria with product before development begins; flag ambiguous or non-verifiable requirements - Define required evidence for changes to be considered complete - Work day-to-day with engineering while maintaining independent reporting of what testing shows - Plan and coordinate user acceptance testing (UAT) with customers; track findings to resolution **Test Automation & Coverage** - Build and maintain automated test suites; integrate into CI/CD as release gates - Establish and report against an automated coverage standard - Build performance/load testing that validates expected user load - Maintain regression coverage across releases **Measurement & Evidence** - Build and maintain labeled evaluation sets with limited-time subject-matter experts - Design and run sampled audits to confirm automated decisions are correct - Measure and report what the system misses (not just what it gets right) - Measure how often users override the system and track trends - Document how each committed measure is computed, what data it uses, and what evidence accompanies a release - Produce evidence independently of the people building the models **Release Governance** - Maintain version inventory across builds, models, configurations, and thresholds - Produce test reports, evaluation records, and rollback criteria for each release - Confirm test environments match production closely enough for meaningful results - Enforce the gate: no release ships without the acceptance evidence --- **Required Qualifications** - **5+ years** in software quality engineering, test automation, or software engineering with substantial dedicated test ownership - Clear point of view on how testing should be done; judgment to set and hold the standard - Strong coding ability for test automation (e.g., **Python + pytest**, or **JavaScript/TypeScript + Playwright/Cypress/Selenium**). Ability to read and reason about application code under test - Rou
Listing freshness
CronJobs last confirmed this listing 46m ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.