[Remote] Staff Test Automation Engineer, AI Tooling

Note: The job is a remote job and is open to candidates in USA. Function Health is hiring a Staff Test Automation Engineer to build AI-powered tools that generate trustworthy tests for a complex codebase. The role owns the technical direction and delivery of test-generation systems, integrating them into engineering workflows and collaborating across teams.

Responsibilities

• Build AI-powered test generation tools. Develop tooling that uses source code, requirements, API contracts, existing tests, and bug reports to generate useful unit, integration, and regression tests
• Test code that already contains defects. Create approaches that expose incorrect behavior without treating the current implementation as the source of truth. Clearly distinguish tests that document existing behavior from tests that verify intended behavior
• Establish reliable expected outcomes. Design ways to determine what a test should assert, reconcile conflicting evidence, and flag ambiguity for human review
• Turn failures into lasting coverage. Build workflows that reproduce reported bugs, reduce failures to clear test cases, and preserve regression tests after fixes
• Build the supporting infrastructure. Create safe, reproducible environments for generated tests, including dependency isolation, fixtures, test data, and controlled execution
• Evaluate test quality. Measure defect detection, mutation score, assertion quality, flakiness, and maintenance burden—not just the number of tests generated or lines covered
• Integrate into developer workflows. Bring test generation and review into local development, pull requests, and CI with clear explanations and actionable feedback
• Set technical direction. Define architecture, prioritize investments, mentor engineers, and establish standards for AI-assisted testing across teams

Skills

• Experience operating at Staff Engineer scope: leading ambiguous technical initiatives, making architectural decisions, and influencing multiple engineering teams
• Deep expertise in automated testing, including unit, integration, contract, and end-to-end testing, with sound judgment about where each adds value
• Strong programming skills and experience building production-quality developer tools, test frameworks, or engineering infrastructure
• Hands-on experience building LLM-powered applications, including code context retrieval, structured outputs, tool use, and evaluation
• Experience working with legacy or defect-prone systems, including isolating dependencies, diagnosing flaky tests, and introducing coverage incrementally
• An understanding of the limitations of AI-generated code: plausible but incorrect assertions, brittle mocks, missing edge cases, and tests that reproduce implementation mistakes
• Experience with CI/CD, reproducible environments, and safe execution of generated code
• Clear communication and the ability to turn uncertain requirements into concrete technical decisions

Benefits

• Flexible working hours
• A dynamic work environment where creativity and innovation are encouraged

Company Overview

Company H1B Sponsorship