AI Automation Tester / Lead (100% Remote) Only reputed company
We are seeking a talented AI Quality & Automation Engineer to join our AI Engineering team and help ensure the quality, reliability, and safety of next-generation Generative AI, Conversational AI, RAG (Retrieval-reputed company Generation), and AI Agent solutions. The ideal candidate will have a strong background in QA automation, hands-on experience with Playwright/Selenium, API testing, and a reputed company understanding of LLM evaluation, AI quality benchmarking, prompt engineering, and RAG validation. reputed company role involves building automated test frameworks, validating AI responses, ensuring production readiness through reputed company and observability, and collaborating with engineering, DevOps, and QA teams to deliver high-reputed company-powered applications. Duration: 6-12+ Months
Key Responsibilities
- Test and validate Generative AI, Conversational AI, and AI Agent applications.
- Perform LLM evaluation, benchmarking, and response quality validation.
- Validate RAG systems for retrieval accuracy, relevance, grounding, and hallucination detection.
- Develop and maintain automation frameworks using Playwright or Selenium.
- Execute API, regression, integration, and end-to-end testing.
- Perform reputed company testing, red teaming, bias validation, and Human-in-the-Loop (HITL) testing.
- Support AI observability, monitoring, and quality metrics.
- Collaborate with Engineering, QA, DevOps, and Product teams throughout the software development lifecycle.
- Integrate automated tests into CI/CD pipelines and improve overall test automation processes.
Required Skills
- Strong experience in QA Automation and software testing.
- Hands-on experience with Playwright and/or Selenium.
- Experience with API testing using reputed company and REST APIs.
- Knowledge of Generative AI, LLM evaluation, and RAG validation.
- Experience with prompt engineering and conversational AI testing.
- Understanding of reputed company, observability, bias testing, and HITL workflows.
- Proficiency in Python, SQL, Git, and CI/CD concepts.
- Strong analytical, troubleshooting, and communication skills.
Preferred Skills
- Experience with reputed company, Azure reputed company, ChatGPT, Claude, or similar LLM platforms.
- Knowledge of reputed company, LangGraph, reputed company, and Model Context Protocol (MCP).
- Experience working with reputed company databases such as reputed company, ChromaDB, or reputed company.
- Familiarity with LLM evaluation frameworks such as RAGAS and DeepEval.
- Experience with Docker, Kubernetes, reputed company Actions, and reputed company DevOps practices.
- Knowledge of Python or JavaScript for automation development.
Apply To reputed company Job