AI Agent Tester Real-World Tasks - Remote

Job Title: AI Agent Tester - Real-World Tasks Role Type: Contractor Location: United States Job Summary We are seeking detail-oriented, U.S.-based AI Agent Testers to evaluate how AI assistants handle everyday tasks. You will test two AI agents using your own accounts and smartphone, assess their performance, and provide feedback to improve their real-world usability.

Key Responsibilities

Test AI agents on tasks such as shopping, bookings, bill payments, returns, email follow-ups, travel, and online forms. Approve or stop agents at important decision points. Screen-record every task attempt. Score performance based on completion, quality, user control, satisfaction, and time. Provide clear written feedback on the overall user experience.

Required Qualifications

Must be based in the United States. Bachelor's degree required. 2-3 years of professional experience, preferably in engineering, business management, or operations. Regular use of AI tools in daily life. Strong attention to detail, written communication, and documentation skills. An active personal AI assistant account (Plus tier), active Muse and Dots access, a smartphone with screen-recording capability, and a payment card for reimbursable test purchases. No coding or AI technical background is required. Generalists are welcome, and domain experts are preferred. Project Details Kickoff: Project is already active. Onboarding: Within 24-48 hours of selection. Work Schedule: Flexible, entirely from your phone. Task Duration: Approximately 1.5 hours per run, including evidence and scoring. Selection Process Apply and complete the screening questions. Attend a 15-30-minute interview. Complete the profile review. Receive onboarding instructions from the Talent Operations team. Why Join? Work with AI tools and evaluate real-world performance. Enjoy flexible, phone-based work. Help shape and improve next-generation AI agents.