Quality Analyst
Scale AI is seeking English-speaking AI Media Evaluators to assess AI-generated conversations, images, videos, and audio. This remote, project-based opportunity is suited to detail-oriented contributors who can make evidence-based judgments and communicate their observations clearly. Work is completed through the Outlier platform, where contributors help improve advanced AI systems through evaluation and feedback. About The Company Scale AI builds data and evaluation solutions that support the development of advanced artificial intelligence. Through Outlier, its AI training platform, independent experts contribute to projects that help AI systems learn, reason, and perform more effectively in real-world settings. Outlier brings together a global community of contributors working on a range of AI training and evaluation tasks. About The Role As an AI Media Evaluator, you will review AI-generated media and assess how well it meets specific prompts and quality standards. Tasks may include holding scenario-based conversations with AI voice models, comparing generated images or videos, evaluating speech, and writing concise explanations of your decisions. You will work remotely and choose when to complete available tasks, subject to project requirements and availability. This is freelance project work, not an employee position, and does not guarantee a set number of hours or ongoing assignments.
Qualifications
Fluent in English, with strong written communication skills and comfort speaking naturally in everyday scenarios. Strong attention to detail and patience when reviewing visual and audio content, including replaying clips or zooming in to inspect specific elements. Ability to make clear, consistent decisions when comparing outputs, including when both options have noticeable flaws. Experience writing concise, evidence-based reviews, explanations, or evaluations. Comfort with digital tools and remote project work. A bachelor’s degree or higher is an asset, but is not required for all projects. Applicants must have a valid government-issued ID and mobile phone from their country of residence, along with an up-to-date resume and a LinkedIn profile that reflects their education and work experience. Applicants are currently accepted from Australia, Canada, Ireland, New Zealand, the United Kingdom, and the United States. You must be authorized to work in your country of residence.
Responsibilities
Conduct live, scenario-based conversations in English with AI voice models and assess naturalness, clarity, and usefulness. Compare AI-generated images and videos with prompts or source material, evaluating instruction adherence, visual quality, realism, and artifacts. Inspect media carefully for issues such as distorted anatomy, flickering, unnatural movement, inaccurate text, or mismatched audio and video. Evaluate generated speech for naturalness, pronunciation, and similarity to a target voice; some assignments may require a short recorded explanation. Label first-person videos of people or robots performing hands-on tasks by reviewing action sequences and correcting or refining AI-generated caption drafts. Provide specific written feedback that identifies what you observed or heard and explains your ratings. Follow project guidelines and complete assigned work accurately and consistently.
Benefits
Flexible remote work with no minimum weekly hours; set your schedule around available projects. Project compensation may be up to USD $15 per hour. Rates vary according to expertise, assessment results, location, project requirements, and task type. Payments are generally processed weekly on Tuesdays for work completed during the previous Tuesday-to-Monday pay period. Available payment methods include PayPal, AirTM, and ACH transfer, subject to eligibility. Some projects may offer additional incentives. Outlier reports that contributors have earned an average of 7.5% in additional Mission incentives, with the top quartile averaging 11%; incentive availability and amounts vary. Access to leading large language models and opportunities to contribute feedback on developing AI systems alongside a global expert community. To apply, create an Outlier account, upload your resume and valid ID, and complete the required onboarding steps. Rates, task availability, and project terms are provided through the platform. Lower rates may apply to onboarding or other non-core work. Review the applicable terms for each project before accepting tasks. Equal Opportunity Scale AI welcomes qualified applicants from eligible locations and considers contributors based on relevant skills, experience, and project requirements. This opportunity is offered through Outlier as an independent-contractor engagement; contributors are not employees of Scale AI. The opportunity does not include employment benefits, internships, or visa sponsorship. International students should consult a tax or immigration professional regarding their individual circumstances.