Senior Video Data Annotation & Quality Assurance Specialist – LATAM Remote – AI Model Evaluation & Trust & Safety
About brighthush
At brighthush, we are on a mission to empower the next generation of artificial intelligence with human insight. Our team of data scientists, engineers, and domain experts collaborate to build reliable, safe, and high‑quality AI systems that serve people worldwide. We pride ourselves on a culture of curiosity, continuous learning, and impact‑driven work. If you thrive in a fast‑moving, remote environment and want to shape the future of AI, brighthush is the place for you.
Why This Role Matters
As a Senior Video Data Annotation & Quality Assurance Specialist, you will be the critical bridge between raw model outputs and the polished, trustworthy data that fuels brighthush’s AI products. Your meticulous evaluations will turn unstructured video responses into auditable labels, rationales, and actionable insights for our modeling team. By ensuring that every piece of data meets our rigorous quality standards, you help us deliver safer, more accurate AI experiences to users around the globe.
Key Responsibilities
- Model Output Evaluation: Review video data entry model responses against a versioned rubric, assigning severity tags and selecting the superior answer with a clear, written rationale.
- Edge‑Case Identification: Detect hallucinations, instruction‑following failures, and unsafe content, tagging them with structured categories for downstream review.
- Rubric Calibration: Maintain reviewer‑quality scores by calibrating against gold‑standard examples each week, ensuring consistency across the team.
- Ambiguity Management: Capture ambiguous prompts and route them back to the program team for rubric updates, improving clarity for future evaluations.
- Failure Mode Documentation: Identify recurring failure patterns and document them so the modeling team can target these issues in subsequent training runs.
- Batch Auditing: Perform comprehensive audits of large batches (e.g., 50‑row sets), reporting drift and ensuring rubric consistency.
- Rubric Enhancement: Propose clarifications or updates to the rubric when you spot recurring ambiguities or errors, contributing to continuous improvement.
Essential Qualifications
- Prior experience as a human rater, annotation specialist, or evaluator on video data entry or related content.
- Comfort applying multi‑page rubrics consistently across long batches of data.
- Strong written reasoning skills: ability to name the issue and reference the specific rubric clause applied.
- Exceptional attention to detail and the capacity to flag when a prompt itself is problematic.
- Reliable asynchronous availability for at least 10 hours per week, with the flexibility to adjust as project needs evolve.
Preferred Background
- Academic or professional background in linguistics, content moderation, or trust & safety.
- Experience with inter‑rater agreement metrics and calibration cycles.
- Domain expertise that enables you to spot subject‑matter errors that automated checks miss.
Core Skills & Competencies
- Model output evaluation and comparison
- Rubric‑based annotation and severity tagging
- Inter‑rater calibration and quality control
- Video data entry evaluation and analysis
- Design thinking for rubric improvement
- Strong written communication and documentation
- Analytical mindset with a focus on data integrity
Career Growth & Learning Opportunities
At brighthush, we believe that every role is a stepping stone to greater impact. As a Video Data Entry Specialist, you will:
- Gain deep exposure to cutting‑edge AI model development and evaluation pipelines.
- Collaborate with cross‑functional teams, including data scientists, product managers, and policy experts.
- Contribute to the creation of industry‑leading quality standards and best practices.
- Participate in regular knowledge‑sharing sessions and workshops.
- Receive mentorship from senior data scientists and quality assurance leaders.
- Have the opportunity to transition into roles such as Data Quality Lead, Trust & Safety Analyst, or AI Research Associate.
Work Environment & Culture
brighthush is a fully remote, distributed organization that values autonomy, transparency, and collaboration. Our culture is built on:
- Flexibility: Work from anywhere in the U.S. with a schedule that fits your life.
- Community: Weekly virtual coffee chats, hackathons, and cross‑team knowledge sessions.
- Impact: Your work directly influences the safety and reliability of AI products used by millions.
- Diversity & Inclusion: A commitment to creating a welcoming environment for people of all backgrounds.
- Continuous Learning: Access to online courses, conferences, and internal training programs.
Compensation & Benefits
As a contractor, you will receive a competitive hourly rate that is confirmed after the interview process. In addition to fair pay, you will enjoy:
- Flexible working hours and the ability to choose your own workspace.
- Access to a digital benefits package, including health, dental, and vision insurance options for eligible contractors.
- Paid time off for holidays and personal days, subject to contract terms.
- Professional development stipend for courses, certifications, or conferences.
- Regular performance reviews and feedback loops to support your growth.
Application Process
Ready to bring your expertise to brighthush? Follow these steps:
- Submit your application through our specialist intake portal. This will route your profile to the appropriate hiring team.
- Complete a short evaluation task that demonstrates your rubric application skills.
- Participate in a brief interview with our hiring manager and a senior data scientist.
- Receive a detailed project scope, schedule, and contractor terms before placement.
We value diversity and encourage candidates from all backgrounds to apply. If you are authorized to work in the U.S. and can commit to at least 10 hours per week, we would love to hear from you.
Why Join brighthush?
At brighthush, you will be part of a forward‑thinking team that is redefining how AI interacts with the world. Your meticulous evaluations will shape the safety and reliability of products that touch millions of lives. You will work alongside passionate experts, receive continuous learning opportunities, and enjoy a flexible, remote work environment that respects your personal and professional growth.
Take the next step in your career and help us build AI that is not only intelligent but also trustworthy. Apply today and become a key contributor to brighthush’s mission of creating responsible, high‑quality AI solutions.
Apply for this job