Software Engineer, DevOps AI Rater/Evaluator

Job Description:

• Evaluate AI outputs related to software engineering, DevOps, and infrastructure topics
• Perform structured scoring, comparison, classification, and judgment tasks
• Assess technical correctness, completeness, security implications, and best-practice alignment
• Identify hallucinations, incorrect code, unsafe recommendations, or misleading system guidance
• Apply domain-specific engineering and DevOps guidelines consistently across tasks Requirements:
• Software engineers, site reliability engineers, DevOps engineers, or platform engineers
• Experience with production systems, CI/CD pipelines, cloud infrastructure, or distributed systems
• Strong attention to detail and comfort working with structured evaluation criteria
• Native or professional fluency in one or more supported languages is required
• English fluency is required for guidelines, feedback, and collaboration Benefits:
• Contract-based, flexible participation
• Project-based work with clear expectations and timelines
• Opportunities for recurring work based on performance and demand
• Compensation communicated upfront per project or task type Apply tot his job