Remote Position
[Remote] AI Operations Specialist/ AI LLM Evaluation Specialist
🌍 Remote · United States
💼 Full-time
Note: The job is a remote job and is open to candidates in USA. Trilyon, Inc. is seeking an AI Operations Specialist / AI LLM Evaluation Specialist to review AI-generated customer responses and handle inquiries requiring human attention. The role uses AI tools to categorize questions, improve response quality, and build automated workflows in collaboration with cross-functional teams.
Responsibilities
- Review and improve AI-generated email responses
- Handle customer inquiries and identify cases that need human attention
- Use AI tools to categorize questions and draft accurate responses
- Build and improve automated workflows to reduce repetitive manual work
- Identify common issues and help improve response quality
- Work with cross-functional teams to ensure accurate and consistent communication
Skills
- Experience in customer support, operations, Trust & Safety, or a similar role
- Hands-on experience using AI tools or LLMs to review responses or automate tasks
- Strong written communication and attention to detail
- Good judgment when reviewing AI-generated content
- Experience with workflow automation is a plus
- Familiarity with Claude, ChatGPT, Python, JavaScript, or AI agents is an advantage
Company Overview