[Remote] Business Analyst - AI Response Evaluation Specialist

Note: The job is a remote job and is open to candidates in USA. Gramian Consulting is hiring a Business Analyst - AI Evaluation Specialist to support a project assessing and comparing chatbot responses. The role evaluates AI-generated outputs against business requirements and quality criteria, analyzes processes, and delivers structured assessments to improve AI system accuracy and usefulness.

Responsibilities

• Gather, analyze, and document business requirements, stakeholder expectations, and project objectives
• Compare multiple chatbot responses and assess their accuracy, relevance, completeness, and alignment with defined evaluation criteria
• Conduct structured evaluations and provide clear, evidence-based final assessments of AI-generated outputs
• Translate business requirements into functional specifications, evaluation criteria, and actionable documentation
• Analyze workflows, business processes, and data to identify gaps, inefficiencies, and improvement opportunities
• Create reports, dashboards, process maps, and other project documentation
• Support project planning, testing, implementation, and user acceptance testing (UAT)
• Collaborate with technical teams and business stakeholders to ensure project requirements are understood and met
• Track project progress, risks, dependencies, and key performance indicators (KPIs)
• Present findings, insights, and recommendations to support informed decision-making
• Maintain consistent evaluation quality and meet project deadlines and task completion targets

Skills

• Bachelor's degree in Business Administration, Finance, Computer Science, or a related field
• At least 3 years of professional experience as a Business Analyst or in a similar analytical role
• Strong analytical thinking, problem-solving, and decision-making skills
• Proficiency in Excel and experience with SQL, Power BI, Tableau, or comparable analytical and reporting tools
• Experience gathering requirements, documenting functional specifications, and mapping business processes
• Strong attention to detail and the ability to compare complex information against defined criteria
• Excellent written communication, reporting, and documentation skills
• Ability to collaborate with cross-functional teams and manage multiple priorities independently

Company Overview