[Remote] Domain Expert, AI Evaluation Task Writer: Finance (Contract, Remote)

Note: The job is a remote job and is open to candidates in USA. Patronus AI is hiring a finance domain expert to create realistic tasks that evaluate and train frontier AI models. The role involves building scenario-based evidence files, writing task requests and reference solutions, defining grading criteria, and testing and reviewing tasks for quality.

Responsibilities

• Build realistic evidence files from a provided scenario: bank statements and cash positions, deal confirmations and settlement records, trial balances and GL extracts, audit workpapers and control evidence, trade blotters and rebalance instructions, position and cash reconciliations, cap tables, grant records and board consents
• Write clear task requests that state what the deliverable is and how uncertainty and justified assumptions should be handled
• Solve each task end to end and document a reference solution with the reasoning behind every step
• Write explicit scoring criteria that let two independent graders reach the same verdict, including acceptable alternative answers
• Run tasks against current models, find where and why they fail, and adjust difficulty
• Check your own and peers' tasks for ambiguity, errors, answers exposed in the materials and shortcuts that skip the intended reasoning
• Incorporate reviewer feedback within agreed turnaround times and flag guideline gaps or edge cases you run into

Skills

• 3+ years of hands-on experience in one or more of the four areas above: treasury operations, audit and assurance, trading and portfolio operations, or equity administration
• Working command of the standards, controls and rules in your area (for example US GAAP or IFRS, PCAOB/AICPA auditing standards and SOX, treasury policies and bank controls, trade compliance and restricted-list rules, equity plan documents and fair-market-value requirements)
• Strong spreadsheet skills, and the habit of tying every figure back to its source
• Clear, precise writing in professional English
• Ability to work independently and asynchronously against deadlines
• CPA, CA, ACCA, CISA, CTP, CFA or CEP
• Big Four, public company, bank, asset manager or fund experience
• Experience with the systems used in your area, such as SAP Treasury or another treasury management system, Bloomberg, AIM or EMSX, or Carta or Shareworks
• Prior experience writing or reviewing AI evaluation tasks, exam questions, case studies or training material
• Familiarity with how AI models fail (hallucinated figures, confident arithmetic errors, misread documents)

Benefits

• Fully remote, asynchronous work
• Potential bonus for each accepted task
• The trial task is paid

Company Overview

Company H1B Sponsorship