Remote Position
[Remote] Staff Software Engineer (AI Evals)
🌍 Remote · United States
💼 Full-time
Note: The job is a remote job and is open to candidates in USA. G2i is seeking an experienced software engineer to evaluate the quality of interactions with AI coding agents. The role focuses on assessing usefulness, engineering judgment, explanations, and reasoning, and providing feedback to help define high-quality interactions.
Responsibilities
- Evaluate AI-generated coding interactions end-to-end
- Judge whether outputs are:
- Useful
- Correct (at a high level)
- Aligned with how a strong engineer would think
- Assess the quality of explanations and reasoning, not just code
- Distinguish between different levels of response quality (e.g. what makes something a •2 vs 4•)
- Provide clear, opinionated feedback on:
- What worked
- What didn’t
- What felt “off” or misleading
- Help define what •great• looks like when interacting with tools like Cursor
Skills
- Staff / Principal-level engineer (or equivalent experience)
- Strong background in one of the below: + Go + Python + Rust + TypeScript / JavaScript
- Hands-on experience using: + OpenAI Codex + Claude Code + Cursor
- Deep familiarity with modern AI-assisted dev workflows
- Able to evaluate code without needing to fully execute or deeply review every line
- Comfortable giving direct, opinionated feedback
- High bar for what “good engineering” looks like
- Experience with tools like Cursor or similar AI-first IDEs
- Prior exposure to prompt design or evaluation workflows
- Experience mentoring senior engineers or defining engineering standards
Benefits
- Remote work
Company Overview