Remote Position
[Remote] Sr. Staff Software Engineer, AI Platform
🌍 Remote · United States
💼 Full-time
Note: The job is a remote job and is open to candidates in USA. Pinterest’s AI Platform team builds tools and infrastructure for AI and ML engineers across the company. The Senior Staff Software Engineer will shape and scale a unified, reliable AI/ML evaluation platform, set technical strategy, and partner across teams to improve quality, launch decisions, and trust in AI-powered experiences.
Responsibilities
- Identify modernization opportunities and scale the roadmap for a company-wide evaluation system in partnership with technical leadership
- Define cross-team technical strategy, align partner teams, and drive broad adoption of shared evaluation capabilities
- Make durable architectural decisions that improve the user experience and reliability of Pinterest’s AI/ML evaluation platform
- Establish evaluation methodologies and standards that balance automated metrics, expert judgment, human feedback, safety, and business outcomes
- Lead complex cross-team workstreams, resolve technical tradeoffs and risks, and deliver impact
Skills
- 3+ years of deep, hands-on experience designing and operating evaluation systems for GenAI and complex ML systems, including benchmark design, golden datasets, automated metrics, human evaluation, model-based grading, and failure analysis
- Strong programming skills in Python coupled with a solid grasp of distributed systems principles, and comfortable iterating with AI tools on coding and design
- Deep familiarity with GenAI and RecSys evaluation libraries and frameworks, AutoEvals, OpenEvals, Ragas, etc
- 5+ years of experience building scalable platforms for distributed experimentation, data and model versioning, orchestration, observability, release management, and production monitoring
- Strong judgment around GenAI risks, including robustness, bias, privacy, security, and the limitations of automated evaluation signals
- Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent experience
Benefits
- Eligible for equity.
- Can be situated anywhere in the country, with in-office collaboration 1–2 times per quarter.
Company Overview
Company H1B Sponsorship