[Remote] Forward Deployed Machine Learning Engineer

Note: The job is a remote job and is open to candidates in USA. ADVATIX - Advanced Supply Chain and Logistics is seeking a Forward Deployed Machine Learning Engineer to establish the technical foundation for AI model benchmarks and evaluations. The role combines machine learning engineering, backend infrastructure, data pipelines, evaluation systems, and customer-facing technical delivery, including owning projects from feasibility through production deployment.

Responsibilities

• Partner directly with the General Manager, researchers, and early customers to define, design, and build AI benchmarks and evaluation systems
• Develop benchmarks and evaluation frameworks across multiple AI domains and modalities
• Build and own backend infrastructure supporting AI model evaluation
• Design and maintain data pipelines, execution environments, storage systems, and orchestration infrastructure
• Build sandboxed environments for agentic evaluations involving tools, code execution, and multi-step tasks
• Develop and deploy end-to-end evaluation systems for foundation models
• Own the engineering component of customer engagements from initial requirements through technical delivery
• Work directly with enterprise customers to understand technical requirements and develop practical evaluation solutions
• Identify repeatable evaluation patterns that can be transformed into scalable infrastructure and product capabilities
• Identify infrastructure gaps and opportunities that can inform future product development
• Scope ambiguous technical problems, assess feasibility, and independently drive solutions through implementation and delivery
• Collaborate with AI researchers and other technical stakeholders to translate research concepts into reliable production systems
• Develop scalable systems capable of supporting large-scale machine learning evaluation and benchmark workloads
• Communicate technical concepts, project status, tradeoffs, and solutions clearly to customers and internal stakeholders

Skills

• Minimum 4+ years of professional engineering experience with hands-on machine learning model evaluation experience
• Minimum 3–8 years of relevant experience across machine learning engineering, evaluation systems, benchmark development, and end-to-end technical ownership
• Demonstrated experience deploying end-to-end ML evaluation or benchmark systems to production for foundation models
• Strong hands-on experience with ML evaluation frameworks and benchmark design, including approaches such as LLM-as-a-judge
• Prior ownership of backend and infrastructure systems, including:
• Data pipelines
• Execution environments
• Storage
• Orchestration
• Experience building data pipelines capable of supporting large-scale workloads
• Experience working with or deploying machine learning systems in production
• Customer-facing engineering experience, including direct interaction with enterprise customers
• Demonstrated ability to independently scope and execute ambiguous technical problems from feasibility through delivery
• Strong written communication skills for customer-facing and cross-functional technical work
• Strong bias toward action and ability to operate effectively in fast-moving, high-ambiguity environments
• Bachelor's degree or higher in Computer Science, Physics, or a related technical field
• Early-stage B2B startups with demonstrated traction
• Forward-deployed engineering organizations or companies with a strong FDE model
• High-ownership generalist roles within larger organizations, such as an Office of the CTO or internal machine learning evaluation team
• Fast-moving engineering environments where individuals have owned projects from initial concept through production deployment
• Working directly with enterprise customers
• Translating customer requirements into technical specifications
• Managing technical components of customer engagements
• Explaining complex ML evaluation concepts clearly to technical and non-technical stakeholders
• Balancing customer-specific requirements with the development of reusable infrastructure
• Working under tight customer deadlines while maintaining engineering quality
• Taking ownership of technical delivery from initial discovery through deployment
• Experience building benchmarks, evaluations, or human-data pipelines for large language models (LLMs) is strongly preferred
• Experience working directly with AI researchers or foundation model labs
• Experience developing evaluation systems for foundation models or LLM-based applications
• Published research, papers, or open-source contributions related to ML evaluations or benchmarks
• GitHub contributions involving ML evaluation, benchmark systems, LLM evaluation, or related infrastructure
• Experience with agentic AI evaluation environments involving tools, code execution, or multi-step workflows
• Experience developing human-data pipelines for machine learning evaluation
• Experience working across multiple AI modalities or evaluation domains

Benefits

• Remote work setup

Company Overview