[Remote] Staff AI Platform Engineer

Note: The job is a remote job and is open to candidates in USA. Code Metal is hiring a Staff AI Platform Engineer to build and operate the internal AI platform that supports its engineering teams’ AI-driven products. The role leads platform architecture and delivers production capabilities for model inference, gateways, agent orchestration, context engineering, observability, and experimentation.

Responsibilities

• Set the technical direction and architecture for Code Metal's AI platform and the team building it. Own the design docs and RFCs, help with build-vs-buy decisions, and align with other technical stakeholders
• Deploy, benchmark, and tune elastic, production inference for open-weight models on runtimes such as vLLM, SGLang, and TensorRT-LLM
• Own the model gateway that teams use to reach self-hosted and commercial models, with consistent auth, routing, failover, quotas, and cost attribution
• Design reusable agent harnesses and orchestration primitives that product teams can compose into reliable, verifiable workflows
• Build context-engineering services for memory, retrieval, and data discovery, so agents get the information specific to their tasks
• Instrument the stack end to end, for example, with OpenTelemetry traces and service metrics. Build the experiment-tracking and artifact layer that lets engineers and researchers reproduce and compare results
• Design for productization from day one (versioned APIs, security, deployment in customer environments), and communicate with business and product teams about the roadmap

Skills

• Production-grade Python and strong platform engineering fundamentals: API and service design, distributed systems, containers and Kubernetes, CI/CD, and testing
• Shipped production agentic systems, with a clear sense of where they break and how to make them reliable
• Experience with context engineering: retrieval-augmented generation, embeddings, vector / hybrid search, and memory for agents, ideally over code or large technical corpora
• Experience instrumenting services (for example, with OpenTelemetry tracing and metrics) and operating AI services against SLOs
• Solid data science and AI research fundamentals: how transformers and LLM inference work, experiment design, benchmarking, and model evaluation
• Working familiarity with PyTorch and Hugging Face, and experience fine-tuning, evaluating, or serving language models
• Staff-level technical leadership: owned architecture across multiple systems or teams, written design docs and RFCs, turned ambiguous needs from several internal customers into a roadmap, and mentored engineers
• Experience building an evaluation harnesses for LLMs and agents, and experiment-tracking or artifact management systems such as MLflow or Weights & Biases
• Typically 8+ years of software engineering experience, including 4+ years building and operating ML/AI or LLM systems in production, with demonstrated Staff-level scope and impact
• Highly desirable: Production experience running an LLM gateway or proxy such as SMG or Bifrost, or equivalent experience building an API gateway, including routing, auth, rate limiting, quotas, failover, and cost attribution
• Hands-on experience deploying and tuning LLM inference engines such as vLLM, SGLang, or TensorRT-LLM on GPU infrastructure, with measurable gains in throughput, latency, or cost per token
• Familiarity with inference optimization: speculative decoding, prefix caching, tensor/pipeline/expert parallelism, disaggregated prefill and decode, GPU profiling
• Experience taking an internal platform to an external product: multi-tenancy, SDKs, versioned APIs, and documentation
• Experience deploying AI systems on-prem or in regulated domains such as defense or aerospace

Benefits

• Offers Equity
• Health care plan with 100% premium coverage, including medical, dental, and vision
• 401k with 5% matching
• Paid Time Off (uncapped vacation, plus sick and public holidays)
• Flexible hybrid or remote work arrangement
• Relocation assistance for qualifying employees

Company Overview