[Remote] Senior Backend Engineer, Inference Platform

Note: The job is a remote job and is open to candidates in USA. You.com is hiring a Senior Backend Engineer to own and scale its inference gateway and build the control plane for managing model access, performance, and cost. The role focuses on model routing and fallback, cost attribution and metering, and reliable request-path enforcement.

Responsibilities

• Own the inference gateway that every model call routes through, including provider and model selection, fallback, and caps
• Build intelligence layer to autonomously pick the right model for the right turn in the task
• Make routing decisions that account for prefix-cache behavior, batching, and time to first token, not only price and availability
• Build cost attribution and metering across providers, including cached-token accounting, accurate enough to bill and enforce on
• Build request-path enforcement that degrades gracefully and never takes down a customer's inference call
• Ship inference-time changes in shadow mode first, with the measurement to prove they work

Skills

• Hands-on production experience with model serving on stacks such as vLLM, SGLang (or similar platforms), with a working understanding of continuous batching, KV and prefix caching, and the tradeoff between time to first token and throughput
• You have tuned a serving stack against live traffic and know what moves p99 in practice, not only in theory
• Fluency across provider APIs: tokenization, prompt-caching semantics, and the differences in how usage and cached tokens are reported and priced
• Experience building or operating a proxy, gateway, or data plane on the critical path of production traffic, with sound instincts for failing open
• Comfort with streaming responses and the parts that get awkward: partial state, mid-stream failures, and work that can only happen once the stream closes
• A strong backend generalist (Python, Kubernetes). We expect you to pick up whatever you have not touched yet
• We weigh trajectory and recent growth more heavily than years of experience
• Experience at an inference provider or GPU cloud, or contributions to an open-source serving stack
• Fine-tuning and evaluating models in production
• Usage metering, billing, or cost optimization at scale

Benefits

• Hubs in San Francisco and New York City offering regular in-person gatherings and co-working sessions
• Flexible PTO with U.S. holidays observed and an end of year shutdown in December to rest and recharge (eligibility varies by employment type and tenure)
• A competitive health insurance plan covers 100% of the policyholder and 75% for dependents (eligibility varies by employment type and tenure)
• Function Health membership
• 12 weeks of paid parental leave in the US (eligibility varies by employment type and tenure)
• 401k program, 3% match - vested immediately (eligibility varies by employment type and tenure)
• $500 work-from-home stipend to be used up to a year of your start date (eligibility varies by employment type and tenure)
• $600 technology stipend to support a portion of your hybrid/remote team's cell phone and internet expenses (eligibility varies by employment type and tenure)
• $1,200 per year Health & Wellness Allowance to support your personal goals (eligibility varies by employment type and tenure)
• The chance to collaborate with a team at the forefront of AI research

Company Overview

Company H1B Sponsorship