Full-Stack AI Engineer

Experience: 6+ years in software engineering, including 2+ year building LLM or agent solutions The role - You build, evaluate and deploy agentic solutions from an approved spec, using coding agents for most code, tests and documentation. - You direct the agents, check their output and own the result. - You deliver working software that is instrumented, evaluated, and ready for the business to accept. What you will do - Size stories with the product owner, flag effort and risks, and confirm data and access early. - Test the spec for buildability, prepare the repo and agent context, and record open questions. - Slice stories into tasks, estimate in story points and commit the sprint with the POD. - Build with coding agents: code, unit and integration tests, and documentation. - Implement agents on Copilot Studio, Azure AI Foundry Agents or Microsoft Agent Framework, including prompts, tools, connectors, workflows and integrations. - Build the data and retrieval layer: Document extraction pipelines with Azure AI Document Intelligence, including validation and confidence handling. - Chunking, embedding, indexing and hybrid search in a vector store such as Azure AI Search. - Calls to decision and rules APIs, with the LLM handling interpretation and drafting only. - Apply context engineering: write and version prompts and instructions, manage context windows, memory and tool results, and keep token use within budget. - Handle LLM calls reliably: structured outputs, retries, fallbacks, caching and rate limits. - Build evaluation sets and run evals on every release: automated metrics, LLM-as-judge and human review samples. - Fix findings and resubmit. - Instrument every agent with OpenTelemetry. - Build traces, dashboards and alerts for latency, token use, cost, errors and quality. - Run A/B tests as the architect designs them: variant routing, metric capture and result readouts. - Deploy through CI/CD, compile release evidence for go/no-go, and support the business through validation and handover. Must have - Strong full-stack engineering in Python and/or C#/.NET, plus API and integration development. - Hands-on delivery of LLM or agent applications: prompt design, function and tool calling, RAG and workflow orchestration. - Practical experience with at least one of Azure AI Foundry, Copilot Studio or Microsoft Agent Framework. - Experience with a document intelligence service (Azure AI Document Intelligence or similar) on real, messy documents. - Experience with a vector database or search index: embeddings, chunking, hybrid search and relevance tuning. - Working knowledge of context engineering and token handling. - Evaluation practice: building golden datasets, running evals in CI, and interpreting results for non-deterministic output. - OpenTelemetry instrumentation and use of Azure Monitor, Application Insights or an equivalent observability tool to diagnose agent failures. - Familiarity with A/B testing, feature flags and traffic splitting. - Daily use of AI coding agents (GitHub Copilot, Claude Code, Cursor or similar). - You write clear specs and context files, review agent code critically and catch its mistakes. - CI/CD, Git workflows, and Azure fundamentals (identity, secrets, networking, monitoring). - Clear communication with business users and daily Agile collaboration. - Microsoft certifications (Azure AI Engineer Associate / Azure AI Apps and Agents Developer Associate) Nice to have - Natural-language-to-data solutions and data pipelines (Azure Data Factory, Databricks or Fabric, SQL). - Rules engines or workflow automation (Power Automate, Logic Apps). - Healthcare or payer exposure, including PHI handling.