Senior AI Engineer, Data Gateway

Senior AI Engineer, Data Gateway Location: Remote Department: Product Development, Data Gateway Reports To: Director, Data Gateway Engineering

About the Role

We are adding AI engineering to the Data Gateway team, the product group that defines how Clearsense acquires, curates, and delivers healthcare data. Like our Data Gateway Engineers, you will build the frameworks and tools other engineers use rather than run day-to-day ingestion projects. Your medium is different: semantic knowledge bases and internal AI agents. You will turn what the Data Gateway team knows (source-system profiles, extraction and curation patterns, data dictionaries, runbooks, and curated document collections) into retrieval-ready knowledge bases. Then you will build agent tools on top of them that help engineers discover, map, validate, and document data. Almost all of it runs on AWS, mainly Amazon Bedrock and Amazon Bedrock AgentCore, under our HIPAA and HITRUST controls. This is a hands-on, senior individual contributor role. It blends applied AI engineering with a product mindset: build for reuse, measure quality, and hand off tools other teams can trust. Success Metrics: IN 1 MONTH Learn the Data Gateway workflow end to end (acquisition, curation, delivery) and where engineers lose time to lookup, mapping, and documentation. Inventory the team's knowledge sources and propose a knowledge base design with an evaluation plan. Ship a working prototype: one knowledge base and one agent tool used by a Data Gateway Engineer on live work. IN 3 MONTHS Put at least one knowledge base into production with automated ingestion, access controls, and a retrieval-quality baseline scored on every change. Deliver at least one agent tool that measurably cuts time on a recurring Data Gateway task, such as source profiling, field mapping, or validation triage. Stand up evaluation, observability, and cost reporting for every AI tool the team runs. IN 6 MONTHS Maintain a golden path (templates, infrastructure as code, docs) that lets the team stand up a new knowledge base or agent tool in days, not weeks. Drive adoption of AI tooling across Data Gateway and at least one Operations Engineering team. Keep a living library of retrieval patterns, tool definitions, prompts, and evaluation sets, and pilot at least one emerging AWS capability with an adopt-or-hold recommendation. Key Responsibilities: Semantic Knowledge Bases Design and build the knowledge bases the Data Gateway team relies on: source-system profiles, data dictionaries, extraction and curation patterns, runbooks, and curated document collections. Own the retrieval pipeline end to end: parsing (including OCR'd multi-page PDFs), chunking, metadata, embeddings, hybrid and structured retrieval, and reranking. Pick the right store for each corpus, such as Amazon Bedrock Knowledge Bases (managed or custom), Amazon S3 Vectors, Amazon OpenSearch Serverless, Aurora PostgreSQL with pgvector, or graph retrieval on Amazon Neptune Analytics, and document the trade-offs. Keep knowledge current with automated sync, versioning, lineage back to source, and retirement of stale content. Internal AI Agent Tools Build agents and tools that help engineers profile sources, map fields, generate and review transformation code, triage validation failures, and draft documentation. Build on Amazon Bedrock AgentCore (Runtime, Gateway, Memory, Identity, Policy, Observability, Evaluations), and expose internal systems to agents as MCP tools through AgentCore Gateway. Keep a human in the loop where it matters: approval steps, audit trails, and clear limits on what an agent can change. Quality, Security & Operations Define evaluation for every knowledge base and agent: golden datasets, retrieval and answer-quality metrics, and regression checks in CI/CD. Apply security by default for PHI: least-privilege IAM, encryption, network isolation, Amazon Bedrock Guardrails, and logging that meets HIPAA and Qualifications: Required -5+ years in software, data, or ML engineering, including 2+ years building production LLM applications (RAG, agents, or both). -Hands-on experience with Amazon Bedrock: model invocation, Knowledge Bases, Agents or AgentCore, and Guardrails. -Built and tuned retrieval systems end to end: chunking, embeddings, vector and hybrid search, reranking, and evaluation. -Strong Python and solid SQL. -AWS depth beyond AI services: IAM, S3, Lambda, Step Functions, ECS/Fargate, VPC networking, and CloudWatch, plus infrastructure as code (CDK or Terraform) and CI/CD. -Evaluation discipline: you measure retrieval and answer quality and ship against the numbers. -Experience handling sensitive or regulated data with the right controls. -Clear communicator who documents well and can teach other engineers. Preferred -Healthcare data experience: HL7, FHIR, or CCD, and EHRs such as Epic, Oracle Health (Cerner), or MEDITECH. -Elasticsearch experience, especially semantic or hybrid search over large document indices. -Document AI: OCR pipelines, Amazon Textract, and layout-aware parsing of scanned clinical PDFs. -Knowledge graphs or GraphRAG (Amazon Neptune or similar). -Building MCP servers and tools, and agent frameworks such as Strands Agents or LangGraph. -Work in HIPAA or HITRUST environments. -Data engineering background (Airflow, Spark, NiFi) to work alongside Data Gateway Engineers on pipelines. Why Join Us? You will shape how AI changes the way healthcare data moves into the Clearsense platform. The knowledge bases and agents you build become the Data Gateway team's shared memory and its force multiplier, so every engineer on Data Gateway and Operations Engineering delivers faster and more consistently. This is a builder's role with real data, real internal users, and room to set the standard for applied AI at Clearsense.