Senior Data Engineer (Quantitative)

Senior Data Engineer (Quantitative) AmbientSkye · Remote, based in Connecticut · Senior Base salary: $145,000 – $175,000, plus equity Reports to: Director of Software Development ROLE SUMMARY The Senior Data Engineer (Quantitative) owns the data foundations and the quantitative validation behind AmbientSkye's findings. Every figure the platform reports has to trace to its source, so the role builds the ingestion pipelines, the knowledge graph that links entities, evidence and assumptions, and the evaluation suite that measures whether the platform's AI output is accurate and well calibrated. The work carries across every application the platform serves — life-sciences and regulatory data, financial filings, and the industrial and economic data behind government scenario analysis — so the requirements below are the same whichever application the role is working on. KEY RESPONSIBILITIES Data ingestion and datasets Build source adapters and ingestion pipelines for public and customer-supplied data that are idempotent, efficient at bulk volume, and keep provenance on every record. Assemble documented, cited datasets from heterogeneous sources: harmonise units, periods and categories; detect and flag conflicts rather than averaging them away; write the data dictionary. Own the platform's existing pipelines for life-sciences, regulatory and financial filings data. Knowledge graph Extend the knowledge graph that links entities, evidence and assumptions, and keep it consistent as inputs change. Quantitative validation and evaluation Calibrate and validate the quantitative scoring engine, which ranks candidate outcomes and measures how uncertain that ranking is: coverage testing, assumption sensitivity, and an honest write-up of limitations and failure cases, including when the engine is applied to a new problem domain. Turn the platform's accuracy scorers and replay fixtures into a recorded evaluation suite that runs on a schedule. COLLABORATION & TEAMWORK AmbientSkye is a small, tightly collaborative team; this role succeeds through partnership, not silos. Works most closely with the Director of Software Development on day-to-day priorities, data architecture, code review and delivery commitments. Partners with the COO, who designed the scoring and calibration method, on validation work. Works with the Senior Backend and Agent Engineer so the agents and scenario engines read data the way it was ingested, with provenance intact. Writes dataset documentation and validation findings as deliverables in their own right, clear enough for non-engineering partners and customers. QUALIFICATIONS & EXPERIENCE Required 6+ years of data engineering in Python, with strong PostgreSQL: bulk loading (COPY), upserts, query tuning. Working statistics: calibration, confidence or prediction sets, out-of-sample testing, small-sample pitfalls and leakage. Can tell when a result is too good to be true. A record of building datasets from messy public sources with traceable provenance. Hand-written SQL and comfort with row-level-security tenant isolation (no ORM). Clear technical writing. US person (for example, a US citizen or permanent resident). Preferred Conformal prediction, Bayesian methods or forecasting evaluation. Experience evaluating LLM output: labelled sets, precision and recall of extracted facts, LLM-as-judge. Familiarity with economic, industrial, healthcare (NPPES, Open Payments, FDA) or SEC EDGAR data. Knowledge-graph or entity-resolution experience; pgvector. SECURITY & COMPLIANCE RESPONSIBILITIES (SOC 2) Keeps customer-supplied data inside its tenant: row-level security on every table and query the role writes. Maintains provenance and access controls on every ingested record, so any figure can be traced and any access explained. Honours the licence and usage terms of third-party data sources. Follows change management for schema changes: versioned migrations, reviewed pull requests and CI gates. Emits audit events for state-changing data operations, and supports incident response for the pipelines the role owns. COMPENSATION & BENEFITS Base salary $145,000 – $175,000, plus equity. Health Insurance, Paid Leave, Equity