Sr. to Lead Data Engineer - Consumer Health Platform

Sr. to Lead Data Engineer - Consumer Health Platform Location: Redwood City, CA (you can work from home 3x/week). The company is based in Redwood City, CA. You could work from home up to 3x/week. The company seeks candidates who enthusiastic to work in the office at least 2 days/week. The company's product is in the realms of healthcare, consumer web, consumer mobile, and enterprise software. The company is reducing the cost and improving the quality of healthcare. It is also enabling people to have access to their healthcare in a convenient and safer way. The company has about 150 employees and 50 engineers, and is rapidly growing. In this position you would be owning and scaling the company's data platform, designing and building batch and streaming data pipelines using Python, PySpark, SQL, and AWS technologies. You would be developing data warehouses, optimizing database and query performance, and ensuring the reliability, security, and compliance of data infrastructure. You would collaborate with engineering, product, and analytics teams to improve data accessibility, build AI-powered tools, and mentor other engineers. The company will pay salary up to $250k, yearly bonuses, and equity which could be very valuable. Job Responsibilities: Owning the data platform end-to-end — ingestion, transformation, storage, query, and access — and driving its roadmap as the company’s data needs grow. Building and evolving batch and streaming pipelines on PySpark/EMR, Kinesis, Lambda, and Step Functions, ingesting from Postgres, Salesforce, third-party vendors, and product event streams into the Iceberg-based lake. Modeling the warehouse: designing SCD tables, event tables, and the conventions other engineers and analysts follow when adding new data. Partnering with product, engineering, analytics, and operations stakeholders across the company to turn data requests into well-scoped, reliable pipelines. Writing the docs and tooling that let them self-serve next time. Owning the security and compliance backbone of data systems, including audit logging, access control, and temporary-access workflows. Optimizing backend query performance where the data layer meets product code — read-replica routing, indexing, caching, and IO instrumentation in Java/Spring services. Leading investigation and remediation when data infrastructure misbehaves — IOPS spikes, pipeline failures, schema drift, late data — and making the fixes durable. Using AI as a daily accelerant for pipeline scaffolding, schema work, and ad-hoc investigations — and shipping internal AI tooling that lets other teams do the same. Mentoring engineers and analysts across the company on how to work with the data platform. Qualifications: Experience as a Data Engineer in a startup. At least 5 years (preferably more) of experience building production data pipelines and platforms. Expertise in Python (PySpark) and SQL fluency, including tuning Spark jobs at scale. Skill and willingness to work on the services around the data layer such as Java and Spring Boot. Experience with distributed compute (Spark/EMR), streaming (Kinesis), and object storage (S3). Postgres fundamentals — query optimization, indexing, replication, replica routing, and a feel for when the database is the bottleneck. Skilled with Iceberg and Trino or similar. Comfort with CI/CD and Terraform. Skilled in building with AI. Understanding of frontier models, agentic coding tools, and the ability to hack things together using AI. A track record of working with cross-functional teams such as product, operations, etc. Ability to take ownership of the platform’s reliability, cost, accessibility, and compliance. Ability to think about security and PII/PHI handling as core engineering work rather than as someone else's problem. Ability to be a force multiplier. The docs, tools, and skills you leave behind make other engineers and analysts faster long after the original ticket closes. A bias for shipping iteratively and instrumenting what you ship — you’d rather demo a rough v1 in two days than a polished v3 in two weeks. Taking pride in your work and having good judgment about what to prioritize when everything feels urgent. About Skyrocket Ventures Skyrocket Ventures is a recruiting firm for hundreds of high growth technology companies that range from industry leaders to top-tier startups. This opportunity is with one of our client companies for a full-time permanent hire. Please only apply if you are authorized to work in the U.S. Please note that even if this job is not a perfect match, we encourage you to apply as long as it is in the ballpark. Companies are often flexible in hiring candidates who do not perfectly fit their written job description, as long as the most important qualifications are there and the candidate is good in general. Most of the jobs we are recruiting for are not posted online, so if you would like to know of all the opportunities we have that match your interests and qualifications, then please get in touch with us. After you apply to this job posting, we’ll consider you for this job as well as any other potential matches with our client companies. If we have any potential matches, we’ll share your resume with those companies and contact you about any interview opportunities we can get you. Thank you, and we wish you a great job search!