Remote Position
[Remote] Senior Systems Software Engineer - Accelerated Spark Applications
🌍 Remote · United States
💼 Full-time
Note: The job is a remote job and is open to candidates in USA. NVIDIA is hiring a Senior Systems Software Engineer for its Apache Spark Acceleration group. The role builds and operates a hosted Spark service on Kubernetes, extends GPU-accelerated Spark runtime and workload-routing components, and supports production workloads through monitoring, incident response, and technical guidance.
Responsibilities
- Design, build and operate hosted Spark service control plane on Kubernetes: the Java REST API, the Spark Connect runner, Kyuubi, and deployment using Helm charts and Flux
- Own code review for one or more services end to end and keep them releasable on the release train
- Extend the cuDF Spark runtime and the CPU/GPU workload router in Scala and Java, including interoperability with Delta Lake, Apache Iceberg and Unity Catalog
- Investigate production failures on clusters running more than a thousand concurrent jobs together with the customer team and upstream library teams, and drive fixes to closure
- Instrument metrics, dashboards and alerts; take part in the on-call rotation and keep the runbooks current
- Provide recommendations and feedback to teams regarding decisions surrounding topics such as infrastructure, continuous integration and testing strategy
Skills
- BS, MS, or PhD in Computer Science, Computer Engineering, or a closely related field (or equivalent experience)
- 9+ years of experience in software development
- Strong distributed-systems fundamentals and proficiency in Java or Scala, plus C++ or Python
- Hands-on experience operating services on Kubernetes, on-premise or cloud
- Experience owning or reviewing a long-lived server: REST front ends, proxies or request routers, JDBC or Thrift services
- Experience supporting enterprise or internal production customers, including on-call
- Experience using AI tools in software development
- Apache Spark internals, Spark Connect, or Kyuubi
- Delta Lake, Apache Iceberg or Unity Catalog integration work
- GPU data processing with cuDF, or CUDA experience
- Significant contributions to data platform projects committed to collaboration and shared development (Apache Spark, Velox, Presto, Apache Arrow, Apache DataFusion)
Benefits
- Eligible for equity
Company Overview
Company H1B Sponsorship