[Remote] Lead Databricks Engineer

Note: The job is a remote job and is open to candidates in USA. Artmac is hiring a Lead Databricks Engineer to lead the design, development, and implementation of scalable data engineering solutions on the Databricks Lakehouse Platform. The role owns technical roadmaps and architecture decisions, establishes engineering frameworks and data architecture patterns, and drives platform automation, security, governance, and delivery across data initiatives.

Responsibilities

• Own the technical roadmap for Databricks-based data engineering initiatives, aligning platform architecture with enterprise data strategy and business goals
• Lead architecture reviews and make high-impact technical decisions involving data platform design, technology selection, scalability, and maintainability
• Define reusable engineering frameworks, reference architectures, coding standards, and development guidelines across multiple data engineering projects
• Drive the implementation of domain-oriented data architecture and data product strategies to improve data accessibility, ownership, and reuse
• Establish architectural patterns for multi-environment data platforms, including development, testing, staging, and production
• Lead the adoption of Infrastructure as Code and automated platform provisioning to standardize Databricks environment management
• Partner with security, governance, and compliance teams to establish enterprise-wide policies for sensitive data, auditability, and regulatory requirements
• Manage technical dependencies, delivery risks, and architectural trade-offs across multiple teams and concurrent data initiatives

Skills

• 8 15 years of experience in designing and delivering enterprise data platforms using Databricks and cloud-native technologies
• Strong proficiency in Apache Spark, PySpark, Python, SQL, and distributed data processing
• Expertise in lakehouse architecture, Delta Lake, data architecture patterns, and large-scale data platform design
• Proven experience establishing data governance, metadata management, access controls, and data lineage practices
• Strong understanding of cloud architecture, networking, identity management, and security across Azure, AWS, or GCP
• Experience with platform automation, Infrastructure as Code, CI/CD, and production deployment strategies
• Ability to evaluate complex technical requirements and translate them into scalable architectural solutions
• Knowledge of Databricks Asset Bundles and automated deployment practices
• Understanding of data governance, security, data quality, and monitoring
• Bachelor's degree or equivalent combination of education and experience

Benefits

• Remote work

Company Overview

Company H1B Sponsorship