[Remote] Mng. Consultant - Databricks Data Engineering Lead

Note: The job is a remote job and is open to candidates in USA. The Judge Group is recruiting for mid-, senior-, and lead-level Databricks openings supporting a federal human-capital data modernization program. The role builds and maintains Databricks data pipelines, migration workflows, quality controls, APIs, curated datasets, and reusable data products.

Responsibilities

• Build and maintain Databricks ingestion, transformation, orchestration, migration, and data-processing pipelines
• Develop scalable ETL/ELT solutions using Databricks, Apache Spark, PySpark, Python, SQL, and Delta Lake
• Implement source-to-target mappings, transformation rules, business logic, data contracts, metadata, lineage, and data-quality requirements
• Develop migrated data feeds, APIs, curated datasets, reporting layers, and reusable data products
• Create reconciliation routines and support source-to-target validation, exception analysis, and defect resolution
• Build unit, integration, regression, performance, and data-quality tests and document technical results
• Support Git-based development, CI/CD, release management, deployment, monitoring, troubleshooting, and stabilization
• Analyze and resolve job failures, cluster-performance issues, pipeline bottlenecks, schema changes, and data-quality problems
• Collaborate with data architects, engineering leads, integration engineers, profiling analysts, testers, security specialists, governance leads, and source-system teams
• Document reusable code, technical patterns, runbooks, troubleshooting procedures, and lessons learned for later modernization waves
• Participate in Agile ceremonies, backlog refinement, sprint planning, demonstrations, and technical reviews

Skills

• MUST BE PROCESSED FOR E PUBLIC TRUST CLEARANCE
• Bachelor's degree in computer science, data engineering, information systems, engineering, analytics, or a related field
• Minimum 4 years of experience in data engineering, software development, ETL/ELT, cloud analytics, or a related technical discipline
• Hands-on experience with Databricks, Apache Spark, PySpark, Python, SQL, Delta Lake, and workflow orchestration
• Experience building, testing, and maintaining batch data pipelines, transformations, and curated data layers
• Experience with enterprise data migration, source-to-target mapping, reconciliation, validation, and data-quality controls
• Familiarity with Git, CI/CD, automated testing, deployment, monitoring, logging, and troubleshooting
• Strong analytical, problem-solving, documentation, collaboration, and communication skills
• Ability to obtain and maintain a Federal Public Trust

Benefits

• Healthcare
• 401k
• Medical, dental, and vision insurance are available to qualified candidates who meet eligibility requirements.

Company Overview

Company H1B Sponsorship