Remote Position
[Remote] Lead Engineer - II_Data engineering on cloud
🌍 Remote · United States
💼 Full-time
Note: The job is a remote job and is open to candidates in USA. CitiusTech is hiring a GCP Data Engineer to design and build an automated solution for data discovery, end-to-end lineage, metadata management, and data quality across multiple GCP projects. The role develops metadata pipelines, tracks data movement, integrates lineage with Collibra, and supports metadata analysis and data quality workflows.
Responsibilities
- Build programmatic upstream and downstream data lineage across multiple GCP projects and BigQuery datasets
- Discover datasets, tables, views, stored procedures, jobs, and other relevant data assets
- Trace data movement and intermediate processing through BigQuery, Dataflow, Cloud Data Fusion, stored procedures, Cloud Storage, and related GCP services
- Use GCP Data Lineage/Dataplex APIs, BigQuery INFORMATION_SCHEMA, job metadata, and other GCP APIs to collect and enrich lineage information
- Capture available table-level and column-level lineage transformation details, and pipeline execution metadata
- Extract data quality rules, scans, and results using Dataplex and related metadata services
- Develop Python/SQL-based services and pipelines to collect, normalize, validate, and store metadata in BigQuery
- Generate standardized OpenLineage JSON/events and integrate lineage metadata with Collibra
- Implement secure cross-project access using GCP service accounts, IAM roles, and least-privilege principles
- Build monitoring, logging, error handling, and audit capabilities for metadata and lineage processing
- Support AI/agentic workflows for metadata analysis, lineage enrichment, and identification of lineage or data quality gaps
Skills
- 7+ Years
- Engineering Degree – BE/ME/BTech/MTech/BSc/MSc
- Strong hands-on experience with Python and SQL
- Strong experience with Google BigQuery, including metadata and INFORMATION\_SCHEMA
- Experience with Dataflow and/or Cloud Data Fusion
- Experience integrating with GCP APIs and REST APIs
- Understanding of data lineage, metadata management, data quality, and data governance
- Experience with GCP IAM, service accounts, and cross-project access
- Experience designing and supporting production-grade data pipelines
- Technical certification in multiple technologies is desirable
- Dataplex / GCP Data Lineage API
- Collibra and OpenLineage
- Cloud Run and Cloud Storage
- Terraform / Infrastructure as Code
- Vertex AI / agentic or LLM-based workflows
- SQL parsing and transformation dependency analysis
- Enterprise data governance and metadata platforms
Benefits
- Remote work
- A fun, transparent, non-hierarchical, diverse work culture focused on continuous learning and work-life balance
- Opportunities for faster growth, higher learning, and stronger impact
Company Overview
Company H1B Sponsorship