Data Engineer - WFH
About the Role
Our client is seeking a Data Engineer to design, develop, and maintain scalable data infrastructure that supports analytics, reporting, and operational decision-making. This role focuses on building reliable data pipelines, integrating data from multiple sources, and ensuring information is accurate, accessible, and ready for use across the business. You will work with modern data engineering tools and cloud platforms to manage data ingestion, transformation, warehousing, and real-time processing. Working closely with data analysts, data scientists, and business stakeholders, you will help deliver dependable data solutions while maintaining high standards for data quality, governance, performance, and cost efficiency. Key Responsibilities ● Build, maintain, and optimize ETL/ELT pipelines using Python, SQL, or Scala. ● Orchestrate data workflows using Airflow, Prefect, Dagster, or similar tools. ● Ingest structured and unstructured data from APIs, SaaS platforms, databases, files, and streaming systems. ● Develop scalable data connectors and automated ingestion workflows. ● Manage and optimize cloud data warehouses such as Snowflake, BigQuery, and Redshift. ● Design scalable data schemas using star and snowflake modeling techniques. ● Apply partitioning, clustering, indexing, and other strategies to improve query performance. ● Build clean, analytics-ready datasets for business intelligence and reporting. ● Implement data validation, anomaly detection, logging, and monitoring to maintain data integrity. ● Maintain naming conventions, data lineage, and documentation standards using tools such as dbt or Great Expectations. ● Support audit-ready data processes and compliance with applicable requirements, including GDPR, HIPAA, or relevant industry standards. ● Monitor pipeline health, investigate failures, and resolve data inconsistencies. ● Build and maintain real-time data pipelines using Kafka, Kinesis, Pub/Sub, or similar technologies. ● Support low-latency data ingestion and event-driven architectures for time-sensitive applications. ● Monitor streaming systems and improve throughput, reliability, and performance. ● Collaborate with analysts, data scientists, and business stakeholders to deliver reliable datasets for reporting and analysis. ● Support dashboard and reporting initiatives using Tableau, Looker, or Power BI. ● Translate business requirements into scalable data solutions and data models. ● Maintain technical documentation for pipelines, schemas, transformations, and workflows. ● Containerize data services using Docker and manage deployments through Kubernetes where applicable. ● Automate deployments using CI/CD tools such as GitHub Actions, Jenkins, or GitLab CI. ● Manage cloud infrastructure using Terraform, CloudFormation, or similar Infrastructure-as-Code tools. ● Continuously improve data system reliability, scalability, performance, and cloud cost efficiency. Required Qualifications ● At least 3 years of experience in Data Engineering, Back-End Engineering, or Data Infrastructure roles. ● Strong proficiency in Python and SQL. ● Experience with at least one modern data warehouse, such as Snowflake, Redshift, or BigQuery. ● Hands-on experience with workflow orchestration tools such as Airflow or Prefect. ● Strong understanding of ETL/ELT pipelines, data modeling, and data transformation workflows. ● Familiarity with cloud platforms such as AWS, Google Cloud Platform (GCP), or Microsoft Azure. ● Strong debugging and problem-solving skills, with attention to detail and a focus on data quality and reliability. ● Ability to work effectively with technical and non-technical stakeholders. ● Proactive communication skills and the ability to take ownership of data systems and pipeline performance. Preferred Qualifications ● Experience with dbt for data modeling and transformation management. ● Experience building streaming and event-driven data pipelines using Kafka, Kinesis, Pub/Sub, or similar platforms. ● Familiarity with cloud-native data services such as AWS Glue, GCP Dataflow, or Azure Data Factory. ● Experience with Docker, Kubernetes, Terraform, or CI/CD workflows. ● Background in regulated industries such as healthcare, financial technology, or enterprise SaaS. ● Experience optimizing warehouse costs and query performance at scale. ● Familiarity with data quality and validation tools such as Great Expectations. Work Arrangement ● Employment type: Full-time. ● Work arrangement: Remote. ● Working hours: U.S. client business hours, with flexibility required for pipeline monitoring, deployments, and data refresh cycles. Key Performance Indicators ● Maintain pipeline uptime of at least 99%. ● Keep data freshness within agreed service-level agreements (SLAs). ● Prevent critical data quality issues from reaching downstream reporting systems. ● Improve warehouse query performance and optimize cloud costs. ● Deliver reliable, scalable datasets on time. ● Maintain positive working relationships and feedback from analysts, data scientists, and business stakeholders. Interview Process ● Initial phone screening. ● Video interview with a Pavago recruiter. ● Technical assessment, such as building a small ETL pipeline or optimizing a SQL query. ● Client interview with the Engineering or Data team. ● Offer and background verification.