Remote Position
Senior Data Engineer
🌍 Remote · United States
💼 Full-time
🚀 Welcome to Your Next Adventure!We are looking for a technically-minded engineer to participate in managing and developing our ETL infrastructure. Our current system includes various tools and services such as workflow management systems, message brokers, a data lake, a data warehouse, and infrastructure-as-code tools.In addition to everyday activities, we have room to improve our data management processes. Mainly, this involves adapting and refactoring ETL systems to accommodate company growth; other areas include performance and data governance. This role is a great fit if you have a deep understanding of specific areas within data engineering or the confidence to manage the full scope of requirements listed below. Above all, we value good programmers even if they lack experience in other areas.
🛠 What You’ll Get Your Hands On:
- Integrating new sources using existing batch ETL system
- Creating new real-time pipelines using an infrastructure-as-code approach
- Supporting teams in extracting data from the warehouse into their systems, and working on Reverse ETL processes
- Taking part in data management process improvement, providing feedback about process changes, and proposing new solutions
- Automating processes and performing pipeline performance optimizations
- Taking responsibility for long-term improvement activities, refactoring, cost reduction investigations, and implementation
- Strong programming skills, e.g., experience with several programming languages or Python frameworks
- Software design skills - software design patterns, application design
- Data system design or cloud systems experience (AWS, GCP, message brokers, object storage, query engines)
- Strong SQL skills, including the ability to apply subqueries, window functions, and CTEs
- Warehouse design or ETL systems design experience
- PySpark or other distributed systems knowledge, e.g., distributed computing, performance optimization
- Experience with or ability to use infrastructure as code, e.g., AWS CloudFormation
- Experience with or ability to use the Databricks platform
- GCP platform
- BigQuery
- AWS message brokers
- Databricks
- Spark
- Dbt
- Tableau, Redash
Strong programming skills, SQL, data system design, cloud experience (AWS/GCP), ETL/warehouse design, PySpark or distributed systems, infrastructure-as-code, Databricks platform experience.
Key Responsibilities
- integrating data sources
- building pipelines
- optimizing performance
Skills & Tools
GCP, BigQuery, AWS, Databricks, Spark, Dbt, Tableau, Redash, PySpark, CloudFormation
Job Details
- Category: Data and Analytics
- Seniority: Senior Level
- Commitment: Full Time
- Workplace: Remote — Qatar
- Languages: English
Benefits
- Generous paid time off
- Tuition reimbursement
About Snoonu
Qatar's homegrown Super App reinventing daily life with delivery, shopping, and more. — Industry: Information Technology