[Remote] Data Analyst & Validation Engineer -Redshift, Lake & QA

Note: The job is a remote job and is open to candidates in USA. HCLTech is a technology services company seeking a Data Analyst & Validation Engineer to support data quality and ETL validation initiatives. The role focuses on validating transformation logic, designing and governing QA processes, building automated testing frameworks, monitoring data quality, and collaborating with engineering, product, business, and UI teams.

Responsibilities

• Validate transformation logic against business rules and documented specifications
• Perform source-to-target data reconciliation — verifying completeness, accuracy, and consistency
• Identify data anomalies, silent failures, and drift in pipeline outputs
• Build and maintain automated data validation suites that execute as part of pipeline runs
• Conduct periodic data audits beyond automated checks
• Define acceptance criteria for each ETL pipeline and transformation step — establishing what "correct" means in measurable, testable terms
• Define Definition of Done (DoD) for all data deliverables — specifying when a pipeline output is considered production-ready
• Create and maintain data quality test plans covering functional correctness, edge cases, regression, and performance
• Design test cases for new transformations before they reach production (shift-left testing)
• Establish data quality SLAs — freshness, completeness, and accuracy thresholds
• Define entry and exit criteria for pipeline releases — what must pass before promoting changes
• Maintain a defect taxonomy — categorizing data issues (schema drift, logic errors, source issues, timing issues) for root cause tracking and trend analysis
• Define sign-off workflows — who approves what before data reaches the UI layer
• Design the overall data testing strategy:
• Unit tests for individual transformation logic
• Integration tests for end-to-end pipeline correctness
• Regression tests for ongoing stability
• Smoke tests for post-deployment verification
• Negative tests for resilience (nulls, duplicates, out-of-range values, schema violations)
• Build reusable, parameterized validation patterns applicable across multiple pipelines
• Implement data contract validation — ensuring upstream sources meet agreed schemas and value constraints
• Integrate quality gates into CI/CD pipelines for automated release blocking
• Document expected behavior for each transformation (input, logic, expected output)
• Maintain data quality runbooks — investigation and resolution procedures for common issues
• Create traceability matrices linking business requirements to test cases to pipeline outputs
• Document known data limitations, assumptions, and technical debt
• Maintain a living catalog of data quality rules and their rationale
• Build data quality dashboards tracking pass/fail rates, trend analysis, and SLA compliance
• Configure alerting for data quality threshold breaches
• Track and report data quality metrics (DQ score, defect density, mean time to detect, mean time to resolve)
• Conduct data quality retrospectives — identifying gaps and improving coverage
• Participate in pipeline design reviews — flagging quality risks early in development
• Work with product and business stakeholders to translate vague requirements into testable assertions
• Collaborate with data engineers on pipeline improvements driven by quality findings
• Support UI/frontend teams in verifying rendered data correctness

Skills

• Ful-time role only - Preferred only US Citizens or GC Holders
• Only US Citizens or GC holders required
• Must have Skills - 6-10 years Data Validation& Validation & AWS Data services (Redshift, Lake Formation, Athena, and S3-based data lakes), Python, ETL ELT concepts, QA methodology (Test planning & design, Defect lifecycled
• Advanced — window functions, CTEs, set comparisons, complex joins, data profiling queries
• Hands-on experience querying and validating data in Amazon Redshift, AWS Lake Formation, Athena, and S3-based data lakes
• Validation scripts, data comparison tools, automation frameworks
• Deep understanding of extraction, transformation, and loading patterns, including common failure modes
• Test planning, test case design, acceptance criteria definition, defect lifecycle management
• Statistical profiling, distribution analysis, completeness and uniqueness checks
• Hands-on experience with at least one: Great Expectations, dbt tests, Soda Core, or equivalent custom frameworks
• Git — managing test suites alongside pipeline code
• 6-10 years of combined experience in data engineering, data QA, or analytics engineering
• Has owned data quality for at least one production system end-to-end (not just contributed)
• Has defined acceptance criteria and quality gates that blocked defective releases
• Has built automated validation suites that caught real production issues
• Comfortable reading and reasoning about pipeline code (transformation logic, orchestration DAGs)
• Experience working with curated/aggregated datasets that serve application UIs
• Familiarity with AWS Glue, Redshift Spectrum, and AWS data pipeline services
• Experience with BDD-style data testing (Given/When/Then for data transformations)
• CI/CD integration for data quality — automated gates in deployment pipelines
• Experience defining and tracking data SLAs/SLOs
• Knowledge of regulatory or compliance data requirements
• Performance testing for pipelines — verifying latency and throughput
• Exposure to chaos engineering for data — intentionally injecting bad data to test resilience
• Experience with pipeline orchestration tools (Glue Orchestrator, Step Functions, Airflow)
• Experience with IAM permissions and Lake Formation access controls for data governance

Benefits

• Performance-based bonuses, subject to company policies
• Medical insurance
• Dental insurance
• Vision insurance
• Pharmacy insurance
• Life insurance
• Accidental death & dismemberment insurance
• Disability insurance
• Employee assistance program
• 401(k) retirement plan
• 10 days of paid time off per year
• Need-based leave with no designated number of leave days per year for some positions
• 10 paid holidays per year
• Continuous opportunities to learn and grow
• Transparent communication with senior-level employees
• Learning and career development programs at every level
• Opportunities to experiment in different roles or pivot industries
• Unlimited opportunities to find the role that fits you best

Company Overview