Big Data Engineer - Streaming
This a Full Remote job, the offer is available from: Indiana (USA) This role is for one of the Weekday's clients Salary range: Rs 1900000 - Rs 2800000 (ie INR 19-28 LPA) Min Experience: 4 years Location: Remote (India) JobType: full-time We are looking for a highly skilled Big Data Engineer – Streaming to design, build, and maintain large-scale real-time data processing systems. This role is focused exclusively on streaming data pipelines and requires deep expertise in Java-based backend development along with hands-on experience in Apache Spark Streaming or Hadoop streaming ecosystems. The ideal candidate thrives in high-performance environments and has a strong passion for building scalable, low-latency data platforms.
Requirements
Key Responsibilities
- Design, develop, and optimize real-time streaming data pipelines for large-scale, distributed systems.
- Build and maintain robust data ingestion and real-time processing frameworks using Apache Spark Streaming and Hadoop-based streaming technologies.
- Develop high-performance, production-grade solutions using Java as the core programming language.
- Ensure streaming systems meet requirements for scalability, fault tolerance, reliability, and low latency.
- Integrate streaming pipelines with downstream applications, platforms, and analytics systems.
- Monitor, troubleshoot, and optimize performance bottlenecks across streaming data workflows.
- Collaborate closely with platform, infrastructure, and application teams to deliver end-to-end streaming solutions.
- Support CI/CD practices for continuous improvement and reliable deployment of data pipelines. Technical Skills
- Strong expertise in real-time data processing and streaming architectures.
- Hands-on experience with Apache Spark Streaming and/or Hadoop streaming ecosystems (no batch processing).
- Advanced proficiency in Java for large-scale, distributed systems development.
- Experience with CI/CD pipelines and production-grade deployment practices.
- Familiarity with cloud platforms such as AWS, Microsoft Azure, or Google Cloud Platform.
- Solid understanding of system performance tuning, fault tolerance, and distributed computing concepts. Skills Big Data Engineering Real-Time Streaming Apache Spark Streaming Hadoop Streaming Java Cloud Platforms (AWS / Azure / GCP) CI/CD This offer from "Weekday (YC W21)" has been enriched by Jobgether.com and got a 72% flex score. Apply tot his job