Requirement: Spark Developer

Experience : 5+ years

Location: Pune

Primary Skill : Big Data, Code Optimization, PySpark, Big Data Processing, Spark, Hive

About Company

Bridgenext is a global digital consultancy that helps clients innovate with intention and realize their digital aspirations by creating digital products, experiences, and solutions around what real people need. Our global consulting and delivery teams facilitate highly strategic digital initiatives through digital product engineering, automation, data engineering, and infrastructure modernization services, while elevating brands through digital experience, creative content, and customer data analytics services.

Don't just work, thrive. At Bridgenext, you have an opportunity to make a real difference - driving tangible business value for clients, while simultaneously propelling your own career growth. Our flexible and inclusive work culture provides you with the autonomy, resources, and opportunities to succeed.

We are looking for an Engineer with hands-on Data Engineering experience who will work on the internal and customer-based projects for Bridgenext, someone who cares about the quality of the code and who is passionate about providing the best solution to meet the client's needs and anticipates their future needs based on an understanding of the market.

Position Description

Must Have Skills:

• Bachelor’s degree in computer science, Engineering, or a related field.
• 5+ years of experience in data engineering, with a strong focus on large-scale data processing using Apache Spark.
• Experience in handling large scale data(Terra bytes) of data using Spark
• Strong expertise in optimizing Apache Spark jobs on both on-premises (HDFS) and cloud environments (AWS S3).
• Experience with Spark resource tuning and performance optimization, including partitioning, caching, and memory management.
• Proven experience in migrating data pipelines from on-premises systems (HDFS) to cloud platforms
• Experience with Spark SQL query optimization, including working with Catalyst Optimizer.
• Hands-on experience with data migration tools and frameworks, especially for moving data from HDFS to S3 (
• Strong troubleshooting skills, especially in diagnosing and resolving performance issues in Spark applications and data pipelines.
• Experience in Kubernetes is a plus.