Title: Spark Scala Data Engineer

We are seeking a highly skilled Data Engineer with expertise in Scala, Spark, SQL, AWS. The ideal candidate should have experience in building piplelines using Scala with Spark in a fast-paced environment, demonstrate a high degree of ownership, and be comfortable working independently with minimal oversight.

A strong ability to learn new skills quickly is highly desirable, as we would like this individual to also gain expertise in Google Cloud (GCP) over time.

Key Responsibilities:

• Design, develop, and optimize ETL pipelines on AWS using Scala with Spark, SQL, and Python.
• Implement performance tuning techniques to enhance data workflows.
• Develop dashboards and reports using Amazon QuickSight to enable data-driven decision-making.
• Ensure data quality, reliability, and security across all data pipelines.
• Troubleshoot and optimize SQL queries and Spark jobs for performance efficiency.
• Work collaboratively with cross-functional teams, including data scientists, analysts, and software engineers, to support business objectives.
• Maintain detailed technical documentation for data processes and pipelines.
• Stay updated with the latest AWS data services and best practices to continuously improve the data architecture.
• Be open to learning and working on Google Cloud (GCP)

Required Skills & Experience:

• 3 years of experience as a Data Engineer, preferably in a fast-paced or startup environment.
• Proficiency in Spark Scala, Python, and SQL for data transformations and processing.
• Strong hands-on experience in AWS services such as S3, EMR, Lambda, Glue, Athena, and QuickSight
• Experience in designing and optimizing ETL processes for large-scale data workloads.
• Strong knowledge of Amazon QuickSight for data visualization and reporting.
• Familiarity with data lake and warehouse architectures on AWS.
• Ability to work independently with minimal supervision and a high sense of ownership and accountability.
• Eagerness to learn and quickly adapt to new technologies, including Google Cloud (GCP).
• Excellent problem-solving skills with a proactive and results-driven mindset.

Preferred Qualifications:

• Experience with CI/CD pipelines for data workflows.
• Knowledge of data governance, security, and compliance best practices.
• Experience working with real-time or streaming data solutions on AWS.
• Exposure to Google Cloud (GCP) services or a willingness to ramp up quickly.
• Background in machine learning or data science pipelines is a plus.

Why Join Us?

• Work on challenging high-impact data engineering projects in a fast-moving environment.
• Be part of a culture that values ownership, innovation, and independence.
• Opportunity to work with cutting-edge AWS technologies and expand expertise into Google Cloud (GCP).