Senior Data Engineer

We are seeking a skilled Senior Data Engineer with experience in the data engineering domain to join our team. The ideal candidate will be responsible for designing, developing, and maintaining robust data pipelines, ensuring data quality, and collaborating with cross-functional teams to support our data infrastructure.

Key Responsibilities

• Troubleshoot data pipelines and address issues related to real-time and batch data processing, as well as integrations.
• Collaborate with cross-functional teams to understand and resolve challenges in data pipelines.
• Ensure data quality, integrity, and governance by implementing best practices in data management.
• Develop ETL processes and data flows using tools such as PySpark and Apache Airflow.
• Maintain and troubleshoot CI/CD pipeline issues using GitHub Actions and JFrog.

Required Skills and Qualifications

• Experience in the Data Engineering domain.
• Proficiency in PySpark and Apache Airflow for building and managing data pipelines.
• Strong knowledge of Unix for system-level operations.
• Experience with Docker and Kubernetes for containerization and orchestration.
• Familiarity with CI/CD tools, including GitHub Actions and JFrog, for automated workflows.
• Excellent problem-solving skills and the ability to work in a collaborative team environment.

Preferred Qualifications

• Experience working with large-scale, distributed data systems.
• Knowledge of data governance frameworks and best practices.
• Strong communication skills to liaise with cross-functional teams.