Senior Data Engineer

Responsibilities:

• Lead the design, development, and implementation of scalable data pipelines and ELT processes using Databricks, DLT, dbt, Airflow, and other tools.
• Collaborate with stakeholders to understand data requirements and deliver high-quality data solutions.
• Optimize and maintain existing data pipelines to ensure data quality, reliability, and performance.
• Develop and enforce data engineering best practices, including coding standards, testing, and documentation.
• Mentor junior data engineers, providing technical leadership and fostering a culture of continuous learning and improvement.
• Monitor and troubleshoot data pipeline issues, ensuring timely resolution and minimal disruption to business operations.
• Stay up to date with the latest industry trends and technologies, and proactively recommend improvements to our data engineering practices.

Requirements/Qualifications:

• 8+ years of experience in data engineering and/or architecture, with a focus on big data technologies.
• Extensive production experience with Databricks, Apache Spark, and other related technologies.
• Familiarity with orchestration and ELT tools like Airflow, dbt, etc.
• Expert SQL knowledge.
• Proficiency in programming languages such as Python, Scala, or Java.
• Strong understanding of data warehousing concepts.
• Experience with cloud platforms such as Azure, AWS, and Google Cloud.
• Excellent problem-solving skills and the ability to work in a fast-paced, collaborative environment.
• Strong communication and leadership skills, with the ability to effectively mentor and guide team members.

Preferred Qualifications:

• Experience with machine learning and data science workflows.
• Knowledge of data governance and security best practices.
• Certification in Databricks, Azure, Google Cloud, or related technologies.