Looking for a highly skilled and experienced Senior Technical Lead (8+ Years) with deep expertise in Azure Databricks to architect and lead data engineering and analytics initiatives. The ideal candidate will have a strong background in Databricks and other big data technologies, cloud architecture, and leading high-performing engineering teams.

Key Responsibilities:

• Lead end-to-end design, development, and deployment of data pipelines using Azure Databricks.
• Architect scalable data solutions integrating various Azure services (e.g., Data Lake, Synapse, Data Factory, Event Hubs, etc.).
• Guide development teams on best practices for Spark optimization, Delta Lake, streaming, and ML integration.
• Collaborate with data scientists, analysts, and business stakeholders to understand and deliver on analytics requirements.
• Define and enforce coding standards, CI/CD processes, and DevOps practices for Databricks environments.
• Troubleshoot complex data issues and performance bottlenecks.
• Mentor junior engineers and contribute to building a strong data engineering culture.
• Stay current with industry trends and advancements in Azure, Databricks, and big data ecosystems.

Required Skills:

• Hands-on experience with Azure Databricks (including PySpark, Delta Lake, notebooks).
• Strong proficiency in Apache Spark
• Expertise in Python, SQL, and optionally Scala.
• Deep understanding of Azure Data Lake Storage (ADLS), Azure Data Factory (ADF), and Azure Synapse.
• Experience with CI/CD pipelines for Databricks (e.g., using Azure DevOps, GitHub Actions).
• Experience with data governance, security (e.g., Unity Catalog), and cost optimization in cloud environments.
• Proven leadership in managing technical teams and delivering large-scale data projects.
• Certified in Databricks