Mandatory skills

6+ years of experience in data engineering and minimum 2+ years as technical lead role.

3+ years of hands-on experience with Azure Data Factory, Azure SQL Database, and other Azure data services.

Experience in DataStage and migrating of ETL from DataStage to ADF

Proven experience with Azure cloud services, including Azure Data Lake, Azure Synapse Analytics, Azure Data Factory, and Azure DevOps.

Develop and maintain ETL pipelines to extract transform and load data from various sources into Databricks.

Implement Azure data engineering best practices including data quality checks data lineage and data governance within the Databricks environment.

Leverage Azure DevOps and CI CD best practices to automate the deployment and management of data pipelines and infrastructure.

Proficient in programming languages such as Python, SQL

Experience with SQL and NoSQL databases, particularly within Azure.

Familiarity with data warehousing concepts and tools (e.g., Azure Synapse Analytics).

Knowledge of Azure data governance, security, and compliance best practices.

Deep understanding of Azure Cloud Platform and Engineering Infrastructure setup and configuration tools and services Data access and sharing and Hands on work.

Optional skills

Handson working experience on Python Py Spark using Databricks Notebooks and Workspace.

Architect and Design Databricks deployment strategy and experience on data engineering tasks including data processing transformation and integration

Strong understanding of distributed computing principles and data processing frameworks.

Deep understanding of Databricks tools and products like Delta Tables Delta Like Tables Unity Catalog Serverless execution and setup and configuring experience