Key Responsibilities:

• Data Lake Development: Must have experience being part of a team that has built a Data Lake from scratch. You will be responsible for designing, implementing, and maintaining the Data Lake infrastructure.
• Azure Data Factory: Utilize Azure Data Factory to orchestrate and automate data movement and transformation. This includes creating and managing data pipelines, integrating various data sources, and ensuring data quality and consistency.
• Databricks: Leverage Databricks for big data processing and analytics. Implement scalable data processing solutions and collaborate with data scientists to optimize data workflows.
• Infrastructure as Code: Use tools like Terraform and Azure Resource Manager (ARM) templates to define and manage infrastructure. Ensure that infrastructure is scalable, secure, and compliant with industry standards.
• CI/CD Experience: Develop and maintain CI/CD pipelines to automate the deployment of data solutions. Ensure that code changes are tested, validated, and deployed efficiently.
• Unity Catalog in Azure: Implement and manage Unity Catalog in Azure to ensure data governance, security, and compliance across the Data Lake.

Qualifications:

• Proven experience in building and managing Data Lakes from scratch.
• Strong proficiency in Azure Data Factory, Databricks, and other Azure services.
• Expertise in Infrastructure as Code (IaC) and CI/CD practices.
• Azure 104 certification is required.
• Experience with Unity Catalog in Azure.
• Excellent problem-solving skills and the ability to troubleshoot complex issues.
• Strong communication and collaboration skills.

Preferred Skills:

• Experience with other cloud platforms and data engineering tools.
• Knowledge of data security best practices and compliance requirements.
• Familiarity with legal industry data requirements and standards.
• Preferred Tech leadership/guidance experience in developing Cloud Solutions in Azure