We are seeking a skilled Azure Data Engineer to join our team. The ideal candidate will design, implement, and manage data solutions on the Microsoft Azure platform. This role involves working with large datasets, optimizing data pipelines, and collaborating with cross-functional teams to support data-driven decision-making.

Key Responsibilities:

• Design and develop scalable data pipelines and workflows using Azure Data Factory, Databricks, and other Azure tools.
• Utilize PySpark for processing large datasets and creating efficient workflows
• Manage and optimize data storage solutions, including Azure Data Lake, Azure SQL Database, and Cosmos DB.
• Implement ETL processes to extract, transform, and load data from various sources into data warehouses or lakes.
• Collaborate with data scientists and analysts to ensure data availability and usability.
• Ensure data security and compliance with organizational and regulatory standards.
• Monitor and troubleshoot data systems to ensure optimal performance.
• Identify opportunities for process improvements and automation.

Qualifications:

• Proven experience as a Data Engineer, preferably with Azure.
• Proficiency in Python and PySpark for data manipulation and pipeline creation.
• Proficiency in SQL and data modeling.
• Familiarity with big data tools like Apache Spark and Azure Databricks.
• Strong understanding of data integration and ETL processes.
• Knowledge of data security practices and compliance standards.
• Excellent problem-solving and communication skills.