Azure Data Engineer

Remote

USC/GC (W2 Contract)

Role Overview:

We are seeking a Azure Data Engineer with extensive experience in Azure cloud services, data integration, and data transformation to lead the development and maintenance of our advanced data solutions. The ideal candidate will have a proven track record of implementing Azure-based data processing, storage, and analytics solutions that meet complex business requirements.

Key Responsibilities:

• Design, develop, and maintain scalable and efficient data pipelines using Azure Data Factory (ADF), Azure Databricks (ADB), and Azure Stream Analytics.
• Expertly manage data ingestion from multiple sources into Azure Blob Storage, Azure Data Lake, and Azure SQL Database, ensuring data integrity and availability.
• Develop and maintain bilateral FHIR data feeds with Azure Data Factory.
• Implement real-time data streaming solutions using Azure Event Hub and Kafka, integrated with Azure Databricks for real-time analytics.
• Develop and optimize ETL processes for data warehousing, utilizing Azure Synapse Analytics, to support OLTP, OLAP, and BI reporting needs.
• Conduct data modelling, normalization, and denormalization to design efficient data structures for Operational Data Store (ODS) and Data Warehouse solutions.
• Lead data migration projects from on-premises infrastructure to Azure Cloud Services, ensuring seamless transitions and minimal downtime.
• Craft complex T-SQL scripts, including triggers, stored procedures, and functions, to support data processing and reporting requirements.
• Monitor and tune database and query performances, utilizing Azure Monitor and other tools to ensure optimal data processing speeds and system reliability.
• Collaborate closely with the DevOps team to implement CI/CD pipelines for data infrastructure projects using Azure DevOps and Git.
• Produce and maintain detailed documentation for data models, ETL processes, and data pipeline architectures to ensure clarity and sustainability.

Professional Qualifications:

• Minimum of 3 years of experience as a Data Engineer or Developer, with a strong focus on Azure Cloud Services.
• Minimum of 2 years of experience with Azure Data Factory, Azure Databricks, Azure SQL Database, and other Azure data services.
• Proficient in data integration, data ingestion, and real-time data processing techniques.
• Advanced skills in Python, PySpark, Spark SQL, and T-SQL scripting.
• Experience with data modeling, ETL development, and data warehousing principles.
• Familiarity with CI/CD practices, Azure DevOps, and Git.
• Proven ability to work with cross-functional teams to translate business requirements into technical solutions.

Education and Certifications:

• Bachelor’s degree in electrical and computer engineering or related field.

Relevant Azure certifications highly preferred