Azure Cloud Data Engineer

4-5 months Contract

REMOTE in the CST

Position Overview

Join our team to support a D365 to Salesforce CRM migration project. You’ll help build and enhance data pipelines and transformations to extract from Dynamics 365, process and stage data in Azure/Databricks, and reliably upsert into Salesforce. Your work will ensure data consistency for both CRM operations and downstream analytics.

Key Responsibilities

• Participate in the end-to-end CRM migration from Dynamics 365 to Salesforce:

Analyze D365 source schemas and Salesforce target models

Develop & maintain ADF pipelines for extraction, transformation, & load of CRM entities

Implement incremental (delta) loads, upsert logic, and robust error-handling patterns

• Author complex transformations in Databricks (PySpark or Scala) for data cleansing, mapping,
• and enrichment
• Follow established CI/CD processes using Azure DevOps pipelines for ADF ARM template and
• notebook deployments
• Collaborate with CRM architects, business analysts, and offshore teams to validate mappings
• and resolve data discrepancies
• Implement monitoring, logging, and alerting for pipeline health, SLA adherence, and data-
• quality checks
• Optimize pipeline performance through smart partitioning, cluster sizing, and query tuning
• Ensure data security and compliance by enforcing least-privilege access and encrypted storage

Required Qualifications

• 3+ years’ experience as a Cloud Data Engineer with hands-on Azure and Databricks expertise
• Proficiency with Azure Data Factory (including Self-Hosted Integration Runtime), ADLS Gen2, and Blob Storage
• Strong skills in PySpark or Scala for Databricks-based ETL/ELT development
• Prior experience migrating or integrating data between enterprise systems (e.g. D365 → Salesforce)
• Familiarity with upsert patterns, delta-load techniques, and referential integrity
• Experience working within CI/CD processes in Azure DevOps
• Excellent problem-solving, attention to detail, and communication skills

Preferred Qualifications

• Knowledge of the Salesforce data model and experience with Bulk or Metadata APIs
• Exposure to data validation and reconciliation tools or frameworks
• Understanding of data governance and data-quality best practices
• Azure Data Engineer or Databricks certification