Must-Have Skills

• Good experience in Pyspark - Including Dataframe core functions and Spark SQL
• Good experience in SQL DBs - Be able to write queries including fair complexity.
• Should have excellent experience in Big Data programming for data transformation and aggregations
• Good at ELT architecture. Business rules processing and data extraction from Data Lake into data streams for business consumption.
• Good customer communication.
• Good Analytical skill
  
  

Technology Skills (Good To Have)

• Building and operationalizing large scale enterprise data solutions and applications using one or more of AZURE data and analytics services in combination with custom solutions - Azure Synapse/Azure SQL DWH, Azure Data Lake, Azure Blob Storage, Spark, HDInsights, Databricks, CosmosDB, EventHub/IOTHub.
• Experience in migrating on-premise data warehouses to data platforms on AZURE cloud.
• Designing and implementing data engineering, ingestion, and transformation functions
• Azure Synapse or Azure SQL data warehouse
• Spark on Azure is available in HD insights and data bricks
  
  

Skills:- PySpark, SQL, Data engineering, Big Data, Spark, Data Warehouse (DWH) and Apache Synapse