SC Cleared Data Engineer - Leeds or London - £44k - £57k base plus benefits - Hybrid - more time at home but you must be prepared to go on-site

What You’ll Be Doing:

Our client's client engaged a company to embark upon a discovery/proof-of-concept for business intelligence/data warehouse over 4 years. This period was interspersed with multiple breaks whilst the client re-prioritised and determined next steps.

The company helped the client gain a better understanding of the technology options suitable for their desired technology roadmap/ecosystem by putting in place a proof-of-concept which showcased an end-to-end capability and vision of ‘what’ and ‘how’ the client's vision could be realised.

Working together and using an agile delivery methodology the client gained a richer and deeper understanding of its own organisations data and learnt more about the Microsoft technology stack allowing them to improve their internal reporting and their understanding of their business.

They now wish to transform a PoC solution into a fully productionised and supported system.

This is anticipated to be achieved using one or more agile teams in a phased approach to deliver key technology deliverables on their enhancement journey.

Skills & Experience:

Azure Data Factory (ADF)

• Create dimensions incrementally, reducing processing and data needs with a hopeful reduction in the incurred costs for compute, data and network processing.
• Changing the authentication method between ADF and SQL Server to use either a managed identity or service principal to enhance security and manageability.
• Establishing parallelism within existing and new ADF pipeline activities.
• Template elements of an existing pipeline to support the ingestion of new datasets through the platform

Azure SQL

• Integrating Database Access with Microsoft Entra ID for enhanced security.
• Implementing STAR schemas on Data Models.
• Establishing naming conventions for database objects.
• Transitioning data processing from Power BI to Azure SQL.
• Segregating data serve level data into multiple schemas or views.
• Data segregation and redaction techniques, such as data masking and data pseudonymisation.

Benefits:

As well as a competitive pension scheme, our client offers an employee share plan, an extensive range of flexible discounted health, wellbeing and lifestyle benefits including including a green care scheme, private health plans and shopping discounts – you may also be eligible for an annual incentive.

SC Cleared Data Engineer - Leeds or London - £44k - £57k base plus benefits - Hybrid - more time at home but you must be prepared to go on-site