Do you have experience of providing data engineering expertise on a complex data improvement project within a financial services environment?
This consulting opportunity is to support our client, a leading wealth management organisation, with their Azure Data Platform transition. We are seeking a team of Data Engineers including one Lead and 2 Engineers, capable of designing and building enterprise-grade data pipelines across Raw, Curated and Data Product layers. The work will focus on developing scalable data pipelines and supporting data modelling and deployment activities within a Databricks environment.
The project needs a consultant who has:
- strong end-to-end data pipeline design and development experience
- strong Python, Azure Data Factory and Databricks data engineering expertise
- experience across curated and product data layers
- practical knowledge of Data Vault 2.0, Kimball dimensional modelling and 3NF
- experience working with CI/CD pipelines and Git-based deployment approaches.
Key Services
The key services that you will be asked to provide are:
- end-to-end data pipeline design and development
- Fact, Dimension and Bridge table implementation
- data quality controls, reconciliation and referential integrity
- workflow orchestration, monitoring and SLA management
- performance optimisation and tuning
- unit, integration and automated testing
- CI/CD and DevOps practices
- data catalogue management and access controls
- data retention, security and privacy requirements
Mandatory Requirements
Successful consultants will be able to demonstrate:
- Azure Data Factory and Azure Databricks experience
- SQL, Python and PySpark expertise
- 3NF and dimensional data modelling experience
- enterprise-grade data pipeline development experience
- data quality, reconciliation and referential integrity experience
- workflow orchestration and monitoring experience
- performance optimisation and testing experience
- CI/CD and DevOps experience
- data governance and access control experience
Optional Requirements
It will also be advantageous if you have:
- experience within wealth management or financial services environments
- experience using VSCode or similar IDE environments
- experience working within distributed remote delivery teams.
Working Pattern: Remote (UK based) with occasional travel to Southampton
Contract Type: Inside IR35
Start: 1st October
Duration: 3-9 months