Sr. Data Engineer

Salary:

Negotiable

Opportunity type:

  1. Interim

Location:

Birmingham, West Midlands

Specialism:

  1. Disruptive Technology

Date posted:

22 Aug 2019

Opportunity reference:

SH9991_1566461406

Opportunity description

Key Responsibilities:

  • Lead the design, implementation, and continuous delivery of pipelines using distributed Azure based big data technologies supporting data processing initiatives across batch and streaming datasets
  • Responsible for development using Scala, Python languages and Big Data Frameworks such as Spark, EMR, Kafka, Storm, Jenkins, Jfrog Artifactory and DataBricks
  • Provide oversight and direction on deployed Azure platform components
  • Process and manage high volume real time customer interaction streams
  • Support the data analytic team in the creation of data models and product/propositions.
  • Development and implementation of data flows and ETL's to ingest data, using both streaming and batch technologies from multiple sources
  • Design and development of the deployment processes using continuous integration
  • Provide technical review of the solution and identify project risks and estimate development time for such solution.
  • Support entire development process and conduct code reviews.