This opportunity is being managed by Sasza Bryza
Consultant
Junior Data Engineer
Salary:
Negotiable
Opportunity type:
Location:
Newcastle upon Tyne, Tyne and Wear
Specialism:
Date posted:
Posting date:
22 Aug 2019
Opportunity reference:
SH10060_1566470170
Opportunity description
This vacancy has now expired. Please see similar roles below...
Key Responsibilities:
- Lead the design, implementation, and continuous delivery of pipelines using distributed Azure based big data technologies supporting data processing initiatives across batch and streaming datasets
- Responsible for development using Scala, Python languages and Big Data Frameworks such as Spark, EMR, Kafka, Storm, Jenkins, Jfrog Artifactory and DataBricks
- Provide oversight and direction on deployed Azure platform components
- Process and manage high volume real time customer interaction streams
- Support the data analytic team in the creation of data models and product/propositions.
- Development and implementation of data flows and ETL's to ingest data, using both streaming and batch technologies from multiple sources
- Design and development of the deployment processes using continuous integration
- Provide technical review of the solution and identify project risks and estimate development time for such solution.
- Support entire development process and conduct code reviews.
Requirements:
- Bachelor's Degree or higher in Computer Sciences or similar
- Development experience with Azure services
- Development experience with GCP services (Cloud Storage, Cloud Spanner, BigQuery)
- Experience with Apache Spark and NoSQL Implementation
- Proficiency working with structured, semi-structured and unstructured data sets including social, web logs and real time streaming data feeds
- Able to tune Big Data solutions to improve performance and end-user experience
- Expert level usage with Jenkins, GitHub CI/CD is preferred
- Must have the legal right to work in the United Kingdom
Similar opportunities
Salary
£65000.00 - £75000.00 per annum + Bonus and Benefits
Location
London
Salary
£100000.00 - £110000.00 per annum + bonus
Location
London
Salary
Up to £500.00 per day + IR35 to be determined
Location
South East England
Salary
Up to £500.00 per day + IR35 to be determined
Location
South East England