Accessability Links

Data Engineer

  • Location: Wiltshire
  • Salary: £40000.00 - £60000.00 per annum
  • Job type: Permanent
  • Sector: BI, Big Data & Analytics

Role: Data Engineer

Location: Wiltshire

Type: Permanent

Salary: Negotiable

Capita are looking for a Data Engineer to be based on client site in Wiltshire working to maintain and enhance the client's data environment across a range of projects spanning in scope and complexity.

Engineering activities including but not limited to ingesting, cleansing and modelling data as per specifications provided.

As a Data Engineer you will be responsible for:

  • Developing, industrialising, and optimising big data platforms across AWS and GCP.
  • You will ingest new data sources, model the data, and enrich it using various means.
  • Working with data from any source, you will understand the best way to serve it up at scale to a global audience of analysts.
  • You will be expected to work closely with data architects and data analysts on the team to ensure that we are building one integrated, high performance solution.
  • You will be working on a vast array of projects requiring different sources of data in size and complexity, you will be required to support all data engineering activities as specified in the project requirements and you will be expected to deliver within the specified timelines.

The experience required is:

  • Strong data analysis skills using SQL and Python (and as a bonus, R) in Relational/Columnar/Big data environments (Joins, Union, rank, group by, order etc.)
  • Experience in designing data ingestion and ETL/ELT pipelines
  • Experience in Hadoop distributions (Apache / Cloudera / Hortonworks)
  • Ability to deploy and maintain multi-node Hadoop cluster
  • Experience working with Big Data eco-system including tools such as Hadoop, Map Reduce, Yarn, Hive, Pig, Impala, Spark , Kafka, and Storm to name a few
  • Knowledgeable in techniques for designing Hadoop-based file layout optimized to meet business needs
  • Understands the trade-offs between different approaches to Hadoop file design
  • Experience with techniques of performance optimization for both data loading and data retrieval
  • Experience with NoSQL Databases - HBase, Apache Cassandra, Vertica, or MongoDB
  • Ability to build and test rapidly Map Reduce code in a rapid, iterative manner
  • Ideally an understanding of AWS or GCP services, and how they accelerate development

Capita IT Resourcing is acting as an Employment Agency in relation to this vacancy.

Start your search again

Similar jobs
Data Scientist
South West England Up to £1 per annum + Completely Negotiable
Back to Top