June 2021

Accelerating high performance data pipelines using EPCC's supercomputing facilities

Author: Guest blogger
Posted: 15 Jun 2021 | 12:26

illuminate logoIlluminate, a commercial collaborator of EPCC, has a mission to accelerate informed decisions by providing their customers with the means to find specific needle in a haystack data points from the mass of their network traffic. As networks evolve, they are continuously innovating to ensure that their data pipelines operate at maximum efficiency.

Data analytics pipelines have three main functions: data ingestion, processing, and analysis. The data ingestion stage focuses on obtaining or extracting data from a range of data sources and importing it into systems that can enable processing and analysis. This is followed by data processing to validate, transform and enrich it, and load it into some form of queryable data storage, such as a data lake. Finally, we utilise Artificial Intelligence/Machine Learning processes to build models from the data in the lake and use them to gain insight.

 

EuroCC@UK: promoting excellence in HPC, HPDA and AI

Author: Mark Sawyer
Posted: 7 Jun 2021 | 10:05

The European Commission has launched a network of National Centres of Excellence in High-Performance Computing (HPC), High-Performance Data Analytics (HPDA) and Artificial Intelligence (AI). The UK centre will be delivered jointly by EPCC and the Hartree Centre.

Blog Archive