Posted: 8 Jan 2018 | 16:03
Big data has always been a part of high-performance computing and the science it supports, but new open-source technologies are now being applied to a wider range of scientific and business problems. We’ve spent time recently testing some of the big data toolkits.
Posted: 20 Dec 2017 | 13:37
I've worked on many data analysis projects in my career and two common themes are that obtaining the data can be a significant challenge and that once you obtain it you’ll notice it is very messy.
Posted: 21 Nov 2017 | 16:10
NHS Scotland allows research using routinely collected, unconsented patient data. Additionally, these data can be linked to social data such as education. The research this enables can have an enormous public benefit but the use of these data must be managed very carefully to safeguard privacy and maintain public trust and support.
EPCC is responsible for building, supporting and hosting the infrastructure of the National Safe Haven, and we continue to develop the infrastructure and software to further enhance the service.
Posted: 10 Aug 2017 | 20:22
Distributed ledgers, the core technology underlying digital currencies such as BitCoin, offer some interesting functionality for constructing distributed data infrastructures.
Ledgers can be considered to be simple data stores. They are styled on accounting ledgers, books where transactions are recorded one after the other, and the overall state of the accounts can be evaluated by working through the recorded transactions to calculate how much money has flowed in and out of the accounts.
Posted: 29 Nov 2016 | 10:07
TensorFlow is an incredibly powerful new framework for deep learning. The “MNIST For ML Beginners” and “Deep MNIST for Experts” TensorFlow tutorials give an excellent introduction to the framework. This article acts as a follow-on tutorial which addresses the following issues:
- The above tutorials use the MNIST dataset of hand written numbers, which pre-exists in TensorFlow TFRecord format and is loaded automatically. This can be a bit mysterious if you have no experience of data format manipulation in TensorFlow.
- Since the MNIST dataset is fixed, there is little scope for experimentation through adjusting the images and network to get a feel for how to deal with particular aspects of real data.
Posted: 7 Sep 2016 | 10:54
The Software Sustainability Institute's Fellowship programme funds researchers in exchange for their expertise and advice.
Find out more about the 2017 programme with the Launch Webinar this Friday 9 Sep at 3pm!
Posted: 29 Aug 2016 | 10:35
With my Software Sustainability Institute hat on, I recently participated in a back-to-back Data Carpentry and Software Carpentry course sponsored by the University's Research Data Service here at the University of Edinburgh. The courses were held in the main University library in a gorgeous room with a glass wall, providing a rather distracting view of the Meadows parkland.
Posted: 8 Jul 2016 | 14:48
Modern genome-sequencing technologies are easily capable of producing data volumes that can swamp a genetic researcher’s existing computing infrastructure. EPCC is working with the breeding company Aviagen to build a system that allows such researchers to scale up their data infrastructures to handle these increases in volume without compromising their analytical pipelines.
Posted: 6 Jul 2016 | 14:36
Safe havens allow data from electronic records to be used to support research when it is not practicable to obtain individual patient consent while protecting patient identity and privacy. EPCC is now the operator of the new NHS National Services Scotland (NSS) national safe haven in collaboration with the Farr Institute of Health Informatics Research which provides the infrastructure.
Posted: 24 May 2016 | 09:49
This is an exciting time for astronomy in the UK, a fact that is reflected by our involvement and leadership of some amazingly ambitious new telescopes.
A number of recent, significant discoveries have propelled astronomy research into the spotlight. The discovery of dark matter and dark energy at the beginning of the 21st century over-turned our understanding of how the Universe works. And the first observation of a gravitational wave earlier this year confirmed Albert Einstein’s long-standing hypothesis precisely 100 years after it was first published in his general theory of relativity.