Data research

Big Data training session: EPCC & EUDAT at ISC Big Data

Author: Adam Carter
Posted: 9 Sep 2014 | 09:53
ISC is Europe’s biggest supercomputing conference, and EPCC has been represented at the event for a number of years now. More recently, as the interest in Big Data has grown, ISC has launched a new conference - ISC Big Data - which is specifically focussed on this new field.

Research data infrastructure: where next?

Author: Rob Baxter
Posted: 30 Jul 2014 | 16:11

The rise of data-driven science, the increasing digitisation of the research process, has spawned its own jargon and acronyms. “Research data infrastructure” is one such term, but what does it mean? 

SPRINT v1.0.6 released! OpenMPI and R V3 compatible

Author: Terry Sloan
Posted: 30 Jun 2014 | 13:30
 
EPCC and the Division of Pathway Medicine at the University of Edinburgh have released version 1.0.6 of the SPRINT R software package. This is compatible with R Version 3 and now supports OpenMPI as well as MPICH.

Causing a Storm in MPI: easier data processing for scientists

Author: Amy Krause
Posted: 17 Jun 2014 | 15:00

After several years of working with users who are not computer scientists (seismologists and geoscientists), we have realised two main points: these communities usually have problems that should be addressed with parallel computing, but they don't often have the skills and training to do so. We set out to build a programming library, Dispel4Py, that both enables users to easily write a description of a data-processing application and takes care of running that application in different parallel environments.

Bootstrapping with R and SPRINT

Author: Terry Sloan
Posted: 24 Feb 2014 | 12:10

EPCC and the Division of Pathway Medicine at the University of Edinburgh have made public the report from their recent study into the performance of bootstrapping within their SPRINT R software package.

Data infrastructure: highlights of the EUDAT Conference 2013

Author: Rob Baxter
Posted: 13 Nov 2013 | 14:27

EUDAT - the European Data Infrastructure project - has reached the end of its second year and has, with some success, distilled the first version of a common, collaborative, horizontal data infrastructure from among the vertical stacks of its various partners.

Data interoperability is a state of mind

Author: Rob Baxter
Posted: 9 Sep 2013 | 09:58

The research data tsunami is firmly upon us. Open access to data is very much on the agenda. One of the hopes for capturing and preserving all these data is that reuse and recombination may yield new science. Improving the interoperability of data from different domains is key to making this a reality.

Now, data interoperability is not technically hard, so why are we not further on?

Securely citing datasets

Author: Guest blogger
Posted: 22 Aug 2013 | 14:50

This post was written by Adrian Mouat, a former EPCC employee who is now an independent software consultant.

Citing a paper is a reasonably straightforward and well-defined task; just give a reference to the author and the publication you found the paper in and you're pretty much there. Anyone else who wants to look up the reference just has to find the publication and they should see exactly the same text you saw.

Unfortunately, citing datasets is not as simple, at least not if you want the security of knowing that readers who follow the citation will find exactly the same data you used.

Wait a minute, where *are* my data?

Author: Rob Baxter
Posted: 7 Aug 2013 | 11:56

Policy restrictions on data storage can make the straightforward technological problems complex, over-constrained and potentially insoluble.

Pic credit:  Jeff Rowley Big Wave Surfer

As the slowly toppling wave of research data begins to overwhelm us all, we're increasingly looking for new ways to automate the management of all these bits. Keeping human curators and data managers in the loop becomes ever more unscalable and unsustainable. So, we're storing data in the Cloud, auto-replicating them five ways so we don't lose any, letting the systems manage the data for us.

iCORDI at EGU2013

Author: Mark Filipiak
Posted: 31 May 2013 | 09:08

Austria Center, Vienna (© IAKW-AG / Marius Höfinger)

This year’s European Geophysical Union General Assembly (EGU2013) was held last month at the Austria Center in Vienna.  About 11,000 participants come together from all fields in Earth science: seismology, oceanography, geology, meteorology, planetology… you name it, it’s there. So, lots of parallel sessions. I gave a presentation on iCORDI and the RDA at two sessions: ‘ICT-based hydrometeorology science and natural disaster societal impact assessment’ and ‘Marine Data Management’.

Pages