Art and artificial intelligence

Image shows a satellite view of agricultural fields in Saudi Arabia (Google Earth, Maxar Technologies, CNES/Airbus, Landsat/Copernicus, 2021). Taken from Inés Cámara Leret’s The Overlay, an early prototype work in part created/inspired by the TNRO platform and which was exhibited at the 2022 Edinburgh International Science Festival.

The main technique currently being used in the TNRO platform is the application of Generative Adversarial Networks (GANs) to images, although GANs can also be used in wider application domains and there are plans to incorporate audio and text input/outputs capabilities.

In a GAN a pair of neural networks  (a generator and a discriminator) are tuned in a zero-sum game on a supplied input data set. This could be eg a combination of images of Scottish plants and cacti. The training of the neural networks then proceeds in a double loop: in the inner loop the generator attempts to create images that look like Scottish plants and/or cacti; in the outer loop the discriminator is trained, using the same data set, to distinguish those images that look like Scottish plants/cacti from those that do not.

The training of a GAN can be very computationally expensive so TNRO employs pretrained GANs, with artist-supplied images used for final tuning – even this can take from a couple of hours to a day on a GPU. Once trained the GAN operates relatively quickly and can transform an image in seconds.

All the supplied images can be mapped to a multi-dimensional abstract space called the latent space. The artist will supply two classes of images – in the latent space a line can be set from the centroid of one of the image’s classes to the centroid of the other class so moving along this line can make a new ‘visual dimension’, as imagined by the artist, that spans Scottish flowers to cacti.

Another aspect of the TNRO platform now comes into play. Using data from the ECMWF Copernicus Climate Data Store that has been incorporated into the TNRO platform, the artist can project the change in temperature, wind or rainfall at a given location on Earth within the next 70 or so years using one of three projection models – from pessimistic to optimistic – to determine whether their image should eg look more like a cactus or a Scottish flower.

This is only one possible use case, and the artists are using the platform in more imaginative and abstract ways.

The New Real Observatory (TNRO) is a sub project of The New Real. The TNRO platform is being developed by Informatics, EPCC, the School of Engineering and the Edinburgh College of Art all at the University of Edinburgh and is funded by the EPSRC via the Alan Turing Institute. The New Real is based at the Edinburgh Futures Institute.

The New Real Observatory https://newreal.cc/

The Overlay by Inés Cámara Leret

https://newreal.cc/artwork/the-overlay

Subscribe to our newsletter

To read more stories like this, subscribe to EPCC News (two issues per year). See link at foot of page.