Getting Wee Archie online: configuring a cluster for network access
Posted: 29 Apr 2016 | 14:42
At the tail end of last year, the EPCC Outreach team launched Wee Archie, a Raspberry Pi cluster designed to demonstrate parallel concepts and the type of work that is carried out on supercomputers such as ARCHER. Since the launch, Wee Archie has travelled around the UK including to Oxford, Birmingham and Dundee.
As part of the upkeep of Wee Archie, the team that looks after the system needs to get new and updated packages and software onto it. However, up to this point Wee Archie has not been able to connect to the internet directly without having another machine to act as a proxy, replacing the functions of Wee Archie's network controller node (which is configured with dnsmasq). This has meant that performing upgrades, or pulling the latest LED animation configurations, managed by a Python script, from a Github repository has been a laborious process.
However, after installing a Wi-Fi adapter to the network controller, it should now be a much simpler task to perform updates to the whole Raspberry Pi cluster. This has not been entirely straightforward as only one node has external network access and most of the time it resides in the JCMB building at the University of Edinburgh. Therefore it is possibly useful to record how this was done as others may be able to re-use this knowledge.
Getting the adapter running
When an adapter was added to the network controller, the network interfaces file (/etc/network/interfaces) was updated to include the following:
iface wlan0 inet manual
iface default inet dhcp
This allows the adapter to be hot-plugged during operation (though in practice this doesn’t happen with Wee Archie) and sets up the interface to use the wpa_supplicant options for wireless networks.
Initially, it was tested with a simple home network. You have to know the Service Set Identifier (SSID) and the network key to do this.
Where do you put your network key? Or configuring wireless at the command line
It should be pointed out that everything here was configured via the command line, no GUI applications or helpers (which would have made the process far easier and more manageable) have been used. After looking at information on setting up a Raspberry Pi with a wi-fi adapter, command line configuration was carried out by editting the wpa_supplicant.conf file (/etc/wpa_supplicant/wpa_supplicant.conf). This file configures the network information (SSID, network key and priorities) for the wi-fi interfaces.
The configuration for the home network looked like:
Here, the network SSID is HOMEMEWF and the shared key is the value MSPOWEEQ.
Just in case the configuration is not automatically updated, the interface wlan0 (the wireless adapter) can be stopped and restarted.
To stop the interface run: sudo ifdown wlan0
To start the interface run: sudo ifup wlan0
NOTE: Depending on how the Pi is configured, “sudo” may not be needed.
So far so good, now the network controller can get onto the internet and download updates.
But what about the rest of the cluster?
None of the other nodes can currently access the internet, as no link exists between the wireless network and the wired network.
Now we could mess around and set up a bridge and have some fancy method to do the routing, but as this is a simple contained system a more expeditious route was used which involved the more basic option of working with iptables and ipmasq.
First, IP forwarding was put in place using the following command:
sudo echo 1 > /proc/sys/net/ipv4/ip_forward
Then ipmasq was installed - this allows the use of MASQUERADE to allow internal nodes to access the internet or external network via an connected node.
sudo apt-get install ipmasq
Then in the file /etc/sysctl.conf, the following line was uncommented and set to:
Rather than regurgitate lots of information about iptables (which can be found via man or google), the following command then allows the other nodes to access the internet:
iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE
This command adds a post routing option to nat (network address translation), where incoming requests from the nodes would be routed through the wireless interface wlan0 using masquerade. It boils down to the idea that the nodes send requests to the network controller node which then routes them to the wireless interface.
Now we find that an external network can now be accessed from the other nodes when we log into
So everything is solved - yes?
Not quite - this was a home network - a simple key system. Previously we established that Wee Archie resides in the JCMB building at the University of Edinburgh most of the time, where two wireless networks are available: central and eduroam.
Initially, we though central would be easier to access - it is an open network accessed via a username/password login page. Ideal, as we can use a command line browser to deal with the login page.
First, adding the following to the wpa_supplicant.conf file:
This means we can connect to open networks (ones with no key) - this is the key_mgmt option, priority here is set to a really low number, which means that we want to connect to this type of network as the least preferred option.
So the central network turned out to not be an option.
The fun of eduroam
eduroam is a network providing roaming access for members of organisations involved in the international research and education community (taken from https://www.eduroam.org/).
eduroam is great for allowing people who travel to different places to have a network they can get access to, EPCC staff have used it to connect to a network across various parts of the world. However, it does have its drawbacks. Even with a GUI application it isn’t the easiest network to set up, there are lots of different options to set, there are certificates to validate/obtain and small issues like that. On top of this only a few places have manual configuration instructions for something like wpa_supplicant.conf as few people configure the access via the command line and somewhat unfortunately the options may be different depending on the implementation an institution has provided.
After no small amount of digging around online guides, instructions and experimenting, the following configuration appears to work for Wee Archie in allowing connection to eduroam at the University of Edinburgh:
The site-specific details were obtained from the Information Services website on eduroam: http://www.ed.ac.uk/information-services/computing/desktop-personal/wireless-networking/wlan-eduroam/eduroam-generic
The key points in using this configuration are that the username and password need to be entered before the connection will work. Currently this is carried out manually.
Wee Archie can now access eduroam, at least in Edinburgh, and the cluster can be updated a little more easily now.
Is this perfect? No, there are likely better ways of configuring the cluster for network access, but this works and is reasonably basic. If you know of one then please do let us know.
The main issue that has been encountered up to this point is that occasionally the network will be dropped without warning and the interface then needs to be restarted to pick the connection up again. Basically this amounts to turning the wifi on and off again on the system.
For the future, a script will be created which will automate a lot of the logging into eduroam - where a person can login and submit their credentials for use in that session of work. The key thing to remember though is that users will need to make sure they remove their credentials from the system lest others gain access to them at a later point.
That was a quick summary of getting Wee Archie on the internet, it was a bit convoluted and at times frustrating but now something works and this is a little bit of a guide on how to get it working.
A guide to creating a Raspberry Pi cluster like Wee Archie will be posted in the coming months, accompanied by the first of our demo codes for the clusters. Look out for an announcement on EPCC's Twitter feed!
You can also read this post on our Medium account.
Find out more
Outreach at EPCC