UCAS uses Splunk to get real-time data on A-level results day

The day when A-level results come out is a dramatic occasion for UCAS’s IT team, which uses Splunk to troubleshoot the network

Thursday 14 August is D-day for the Universities and Colleges Admissions Service (UCAS), the organisation responsible for managing university and college applications. That is the day when the A-level results come out, and UCAS, its IT team and the broader group of suppliers who deliver its technology, spring into life. 

Another crunch day for the organisation came on 5 August, when Scotland's Higher results were announced.

Peter Raymond (pictured), enterprise IT architect at UCAS, says the work of the Joint Operations Centre (JOC) at results time benefits from an operational intelligence capability vouchsafed by machine data indexer Splunk.

The JOC comprises about 20 people, including the organisation’s IT director, systems operations staff, architects and workers from suppliers such as Amazon, Microsoft and Oracle.

UCAS has been using Splunk to monitor its IT infrastructure following a partial migration to the cloud, when it found it needed to aggregate device logs and intelligently search them across multiple servers.

The deployment won a Computer Weekly enterprise software user award earlier this summer.

The technology, which indexes and renders searchable machine data, enables UCAS to troubleshoot, manage performance and use analytics to support the IT team. This means students can access information quickly and easily on Track, UCAS’s online application portal.

The organisation began using the technology in July 2013. “It was a big success for us in how we could visualise user experience response times and the sheer volume of transactions,” says Raymond.

UCAS deploys Splunk Enterprise across 40 servers and about 70 log sources, all of which are deployed through Amazon Web Services, and everything is forwarded to a Splunk server for indexing.

By indexing, searching, alerting and reporting on machine data from data sources across UCAS’s IT infrastructure, Splunk gives the IT team a series of visualisations of their system performance, key operational metrics (broken down by the Higher Education Institution), their usage, the queries they are running and how the various applications are functioning.

UCAS has 15 dashboards that give real-time monitoring and insight into system load and response times.

At its peak on 15 August last year, the Track system saw more than 180 logins per second. It is hosted on the Microsoft Azure cloud service, while AWS supports more than 350 higher education providers.

The deployment of Splunk Enterprise has enabled UCAS to provide a consistent approach to log collection and retention and expose the data in a searchable form. Logs were previously available by accessing each server, which required system administrator time and did not allow related events on different systems to be found easily.

Raymond says: “The Splunk logs give us both a view of the customer experience and also operationally for us. For example, there is one integration between Azure and AWS and we monitor that integration point with Splunk.

“Instead of an error being tucked away in a log file, we can now get automatic notifications.”

The UCAS site in Cheltenham has 10 large screens, two of which are devoted to Splunk. On those two screens are 10 dashboards, created through queries in Splunk. These include dashboards devoted to tracking and monitoring the user experience of “higher education providers”. One dashboard shows the response time on an applicant enquiry, and another gives response time over a 24-hour period, says Raymond.

“UCAS lives and dies on one day a year,” he adds. “There are TV crews here, [government] ministers walking round. It’s a very visible day.”

Read more on Business intelligence and analytics