How startup Once.net and Cloudflare secured the 2023 Eurovision vote

How Based.io works

The Based.io platform uses a client-side service discovery pattern as opposed to network or HTTP load balancers, which means it selects the most suitable server to connect to at any one time, using Cloudflare’s fast cache propagation infrastructure in order to cope with bumpy traffic.

Each of these servers continuously registers a unique access key that turns over ever 15 seconds, which must be used when a client connects. At the same time, the backend servers ping their health data to the service registry every 300 milliseconds, so when a client shows up it can request the healthiest server URL for that instant (and its associated access key) from a central registry and connect to it. The server will disconnect a number of clients should it become overloaded, who will have to move through the process again.

This central registry would naturally appear to be a bottleneck, and as such a prime target for an attacker. This issue is dealt with by putting it behind Cloudflare’s global network, with a cache time of three seconds. Because the system relies on the real-time health stats to distribute the load of incoming traffic fairly, and uses short-lived access keys, this cache must update quickly and without problems, therefore Cloudflare’s Tiered Caching option also comes into play.

Additionally, by removing load balancers, the platform lets clients connect to the backend servers through Cloudflare, ultimately meaning clients see better system performance backed by a more cyber resilient infrastructure – since the load balancers aren’t there, they can’t constitute an attack surface. Furthermore, connections can be better distributed using the real-time health data.

If the platform needs to scale, which as we shall see it did, this can be done automatically by deploying batches of up to 200 machines capable of handling up to 40,000 connected users apiece each reaching out directly to the Cloudflare API to configure its own DNS record and proxy status. Cloudflare’s high-speed DNS system means these changes can be propagated across the global network within seconds.

At a high level, clients – in this case, voters – arrive at the voting landing page, esc.vote. Rather than build this themselves and expend time and effort on owning, configuring and managing their own infrastructure, the Once.net team decided hosted this page on Cloudflare pages, which meant deploying was as quick as a commit to their Git repository, while reachability or scaling worries melted away.

As an additional barrier to address the potential problem of fraudulent or bot voting attempts to rig the contest, De Beers also deployed Cloudflare Turnstile to protect the payment API endpoints that were being used to validate the vote. Turnstile is a service that makes sure a request is not coming from an emulated browser used by a bot by checking in on various characteristics and processes inherent to browsers and how they behave when humans are using them versus when bots do.

Unlike a Captcha, this process was entirely invisible to the voters, but it allowed the contest to benefit from better conversion rates because fraudulent traffic was filtered out and legitimate users could connect quicker.

All right on the night

The two semi-finals went entirely according to plan, with approximately 200,000 concurrent voters  hitting up the system across the two nights.

On the night of the Grand Finale, Saturday 13 May, De Beers and the team first made sure they had enough machines available to handle the initial peak when voting opened, and then conducted ongoing monitoring with Cloudflare.

They did clock a few attempts to DDoS the site, but these were automatically mitigated without any noticeable impact to voters.

As predicted, the central registry server also received some unwelcome attention from attackers, but the combination of Cloudflare’s content delivery network (CDN) and DDoS protection meant that the server’s cache hit ratio – this is a measurement of how many requests a cache is capable of fulfilling versus how many it receives – which for normal traffic should be about 20%, peaked at 80%, well short of becoming overloaded.

When the curtain came down on an historic second victory for Sweden’s Loreen (a feat only matched by Ireland’s Johnny Logan), the Based.io platform had handled a total of 350 million events and served seven million unique users over three hours. At peak, 1.3 million concurrent users were connected, and the voting landing page served 2.3 million requests per second.

“We did get attacks, but they didn’t get further than the first layers,” says De Beers, who reckons the system saw about 100 million malicious requests. “All the malicious IPs were involved got flagged at the access key level. They only attacked for three minutes and then they moved on to other parts of Eurovision…I was super relieved. If they had increased their efforts from last year to Junior Eurovision, we would have seen an increase in threat.”

“It was a great success,” adds Cloudflare CTO John Graham-Cumming. “I was really happy with it, and we obviously internally kept an eye on their account just to make sure there were no problems happening at critical moments. It all went extremely smoothly.”

Planning for the future

Of course, Eurovision viewers and fans in the UK – and most other participating countries – were not included in the online voting experiment, and millions of votes were tallied via SMS or, in the UK’s case, by calling a dedicated phone number to listen to a recording of Graham Norton.

As happens every year, the phone systems were quickly overwhelmed and many voters had to make multiple attempts to get through. So when will online voting be coming to the UK – and will it be before our next win?

“It’s really annoying, I know…The reason why it’s still on SMS is the fear of cyber security threats, the fear of bad actors, the fear that it will go down. So that’s why we started with the rest of the world,” explains De Beer.

“But I think over the next couple of years, we’re going to see more and more countries where we allow online voting. It’s way easier and a much better user experience.

“We’re patient from our side – we’re just super happy that we could use our new technology that is still in closed beta – to power this event. It’s incredible for us,” he adds.

And the future of online voting at the Eurovision Song Contest looks assured, as De Beer notes; for the two participating countries where the online voting experiment did take place – Australia and Israel – more votes than usual were recorded because there were no phone lines to get blocked.