Fancy a little flutter?

Online betting firm has found that person-to-person betting over the Internet is one of few regions of cyberspace...

Online betting firm has found that person-to-person betting over the Internet is one of few regions of cyberspace where real money can be made. But recruiting staff to administer its database during a skills shortage forced it to break some unwritten rules.

Success on the Web has been hit and miss but one area that has come up trumps is online betting.

The sector has been helped by the popularity of the person-to-person betting model, where users take the place of the bookmaker, laying down odds which are then taken up by other users, with the hosting company taking a commission on the winnings.

The approach was pioneered by online start-up The company's director of technical operations Jeff Dunham, says that whilst other sectors continue to suffer, "person-to-person betting is close to a hypergrowth market".

Although a US company, opened its first office in London and the Web site went live in May 2000. However, although the company had been quick to spot an opening and grasp firstmover advantage it hasn't all been plain sailing.

As the number of users grew, the demands placed on the company's IT systems increased accordingly. Dunham says the situation was particularly pressing as the person-to-person model is more demanding than that of a typical business-to-business or business-to-consumer site and creates additional challenges for IT staff.

As a company with an international clientele, operating in multiple times zones, really does have to be a 24x7 operation. This means avoiding downtime and monitoring performance to optimise efficiency, and coping with peaks of activity is crucial.

All Web sites experience peaks of activity but for the situation is exceptional. As Dunham explains, the majority of activity takes place in the five minutes before an event starts. The process also generates high volumes of complex transactions, placing an increased burden on its systems. Even when the event is over, the company's systems are still busy calculating the company's cut and redistributing the betting capital.

Then there is the speed issue. Simply ensuring the site stays up is not enough. "Everything on our site is based on speed," says Dunham. "If a customer sees odds they like they want to move immediately."

Flutter tries to ensure that the time taken to complete a transaction is about two seconds. Up to five seconds is acceptable says Dunham, but beyond that it is not, although he admits that in the past that figure has been as high as 10 to 12 seconds.

"There were a lot of problems that needed to be solved," says Dunham. After the site was opened to the public in May 2000 both the company's systems and the site's Java-based application needed attention. The company had set up its systems and software using the default settings and they were "very buggy". The IT infrastructure came to look increasingly suspect as the volume of traffic on the site increased.

The problems were exacerbated by two key factors. One was the fact that although the main office was in London, the technical office was in San Francisco. Dunham says the initial idea was to be near the epicentre of emerging technology and be "close to the buzz". But as time went on it started to make less of a difference and the benefits became outweighed by the negatives, such as the communication problems between the two offices and the cost considerations. So the company relocated the office to the London site - a process completed in April 2001. "All of a sudden development became a lot easier," says Dunham.

Another problem was the company's IT team was new and relatively unskilled. "A few people were top notch but most were juniors," says Dunham. "What we really needed was some high power talent."

However, although the company felt it could justify a full complement of staff to run the systems administration, it couldn't justify employing a full team of database administrative staff to look after the Oracle 8i databases - even though they formed the core transactional engine on which the whole Java-based application sits.

The company did have two good contract database administrative staff but Dunham says it really needed at least six experienced staff to do the job effectively. The cost would have been prohibitive, however. Then there were the recruitment problems.

"To hire a top-notch Oracle database administrator last year was very, very difficult and, given our cost structure, we couldn't pay what the other companies were offering," says Dunham. The one permanent database administrative guy the company has now, for example, took eight months to find.

So started looking at the outsourcing model. But the story doesn't end there. Dunham says the company was unwilling to simply hand over control of its systems and slap a service level agreement on top. Although it realised it would have to relinquish some control over its systems it still wanted to play an active role. Ideally it wanted to find a third party it could collaborate with.

"What I was really looking for was supplementary horsepower," explains Dunham. "I had a glut of stuff that needed to be done and I wanted to shed some of the maintenance tasks." But a lot of traditional outsourcing companies were too rigid and were unwilling to take on a shared responsibility. "That is where I think the outsourcing model breaks down," he says.

The company came across its eventual partner, Oracle database support and consulting firm Hanston, in the summer of 2000. "It was more flexible," says Dunham, and for flexibility was a key point.

At the time the company was "very fluid, changing things every couple of weeks" and Dunham says "there was a lot of fiddling to make sure things worked". Every time a bug arose the company would have to migrate data and tune and tweak the systems. At the same time the company was making changes to the application itself.

However, although Hanston was more flexible, was still rather hesitant. The company did eventually agree to give Hanston 24x7 remote access to its systems but "it had to pretty much crowbar it out of our hands", laughs Dunham. The company was also keen to make sure that any changes would be well documented and to ensure it could extricate itself from the contract pretty quickly. As such, it built a 60-day get- out clause into the contract.

But when Dunham saw the staff he would be working with he admits he was impressed. And the size of the contract with Hanston was equivalent to the cost of hiring one good database administrator. However, what really sold the company on Hanston was how it reacted one day in November 2000, which Dunham refers to as Black Sunday, when "Oracle went ballistic".

The problem was identified in the morning but Dunham couldn't get hold of any of his database administrative staff until 6pm and his other staff were too junior to deal with it. Hanston, however, was on the case within the hour and was there to collaborate with the Flutter database administrator when he was tracked down.

All in all the company lost close to 10 hours in downtime but without Hanston's hands-on help Dunham says it would have been a lot worse. "The intensity and effort put into solving that problem was what truly clinched it for me," says Dunham. "Hanston wasn't going to quit until it was up again."

But Dunham is quick to point out that this occasion was by no means typical and, from Hanston's point of view, the workload varies enormously from week to week. Dunham says that some weeks it does little more than maintain the health of the database, whereas other weeks Flutter gets "great value for money".

Working together the two built up a sound understanding of the system, addressing key performance issues, and Dunham says that "by about January this year we had a good understanding of the system and by spring we had it down to more of a science". He says the company can now see most problems coming at least four weeks in advance.

Although a lot of the foundation work has been done, Dunham is not willing to rest on his laurels yet and his current goal is to build in operational elegance by putting scripting on top of the databases. However, given time to reflect, he does allow his team and Hanston a little praise. "It's really good now," he says. "I'm really pleased with how it's working."

How strengthened its hand

Benefits of using a third-party collaborative model
  • Cost-effective 24x7 maintenance and support with a fixed annual cost

  • Hanston documents changes, helping continuity of knowledge and skills and educating the in-house database administration team

  • Flexibility helps the company cope with varying workloads

  •'s in-house database administration team is freed up to focus on more important development work

  • The menu-driven service approach means only pays for the services it needs.

Read more on Business applications