Online betting firm Flutter.com has found that person-to-person
betting over the Internet is one of few regions of cyberspace where
real money can be made. But recruiting staff to administer its
database during a skills shortage forced it to break some unwritten
rules.
Success on the Web has been hit and miss but one area that has come
up trumps is online betting.
The sector has been helped by the popularity of the
person-to-person betting model, where users take the place of the
bookmaker, laying down odds which are then taken up by other users,
with the hosting company taking a commission on the winnings.
The approach was pioneered by online start-up Flutter.com. The
company's director of technical operations Jeff Dunham, says that
whilst other sectors continue to suffer, "person-to-person betting
is close to a hypergrowth market".
Although a US company, Flutter.com opened its first office in
London and the Web site went live in May 2000. However, although
the company had been quick to spot an opening and grasp firstmover
advantage it hasn't all been plain sailing.
As the number of users grew, the demands placed on the company's IT
systems increased accordingly. Dunham says the situation was
particularly pressing as the person-to-person model is more
demanding than that of a typical business-to-business or
business-to-consumer site and creates additional challenges for IT
staff.
As a company with an international clientele, operating in multiple
times zones, Flutter.com really does have to be a 24x7 operation.
This means avoiding downtime and monitoring performance to optimise
efficiency, and coping with peaks of activity is crucial.
All Web sites experience peaks of activity but for Flutter.com the
situation is exceptional. As Dunham explains, the majority of
activity takes place in the five minutes before an event starts.
The process also generates high volumes of complex transactions,
placing an increased burden on its systems. Even when the event is
over, the company's systems are still busy calculating the
company's cut and redistributing the betting capital.
Then there is the speed issue. Simply ensuring the site stays up is
not enough. "Everything on our site is based on speed," says
Dunham. "If a customer sees odds they like they want to move
immediately."
Flutter tries to ensure that the time taken to complete a
transaction is about two seconds. Up to five seconds is acceptable
says Dunham, but beyond that it is not, although he admits that in
the past that figure has been as high as 10 to 12 seconds.
"There were a lot of problems that needed to be solved," says
Dunham. After the site was opened to the public in May 2000 both
the company's systems and the site's Java-based application needed
attention. The company had set up its systems and software using
the default settings and they were "very buggy". The IT
infrastructure came to look increasingly suspect as the volume of
traffic on the site increased.
The problems were exacerbated by two key factors. One was the fact
that although the main office was in London, the technical office
was in San Francisco. Dunham says the initial idea was to be near
the epicentre of emerging technology and be "close to the buzz".
But as time went on it started to make less of a difference and the
benefits became outweighed by the negatives, such as the
communication problems between the two offices and the cost
considerations. So the company relocated the office to the London
site - a process completed in April 2001. "All of a sudden
development became a lot easier," says Dunham.
Another problem was the company's IT team was new and relatively
unskilled. "A few people were top notch but most were juniors,"
says Dunham. "What we really needed was some high power talent."
However, although the company felt it could justify a full
complement of staff to run the systems administration, it couldn't
justify employing a full team of database administrative staff to
look after the Oracle 8i databases - even though they formed the
core transactional engine on which the whole Java-based application
sits.
The company did have two good contract database administrative
staff but Dunham says it really needed at least six experienced
staff to do the job effectively. The cost would have been
prohibitive, however. Then there were the recruitment problems.
"To hire a top-notch Oracle database administrator last year was
very, very difficult and, given our cost structure, we couldn't pay
what the other companies were offering," says Dunham. The one
permanent database administrative guy the company has now, for
example, took eight months to find.
So Flutter.com started looking at the outsourcing model. But the
story doesn't end there. Dunham says the company was unwilling to
simply hand over control of its systems and slap a service level
agreement on top. Although it realised it would have to relinquish
some control over its systems it still wanted to play an active
role. Ideally it wanted to find a third party it could collaborate
with.
"What I was really looking for was supplementary horsepower,"
explains Dunham. "I had a glut of stuff that needed to be done and
I wanted to shed some of the maintenance tasks." But a lot of
traditional outsourcing companies were too rigid and were unwilling
to take on a shared responsibility. "That is where I think the
outsourcing model breaks down," he says.
The company came across its eventual partner, Oracle database
support and consulting firm Hanston, in the summer of 2000. "It was
more flexible," says Dunham, and for Flutter.com flexibility was a
key point.
At the time the company was "very fluid, changing things every
couple of weeks" and Dunham says "there was a lot of fiddling to
make sure things worked". Every time a bug arose the company would
have to migrate data and tune and tweak the systems. At the same
time the company was making changes to the application itself.
However, although Hanston was more flexible, Flutter.com was still
rather hesitant. The company did eventually agree to give Hanston
24x7 remote access to its systems but "it had to pretty much
crowbar it out of our hands", laughs Dunham. The company was also
keen to make sure that any changes would be well documented and to
ensure it could extricate itself from the contract pretty quickly.
As such, it built a 60-day get- out clause into the contract.
But when Dunham saw the staff he would be working with he admits he
was impressed. And the size of the contract with Hanston was
equivalent to the cost of hiring one good database administrator.
However, what really sold the company on Hanston was how it reacted
one day in November 2000, which Dunham refers to as Black Sunday,
when "Oracle went ballistic".
The problem was identified in the morning but Dunham couldn't get
hold of any of his database administrative staff until 6pm and his
other staff were too junior to deal with it. Hanston, however, was
on the case within the hour and was there to collaborate with the
Flutter database administrator when he was tracked down.
All in all the company lost close to 10 hours in downtime but
without Hanston's hands-on help Dunham says it would have been a
lot worse. "The intensity and effort put into solving that problem
was what truly clinched it for me," says Dunham. "Hanston wasn't
going to quit until it was up again."
But Dunham is quick to point out that this occasion was by no means
typical and, from Hanston's point of view, the workload varies
enormously from week to week. Dunham says that some weeks it does
little more than maintain the health of the database, whereas other
weeks Flutter gets "great value for money".
Working together the two built up a sound understanding of the
system, addressing key performance issues, and Dunham says that "by
about January this year we had a good understanding of the system
and by spring we had it down to more of a science". He says the
company can now see most problems coming at least four weeks in
advance.
Although a lot of the foundation work has been done, Dunham is not
willing to rest on his laurels yet and his current goal is to build
in operational elegance by putting scripting on top of the
databases. However, given time to reflect, he does allow his team
and Hanston a little praise. "It's really good now," he says. "I'm
really pleased with how it's working."
How Flutter.com strengthened its hand
Benefits of using a third-party collaborative model
- Cost-effective 24x7 maintenance and support with a fixed annual
cost
- Hanston documents changes, helping continuity of knowledge and
skills and educating the in-house database administration team
- Flexibility helps the company cope with varying workloads
- Flutter.com's in-house database administration team is freed up
to focus on more important development work
- The menu-driven service approach means Flutter.com only pays
for the services it needs.