Why do we use databases? Because they’re there, of course.
Obviously, that isn’t the whole story – but it's true that databases are everywhere.
They tend to proliferate, and the speed of proliferation has increased dramatically since the Web got going. Nonetheless, many developers of Web-based applications ignore this rule: "The Database Must Be Kept Separate From The Application". Is that actually a problem? And indeed, why do we spend time and effort separating the database from the application?
Embarking on a Cook’s Tour of computer and database development might shed some light on basic database concepts you should keep in mind.
Let's start with some history. People argue endlessly over the identity of the first computer. But whatever it was, it was definitely created by a Brit. The heavyweight contender is the Difference Engine tabulating machine designed in the 1800s by Charles Babbage (born in December 1791 in England – there’s some dispute over exactly where).
There’s also the Colossus system designed during World War II by Tommy Flowers (born 22 December 1905, London) to decipher encrypted German messages: this is considered the first programmable electronic computer.
Then there’s EDSAC, the first stored-program computer, completed at the University of Cambridge in 1949. Its design was expanded upon in 1951 by J. Lyons & Co. (of teashop fame) to produce LEO 1, purported to be the first computer used for a commercial business application. (Pay no attention to those upstart Yanks and their claims that ENIAC was the first general-purpose, all-electronic computer.)
The advent of database concepts and DBMS software
The applications that ran on these early machines mixed the data, data processing and application into a single program. Initially, that worked pretty well, but people rapidly found it useful to separate the data from the application. For a start, doing so allowed one application to process many different sets of data.
Then people realised that all applications essentially manipulate data in some way – so there was massive duplication of effort going on because all applications were being written with built-in data storage and manipulation capabilities. It was natural, then, to create specialised applications that could hold and manipulate data – what we now know as database engines.
Once you have a database, you need a model of how it’s going to hold the data. Various methods, such as the hierarchical and network database models, were tried with relative success. Then in the mid-1970s, Ted Codd and Chris Date developed the relational model. Relational database management systems (DBMS) came into being, along with a communication mechanism that has become a standard: the SQL language.
Those developments essentially gave us an ‘abstraction layer’ between the application and database. The abstraction layer enables us to do many things. For example, if we use standard SQL, it is theoretically possible to move data between databases – say, from Oracle to Microsoft’s SQL Server to IBM’s DB2 – without the application being aware of any difference.
Admittedly, that scenario rarely works in practice – perhaps only in fairyland, in fact – for the simple reason that very few systems use only standard SQL. Almost all vendor-specific versions of SQL support a range of non-standard add-ons. And even standard SQL may not behave in exactly the same way when run against different DBMS software.
However, portability between databases is still achievable, thanks to the abstraction layer. A well-developed system can keep non-standard SQL separate from standard SQL to make it clear to programmers which parts of the code need attention in order to ensure a clean port.
Not all applications require a database engine
As you'll probably guess, I'm a fan of databases. So you might think I'd tell you that every application must hold its data in a separate database and that anyone who doesn't do so should be punished with a weekend in Watford – but I wouldn’t. In fact, I’ve even written apps without a database. If an application doesn’t use much data, will never need to be ported and has a limited life expectancy, you can just write your data directly to disk. In such cases, using a database is an extra step that you may simply not need.
The message here is not that you must use a database – or else. If there are no benefits to be gained, then don’t use one. But if you understand the benefits that using a database engine can bring, you’re much better placed to make informed decisions about when and when not to employ one.
One final point is that as applications become more complex, the tipping point between using and not using a database will shift. With a simple application, there is clearly less work to do if you don’t use DBMS software. But if the application requires extensive development or is mission-critical, the lack of a database can actually create more work and lead to serious data integrity issues. To avoid possible problems, consider building it with a database engine from the word ‘Go’.
And lest anyone take offense, I have nothing against Watford – I just couldn’t resist the alliteration. I couldn’t hate the place: a great hero of mine, Chris Date, was born there in 1941.
By the way, Ted Codd was also English (born 23 August 1929, Portland Bill). And did I mention that the first programmer was British as well? That would be Ada Lovelace (born 10 December 1815, London), who worked with Charles Babbage on the design of his Analytical Engine, a general-purpose follow-on to the Difference Engine. And then there’s...
Dr. Mark Whitehorn specializes in the areas of data analysis, data modeling and business intelligence (BI). Based in the UK, he works as a consultant for a number of national and international companies and is a mentor with Solid Quality Mentors. In addition, he is a well-recognized commentator on the computer world, publishing articles, white papers and books. Whitehorn also is a senior lecturer in the School of Computing at the University of Dundee, where he teaches the masters course in BI. His academic interests include the application of BI to scientific research.