It is now over 25 years since I wrote my first piece about databases. I started with mainframe databases but found myself being asked to write about Q&A a small flat file database running on a PC. Like many DBA's I was a little skeptical about where this market was going and what it could be used for. Shortly after the Q&A review appeared I was sent a copy of dBase to look at and my attitude changed markedly. Here was something I could not only design a database in but something that would allow me to do all sorts of complex programming.
I showed it to one of the IT people at the publishing house where my day job way and they dismissed it as simply being a gimmick. A year later I showed the PICK, System Creator and the ease with which I could design an application for estimating book reprints. Not only were they equally dismissive but they refused to accept that this was built over a weekend when the production department had been asking for such an application for over a year.
The most positive thing that they showed me was how to use Querymaster on the ICL 2966 that the company owned. Unfortunately, it didn't take long for me to misform a query that caused absolute chaos requiring a reboot - oops. For me, the world had changed but for the IT department, the fact that I was able to cause such chaos just reinforced their resistance to the idea of non DBAs building databases.
In the intervening years, I have seen the role of the DBA seriously diminished. This was partly because of the explosion of the database on the PC, partly as a result of every development vendor telling us that developers could be a DBA and, sadly, partly due to the refusal of many DBAs to embrace change.
Now, as I look back on all of that and start this blog, I find that I have more than a little sympathy for those DBAs and wonder where the next generation will come from. If you are asking yourself why, the answer is simple.
We live in a world where everything is data. Increasingly, that data is being moved into databases for it to be managed. Both server and client operating systems now routinely use databases to hold critical data such as user logins, passwords and other security data. Collaboration products such as SharePoint and Domino, Content Management Systems and Document Management Systems now take large swathes of unstructured data and store them in databases. This provides advanced search utilities as well as a common location to manage versioning of content.
Enterprises are working hard to squeeze as much value as possible out of the data that they own and much of that is being done through the use of Business Intelligence. The target market here is the "Information Worker" who uses desktop tools to create value from increasingly large and complex datasets.
For all of this to work we need to take multiple sources of data and integrate it. The complexity of the integration tools and the need to build proper data integration suites has created a significant market for software and tools vendors.
But at the heart of all of this lies the database. Unless it is designed properly then it won't work well. It needs the logical design to be matched to the physical processes and the hardware on which the database will run. Alongside this effective indexing is absolutely critical if the database is to meet any reasonable performance criteria.
Management of the database environment also means understanding issues such as backup, snapshots and, as Cloud becomes ever more pervasive, latency a key issue when the application and data are not co-located. Always present is the issue of compliance and how to ensure that data is secure and not leaking to the outside world.
Who is going to ensure that all of this works well? The harsh truth is not the desktop support team or the advanced user. Nor will it be the developers in the IT department, they are too busy trying to keep everything else working. The only people with the skills and the experience to ensure that our data driven world continues to function are the DBAs.
So when you find yourself sitting around wondering why you cannot get the data out of your systems quickly or why your BI application runs slower than the old paper system take a good look at your IT department and as yourself "where are the DBAs?" And if you should discover that they exist tell them to stand up and be counted.