Microsoft details cloud BI, data warehousing

The cloud continues to mature, with Microsoft detailing new hosted business intelligence and data warehousing services.

Microsoft has unveiled the details of its upcoming business intelligence (BI) and data warehouse releases, and the watchwords are self-service and scalability.

Previously codenamed Project Gemini, Microsoft plans to release an Excel add-on called PowerPivot that will give casual users access to more data sources and the ability to report and analyze more raw data in Excel than ever before, according to the company.

The aim is to bring self-service BI capabilities to as many users as possible to simultaneously improve their ability to do reporting, dashboarding and ad hoc analysis on their own and to lighten the support burden on IT, said Herain Oberoi, group project manager for Microsoft's SQL Server group.

PowerPivot users will be able to integrate almost unlimited amounts of data into Excel from spreadsheets, corporate databases and data warehouses from multiple vendors, Oberoi said, as well as third-party data from the Web or in the cloud such as RSS feeds with drag-and-drop tools.

Previously, data analysis in Excel was limited to around 64,000 rows of data, and integrating data from multiple sources often involved manual hand-coding. Excel's new computing power is thanks in part to its in-memory technology that highly compresses data, increasing the amount of data that can be stored and analyzed directly at the desktop level, Oberoi said. "You don't have those traditional performance challenges."

Once created, users will be able to share and publish their analysis – which can be automatically updated as and when changes are made to the source systems that feed them -- with tagging and other Web 2.0 style tools on SharePoint Server. This will also allow IT to view and manage user activity, including which data sources are being tapped and how often, with an operations dashboard.

In addition, the new version of SQL Server, previously codenamed Project Kilimanjaro, will also include master data management (MDM) capabilities built on technology from Stratature, which Microsoft acquired in 2007.

James Kobielus, an analyst with Cambridge, Mass.-based Forrester Research, said with the combination of PowerPivot and SharePoint Server, Microsoft has done a good job of balancing the self-service needs of users to mash-up and analyze data with IT's ability to maintain some type of control over the process.

"It's definitely a very strong self-service capability that I think many of their customers will upgrade to," Kobielus said. He added that Microsoft is now in a position to compete with the likes of IBM Cognos and other long-entrenched BI vendors for the self-service BI market.

In order to use PowerPivot, however, organizations will need to upgrade their current implementations.

"So Microsoft finally gets its in-memory BI capability and no longer plays catch-up to rivals like IBM Cognos (Applix) and QlikTech. However, there's an important caveat," wrote Ovum analyst Madan Sheina on her blog. "The PowerPivot capabilities will only work with the latest Excel 2010 and SharePoint 2010 versions."

Microsoft is also getting into the high-end data warehousing game. The vendor plans to release SQL Server 2008 R2 Parallel Data Warehouse -- previously codenamed Madison -- a massive parallel processing (MPP), columnar-based analytic database that it says can scale to the petabyte level.

That would put Microsoft in competition with Teradata, Oracle and IBM for the largest enterprise data warehouse deployments. The new data warehousing capabilities are based on technology acquired from DATAllegro and will utilize the appliance model, Oberoi said. Customers can choose from commodity hardware from a number of vendors to power the warehouse, including IBM, Dell, and HP.

"This is a very strong addition to their BI strategy," Kobielus said. "Their parallel data warehouse will be a strong alternative to Teradata [and others]." Microsoft recently told analysts that pricing for its new parallel data warehouse will start as low as $US13,000 per usable terabyte.

Jennifer Underwood, a consultant with SharePoint360, which helps clients implement and manage Microsoft deployments, said with the realization of projects Gemini and Madison coming closer, customers of all sizes will soon be able to consider Microsoft a viable BI option.

"Two or three years ago, I didn't see many people taking Microsoft seriously [in the BI realm]," Underwood said.

And with Microsoft's huge Office installed base, the company is anything but short on potential customers. "Excel is still the most popular tool by far that people are using," she said.

Kobielus agreed, saying that many users are comfortable with Excel and don't want to use a separate BI application for reporting and analysis. With PowerPivot, they won't have to. Users can instead analyze data within Excel with tools they're already familiar with, like the Office Fluent user interface and pivot charts, according to Microsoft.

Still, Microsoft doesn't quite have all the analytics pieces in place. According to Kobielus, the vendor is still lacking data mining tools and text analytics capabilities. He said Microsoft "definitely need to do some major acquisitions" in those areas to compete with the likes of SPSS (in the process of being acquired by IBM) and SAS Institute.

Microsoft also has yet to integrate data quality software from its Zoomix acquisition into SQL Server.

Marketing its BI capabilities is another challenge for Microsoft, according to Underwood, as many companies still don't think of the vendor as a top-flight BI provider despite its new capabilities and recent analyst accolades.

Jim Collison, an applications manager with the Gallup Organization, said he has long considered Microsoft "behind in the BI curve" but is interested in at least exploring the vendor's upcoming BI and data warehouse releases.

Gallup workers, who use SharePoint Server along with Oracle's business intelligence suite, nonetheless tap Excel spreadsheets in numerous departments.

"We're trying to catch up with them for the moment and see where they're at," Collison said. "On the surface, [PowerPivot for Excel] sounds like something we may look into."

Beta versions of both PowerPivot and SQL Server 2008 R2 Parallel Data Warehouse are available now, with their full release expected sometime in the first half of next year.

Read more on Database management