Object storage: An architecture for the petabyte era?

Object storage offers massively scalablility for unstructured data.. But can it break out of the niche it is in?

The object storage market is attracting growing levels of interest in an era of increasing volumes of unstructured data. That’s according to findings from 451 Research, which recently published a report on object storage.

Three factors are driving a groundswell of interest in object storage technologies.

These include the need to store and access massive and growing volumes of increasingly rich data; the limitations of traditional storage architectures at petabyte-levels of scale; and the emergence of web, cloud, and mobile applications and services.

Object storage use cases have evolved from compliance-centric data retention to archiving large quantities of static content and providing storage for distributed access to unstructured data. Additionally, in recent years, object storage has emerged to underpin many cloud and web-based storage services and applications, most notably Amazon S3.

Outside of content-rich verticals and cloud service providers, object storage remains a niche technology in the enterprise datacentre, with relatively little adoption to date.

Object storage obstacles and opportunities

There are many reasons for muted object storage penetration among traditional enterprises. These include lack of native support for traditional enterprise applications; limited understanding of the nature and benefits of object storage versus traditional storage architectures; and the lack of a single industry standard object storage API.

More on object storage

However, over the next two or three years, object storage will be used increasingly by an emerging breed of enterprise devops software developers and cloud storage builders as those organisations move applications and processes to cloud-like deployment models. 

Over the longer term, object storage could play a key role in the emergence of simplified two-tier enterprise storage architectures, especially for web and cloud, and mobile-based applications and services. Here, object storage can potentially replace multiple layers of lower-tier traditional SAN and NAS storage.

451 Research estimates the object storage software and systems market-generated $461m in aggregate revenue for suppliers in 2013, and will grow at a CAGR of 27% through 2018, to reach $1.5bn. By contrast, we estimate the public cloud storage services market (led by Amazon’s S3 object-based storage service) generated around $4bn in revenue in 2013.

The market opportunity for object storage technology suppliers is two-fold: to target service providers that compete with hyperscale cloud giants who have built their own object storage technologies (eg Amazon), and to target enterprises looking to deploy on-prem and hybrid object storage.

Object storage market immature and fragmented

The object storage technology supplier landscape is highly fragmented. There are around a dozen object storage startups and specialists, but no provider has yet “crossed the chasm” and all have object storage revenues under $50m, so there is currently no clear leader by revenue.

EMC and Hitachi Data Systems (HDS) have, by some margin, generated the most revenue and customer traction in object storage

Most object storage specialists are focused on expanding acceptance of the technology by developing support for additional use-cases, protocols, applications and APIs. Almost all are also focused on expanding from “cheap and deep” archiving and into more strategic primary storage use-cases, such as storage for file sync and share, and tier-2 NAS. The next 18-24 months will be a crucial time for all object storage specialists.

Of the major storage suppliers, EMC and Hitachi Data Systems (HDS) have, by some margin, generated the most revenue and customer traction in object storage. EMC has long been the market leader in object storage from a revenue perspective, and continues to evolve its offerings most aggressively – the latest being ViPR. NetApp and HP offer their own on-prem object storage capabilities. IBM and Dell do not, though IBM has it on its roadmap and Dell has a partnership in place.

Open source object storage

Open-source offerings could have a significant bearing on the evolution of the object storage market. Industry momentum to date has mostly centered on OpenStack Swift, though interest in and adoption of Ceph continues to grow (evidenced by Red Hat’s recent acquisition of Inktank; the company formed to build commercial model around Ceph).

IT organisations will increasingly compare the cost and benefits of implementing their own object stores

Most open-source object storage interest comes from the service provider community. Enterprise interest is growing, though real-world deployments are still limited. Swift also enjoys support from industry giants HP and IBM. Open-source offerings lag behind their proprietary peers in terms of enterprise grade functionality, though these gaps could close over time.

Meanwhile, the hyperscale cloud giants are not to be discounted. Amazon, Google and Microsoft have not built their object storage cloud services on commercial object storage offerings, and recent significant price reductions indicate the level of competition. 

IT organisations will increasingly compare the cost and benefits of implementing their own object stores against offerings from hyperscale cloud providers and other cloud service providers.

Simon Robinson is vice president, storage, at 451 Research. For more details of this and other reports from 451 Research, please contact: [email protected]

Read more on SAN, NAS, solid state, RAID