The upgrade to the WOS7000 object storage platform comprises hardware and software changes to the existing WOS6000, as well as added interoperability with DDN’s high-performance (HPC)/clustered NAS platform, GRIDscaler.
WOS Core system software has been upgraded to offer performance and scalability improvements that are claimed to be four times that of the previous version, to 32 trillion objects in potential maximum total capacity close to 1 exabyte.
That kind of capacity is achieved with 8,129 nodes, however. Each WOS7000 system (4U) comprises two nodes, a cluster 128 systems and the total namespace 8,129 nodes. Each WOS7000 system holds up to 240TB and four billion objects of a maximum size of 5TB, minimum 512B.
Key among the enhancements to WOS7000 is the capability for file metadata to be stored with attributes selected by the customer according to policies set. Each file has its own “key data store” that allows metadata to be searched rapidly, said DDN director of WOS product marketing Tom Leyden.
More on object storage
“The customer can search the entire WOS platform in a few minutes, with key data stores being searched in parallel,” he said.
For now, however, the customer will not be able to change attributes set into metadata. So, if the user wants to search on different attributes to those used when the object was written, it will entail rewriting the object.
“Today metadata is immutable, so changing metadata inside a WOS object and re-indexing requires the whole object of be rewritten. The ability to change metadata inside an existing object is a roadmap item,” said Leyden.
WOS Core also sees the debut of WOS Bridge, which is a gateway link between the WOS object storage platform and DDN’s HPC storage/clustered NAS platform, GRIDscaler.
WOS Bridge allows data to be shared from GRIDscaler to WOS and for separate instances of WOS to access and share data, for example among geographically-distributed academic research networks.
“WOS Bridge is a bridge between WOS and GRIDscaler that allows GRIDscaler customers to architect their systems to WOS for collaboration and data distribution,” said Leyden.
Object storage for big data
DDN’s WOS platform is among a number of object storage platforms from the storage giants (except IBM) and smaller storage array makers.
They include EMC’s Atmos; Dell’s Caringo CAstor-powered software appliances; HP's StoreAll; HDS’s Hitachi Content Platform; and NetApp’s StorageGRID (formerly Bycast). Smaller object storage suppliers include Amplidata, Cleversafe, Scality and SGI.
Object storage can scale to very large numbers of files without suffering the potential performance degradation that comes when billions of files are stored in a hierarchical tree-like file system. Instead, with object storage, that file system overhead is removed and access to files is via a flat system of unique identifiers, similar to the DNS system on the internet.
More on clustered NAS
Also, object storage systems tend to do away with Raid data protection, and in place use Reed Solomon error correction coding (as used on audio CDs) that allows data to be reconstructed if some of the original data is lost. This gives faster, more efficient rebuilds than with Raid, which degrades to unfeasibly slow Raid rebuild times at the petabyte-plus level.
With these attributes, object storage is touted as suitable for environments where big files or very large numbers of files are prevalent, such as in broadcasting and media, oil and gas exploration, scientific research, and big data applications. It is often also marketed as a disk alternative to tape archives.
However, object storage has failed to set the world of storage on fire.
Reasons for this include a lack of demand from customers; little or no support from storage industry giants IBM, NetApp and Dell; and a lack of standard object protocol, such as CIFS or NFS.
Every object storage maker has a different file access method and there is no standard way of moving files between file systems and object storage or between object storage systems. Some users have, however, built object storage systems with a file system front end.
Meanwhile, mature clustered NAS products seem to be sweeping up the big data file access needs of customers and appear to be higher on the roadmap agendas of the big storage array makers.