Global file systems: Hybrid cloud and follow-the-sun access

Managing data storage is ever more complex. IT teams have to wrestle with local, direct-attached storage, storage area networks, network attached storage and cloud storage volumes.

They might be deploying hyper-converged systems, or using on-premise implementations of cloud storage technology. And they are likely to have several, incompatible storage protocols at work, especially for unstructured data.

And all of this is happening as the business is demanding more from its data.

This is driving growing interest in global file systems, sometimes also known as distributed file systems.

Global file systems are not new. Back in the 1980s, Carnegie Mellon University developed its Andrew File System, or AFS, which is still in use today. But since then, commercial suppliers have taken the concept further and applied it across cloud and on-premise storage.

Not just object storage

Bridging the gap between on-premise and cloud storage promises to simplify IT management and cut costs.

Cloud storage is, by its nature, distributed. The end-user does not know, or need to know, where their data is stored (apart from any compliance-based limitations). Cloud suppliers use object storage technology to split data across multiple servers and even multiple datacentres, to achieve economies of scale.

But most operating systems and applications cannot read and write to object storage directly. They expect to see network protocols such as NFS or SMB, or access storage directly or via a SAN. Although suppliers have created storage gateways, and more applications are compatible with object storage (such as Oracle and Openstack), there are still plenty of applications that do not.

Global file systems could be the answer. They offer the flexibility, resilience and capacity of the cloud, but retain the simplicity – at least for applications and operating systems – of NAS.

“Most organisations of any size will have data stored in a variety of places and file formats, which can make it very challenging to find and use,” says Bryan Betts, principal analyst at Freeform Dynamics. “Putting a global file system over the top means all your data is equally visible to everyone with access rights, in a single standard format – a virtualised ‘super-filesystem’, if you like.”

Benefits include the economies of scale of the cloud, better redundancy than can usually be achieved from on-premise systems, the ability to add (or remove) storage capacity quickly, and a move from capital spending to operating expenditure.

Other pluses of enterprise file sharing services include easier collaboration and, potentially, better security.

Further down the line, though, firms could even move to a single file system that spans on-premise hardware – where latency and performance are critical – and cloud-based applications.

Common features of global file systems

Global file systems work by combining a central file service – typically on public or private clouds – with local network hardware for caching and to ensure application compatibility. They do this by placing all the storage in a single namespace. This will be the single, “gold” copy of all data.

Caching and synching is needed to ensure performance. According to CTERA, one of the suppliers in the space, a large enterprise could be moving more than 30TB of data per site.

Secondly, the system needs broad compatibility. The global file system needs to support migration from legacy, on-premise, NAS hardware. Operating systems and applications need to be able to access the global file system as easily as they did previously with NFS or SMB.

The system also needs to be easy to use, ideally transparent to end-users, and able to scale. Few firms will be able to move everything to a new file system at once, so a global file system that can grow as applications move to it, is vital.

Global file systems or file and sync?

As a cloud-based service, global file systems appeal to organisations that need to share information between sites – or with users outside the business perimeter in use cases that were often bolstered during the pandemic.

This, however, leads to overlaps between the capabilities of the global file system, and conventional file-and-sync services. These include the more consumer-oriented services such as Dropbox and OneDrive, often pressed into service to support remote working during Covid-19, as well as SharePoint, Google Drive and enterprise-grade sharing services.

Some global file system suppliers stress that they, too, can provide these services. Certainly, being able to share files externally, or extend desktop search out to cloud-based files, is useful. For most enterprises, however, basic performance, compatibility and ease of migration are likely to rank higher.

“The challenges are, of course, that this can get very big, and if your data is globally distributed, you, or your global file system developer, need to decide how you will deal with things like file locking – to prevent two people or systems updating the same data at the same time – and replication,” says Freeform Dynamics’ Betts.