Two years ago, IBM released the specification for the Linear Tape File System (LTFS). The standard is open, enables drag-and-drop compatibility with desktop file systems and applications, and works with standard Linear Tape-Open (LTO) tape drives.
It removes the need for off-tape catalogues found in existing tape libraries, usually proprietary, that record the location of files – which tape they are on and where on the tape they are to be found.
With traditional tape standards it is difficult to switch to new backup software without transferring all backed up data to a new catalogue via a long, manual process. This inhibits the ability to select a better product from another supplier, should one emerge.
By contrast, LTFS creates a kind of tape NAS. The LTFS specification defines the format and layout of the tape, and it does this on the tape. The LTFS catalogue or index is written in extensible markup language (XML) and stored along the length of the media, not at one end of the tape on a separate partition from the data. This means the index can be modified as needed, without affecting the data partition, and without having to rewind.
Operating systems and applications can present LTFS metadata in standard hierarchical file system style. This makes accessing LTFS-formatted media similar to accessing files stored on NAS. It means applications and users can copy to and from tape using drag and drop or other standard methods of data movement.
Data media, such as LTO data tape, written using LTFS is interchangeable with other LTFS-formatted systems, making them more supplier-independent.
Key benefits include ease of use and portability of tapes, as well as the ability to use a NAS front end to access LTFS tape as nearline storage.
According to Greg Schulz, founder of and senior analyst at infrastructure firm StorageIO, LTFS streamlines data access, exploits tape's ability to stream large amounts of data, and helps to minimise tape's big disadvantage, namely the way it stores data sequentially, which increases access times.
"With LTFS, when the pieces are there – hardware, software, tools – it becomes easier to make tape behave more like disk, such as rapid positioning to where data can be streamed," Schulz says. "One of the benefits of tape is that once it gets to a location, data can be streamed very quickly on or off the media, assuming the disk or storage system and interfaces are fast enough."
But despite the apparent benefits of LTFS, market acceptance has been slow, except in a few verticals. Schulz attributes this to a lack of awareness and education. "There is mixed adoption out there, depending on who you talk to, likewise mixed perceptions of what LTFS can and can’t do," he says.
"Some positioning of LTFS as an alternative to NAS set expectations too high. Some people also believe you need specialised software, perhaps with extra fees, and that it only works with high-end tape libraries,” he adds.
"So there needs to be more awareness of what LTFS really is, and how value is added with additional software. It also needs a reset of expectation levels, so the LTO/Ultrium consortium and its members should do more to make people aware of their tools and LTFS's ease of use."
LTFS has been adopted most noticeably in the media industry, where the need for it is arguably more pressing. Digital film and video production teams create very large volumes of data, in large files that require high transfer rates, especially as ultra high-definition formats start to come on-stream.
Suppliers such as IBM and Quantum have targeted such users because, for them, LTFS offers a number of key benefits. It can remove one copying stage from the workflow, as data can be copied directly to a tape rather than using it mainly as a backup or archiving medium.
Portable disks are the more usual way to move large volumes of data in the media sector. But tape is much cheaper, at $0.04/GB, compared with spinning disk, at $0.10/GB, according to Quantum.
More on LTFS
- Linear Tape File System needs a boost
- Archiving stays active with LTFS and the cloud
- LTFS tape NAS: What it is and how to build it
- Emerging use cases for LTFS (Linear Tape File System)
- What you need to know about LTO-5 and the Linear Tape File System (LTFS)
- Linear Tape File System (LTFS) products seen as archiving boon
LTFS is around two years old. And from the potential customer's point of view, that is not a long period in which to evaluate an archiving technology – an area where longevity is key. So, there is some uncertainty around LTFS.
For example, Bristol University's senior systems administrator Harvey Ditchfield uses dual Spectra Logic LTO-5 tape libraries with a total (uncompressed) capacity of around 2PB, front-ended by a NexSan E48 disk array.
The tape drives have LTFS capability, but Ditchfield says he is reluctant to deploy it because there remain unanswered questions. In particular, he says he wants to see LTFS have a much longer track record in the field. "Look at ZFS,” he says. “It's great now, but it's taken 10 years to mature."
Ditchfield wants to see how LTFS copes with very large volumes of data, and billions of files. "Most technology dives when you do that – it doesn't cope well. We want to archive lots of 2GB objects, but how does it cope with writing that volume of data?"
He says that whenever he has asked suppliers, they have not provided satisfactory answers.
There are other reasons why acceptance has not been overwhelming, according to StorageIO’s Schulz. "Traditional tape customers are either using it or exploring options, and in some cases waiting for software tools to evolve to be able to leverage it," he says.
Two years is not a long period in which to evaluate an archiving technology, so there is some uncertainty around LTFS
"Others are using it with tape in new and different ways. Not for primary backup, rather for active or long-term archives, as well as to protect disk-based data protection. Tape is also being used in different roles in big data and big bandwidth applications such as video and entertainment, among others."
So what's the future for LTFS? A large proportion of the storage industry is invested in spinning media and in flash, where prices per gigabyte are higher and profit margins potentially greater. Such companies would seem to have little incentive to provide much technical or marketing support for LTFS.
Schulz says: "Will LTFS challenge disk for random access? I guess, in theory, the concept is interesting. However, with the cost of disk declining, why would you want to? Although you could bring the different tools together to improve both."
London-based Shed Media is a producer and distributor of audio-visual content, and so generates large volumes of data. Adam Downey, group head of post-production, said the company creates around 20TB every week, all of which needs to be stored for at least five years.
Shed Media has used the LTO family of archiving technologies over the years, from LTO-3 to LTO-5, and is now moving to LTO-6. However, the problem has been vendor lock-in. "You can only recover the data using the software you create the archive in. We went through four different systems, all needing different software,” says Downey.
This creates extra complexity because restores from the archive can be frequent during the lifetime of a project. The company also used hard drives on-set to transfer data but they frequently created workflow bottlenecks.
More on tape storage
The solution was LTFS, which enables the company to create a long-term archive on set because of its compatibility with PCs and Macs using standard drag-and-drop copying, with no additional expertise required.
"It's the simplicity of the system, you can take the tape anywhere and the catalogue is stored on the tape, which gives us security, especially when you consider our history using LTO-3, 4 and 5. It will be a long time before I have to rethink," Downey says.
Are there any downsides to LTFS? "We've seen none yet though it's still early days," Downey says. "When it first came out there were problems so we waited for the hardware to stabilise. And we've seen no incompatibilities, though we've seen plenty with plenty with LTO technology."