The drive for top storage: today’s professional backup options
Selecting the right storage technology still comes down to careful deliberation and considered business strategy
Internet video distribution has long since beaten the technical quality of broadcast TV. The network shifts masses of data faster than most satellite or broadcast channels, and laptops, phones and even modern TVs have lots of processing power and can be updated to handle new codecs. It sounds great, but all this quality means the sheer amount of data involved in producing content can be huge, even if the project is not for a traditional distribution channel.
Until recently, the only way to store that sort of data was on a bank of hard disks in the same building – if not the same box – as the computer doing the work. And throwing commodity hard disks in a RAID enclosure is still a cost-effective approach. Companies like G-Technology offer devices designed to do that off the shelf. For under £600, a G-RAID II storage system provides 12TB of storage and can achieve over 400MB/s while still remaining portable enough to take on set.
Choosing the right backup option can be challenging and sometimes there’s no simple choice. Camera cards are too expensive to keep on the shelf for months, offloading can be a pressured rush that can’t delay the shoot and shoots may happen miles from a fast network connection, making online storage unreliable. And no matter what the storage medium, reliability is crucial. The first instinct is to specify a RAID, but it’s not quite that simple. A RAID of many disks might survive the failure of any one, but it’s still one RAID, with single points of failure. It will also carefully preserve human errors. RAID is not backup.
LTO tape, on the other hand, is backup. Two copies on LTO and one on disk will make most insurance companies happy. LTO drives come in internal or external formats, mainly for on-premises backup of bulk data such as video files. The latest LTO-8 drives, made by Quantum, HP and IBM, achieve 360MB/s onto 12TB tapes and sell between £4000 and £6000, depending on the configuration. LTO-8 media is probably unavailable until mid-2019, given a legal disagreement between two manufacturers, but 6TB LTO-7 tapes can be found for around £50 apiece.
Running hot and cold
Adam Bushnaq is business development manager at Zoolz, a provider of cloud backup for small and medium businesses. He says there are huge advantages to online and cloud storage, explaining: “You can upload data from any device. You can access it from anywhere. You can share it with anyone. You can have passwords with an expiry date. If you are a broadcaster and you want to get opinions before you post, you can share it.”
The cloud brings sophistication. Zoolz’ AI-based BigMIND service provides intelligent search and content recognition for both personal and business customers, and there are two ways to store arbitrary data: hot storage and cold storage.
“We have cold storage,” Bushnaq explains, “which is cost-efficient storage, and we have hot storage. The main difference is restoration time. On cold storage, you have to wait three to five hours for your files to be available. With hot storage, if you request a download, it starts immediately.”
Bushnaq adds that another neat trick is the automatic transcoding of large media files to smaller proxies. “You can store your full-resolution videos, 4K or Full HD, on cold storage to save money. Then, on demand, you can have SD, HD or both.”
Prices for the smart archive solution start at £30 a month for 1TB, with unlimited uploads and downloads and, while a moving-image production could fill that in a single morning, the cloud allows for ever-expanding capacity. It offers security as well. “If anything happens to your device,” Bushnaq affirms, “you’ll find a backup of this file safe in the cloud.”
Nearline made simple
Those with an existing infrastructure might not want to do a wholesale cloud migration, but instead incorporate cloud features into what they already have. Avid recently unveiled Cloudspaces, an extension to its Nexis on-premises storage system, at its Connect event to coincide with the NAB Show in Las Vegas.
David Colantuoni, Avid’s vice-president of product management, describes Cloudspaces as a simplifed, demystified approach to nearline storage. “It’s the same thing as a workspace on a Nexis,” he says. “You can move your media to the cloud. Why would you want to do that? Well, often your storage is defined by the amount you’ve purchased. You have a fixed capacity. Sometimes you don’t have the capacity to buy more storage, or you need to temporarily park media while you finish a project. Or maybe it’s for safekeeping.
“The workflows aren’t different. The storage is just located somewhere else. You can have an editor that’s in a different area from where the content’s been shot.”
As ever, performance relies on a speedy internet connection, and Colantuoni finds that connectivity problems are becoming a thing of the past. “I don’t hear that concern as much anymore. There are limited bandwidth capabilities in some places, but transferring data is something you can do in the background while you’re working,” he says.
Cloudspaces is implemented on Microsoft’s Azure infrastructure, satisfying both performance and security concerns. “The facilities are secure,” Colantuoni explains. “Microsoft has a bunch of web pages dedicated to this. They’re serious about it. Cloudspaces goes through a security audit internally at Avid and another one at Microsoft. It’s a humongous concern, but we do take so many steps.”
The set-up process is, he says, “an engagement”, adding: “We have teams that will go look at your workflow: how many editors you want to have, where are the productions going to be located, so we can find the best bandwidth and storage.”
Blackbird ascending
Editors eager to dive deeper into the cloud might consider Blackbird and its Ascent editor. Ascent is a web application, available on most connected devices, intended to assemble and publish simple edits quickly. It handles video stored almost anywhere thanks to Blackbird’s Edge server.
“Say you’ve got some video files in on-premises deployment,” explains Blackbird product manager, Huw Dymond. “In addition, you might have some files or streams that exist within AWS, or Azure or a private data centre. You deploy Edge wherever your content is, users then use our applications through a normalised dashboard that allows them to get access to content wherever it’s located. The thing we always note to our customers is that we’ve built the codec and our apps to run in a standard web browser.”
“The key value is that you don’t have to move your original source to the cloud,” Dymond adds. “If it’s in the cloud, great. If it’s not, Edge gets deployed wherever that media is.”
The company anticipates three main workflows: “The first is that I want to take all my edit decisions – including transitions, stills, animations, video layers – and then conform and render those decisions. That means Edge is responsible for determining which of the resources were used. It does a conform and stitch from each of those resources. The outputs would then go to some on-premises storage, into direct submission to a content management system or social media, but the idea is that the user never has to worry about where the content is.”
The other two approaches involve extracting an edit decision list for use in other NLEs for fine-tuning, or to submit an edit for internal review using the Blackbird Player application.
Blackbird, Dymond says, avoids being an actual provider of storage. “We’ve been pretty strict on our sales to avoid getting into the cloud storage market. We work with the customer’s existing storage infrastructure, or on AWS or Azure.”
Bigger and bigger data
One company involved in that global infrastructure is Cloudian, a company founded on the idea of object storage. Object storage is intended, among other things, to reduce the complexity of managing very large amounts of data. Cloudian is a provider of major cloud-based infrastructure, but its products are also available for on-premises installation. Neil Stobart, vice-president of global sales engineering, notes that use cases vary.
“I’ve done work where my biggest client had the least amount of staff out of all my customers,” Stobart says, describing warehouses filled with unattended servers. “But if you’re in the data creation business you’re probably going to have your own storage when you buy hardware and software from Cloudian. They’ll stand up a big storage cluster in their data centre.”
Cloudian’s storage is compatible with the S3 API, created by Amazon for its AWS service. “That’s why we can emulate Amazon – because we’re using their API.”
Control of where data goes and who accesses it, Stobart explains, is key. “This is one of our unique selling points. We have this concept of policies that we can put on that users’ bucket of data. We provide the storage to the biggest service provider in Europe, a company called GTT Communications, and they have fifteen data centres – fifteen physical locations in countries all over Europe. They have a single cluster that spans all those locations.”
Stobart says it would be theoretically possible to automatically keep backup copies of a file at every one of those locations, though two or three generally satisfies people keen on disaster recovery.
Stobart concludes by returning to the core issue: the limited availability of network bandwidth. “I was just saying this morning that data is getting bigger and bigger much faster than networking is getting quicker. We’re only just scraping 4K. I was at NAB four years ago and they were talking about 8K then. As that data grows, it’s going to become harder and harder,” he says, before adding: “The networking guys have to up their game.”
The bigger the better? The increase in internet video quality means that the sheer amount of data involved in producing content for laptops and phones can be huge
This article originally appeared in the June 2019 issue of FEED magazine.
