Masterclass: A seminar in storage
Posted on Sep 28, 2024 by FEED Staff
Creating content is one thing, but storing it presents a different challenge altogether. Our expert panel discusses the storage solutions that serve as the backbone of media and entertainment
The Panel
- Eric Tang, Founder, Livepeer Studio
- Robert Szabó-Rowe, Head, engineering and product mgmt, Tata Comms Media
- Sam Peterson, Chief operating officer, Bitcentral
- Jon Finegold, Chief marketing officer, Signiant
Could you talk us through the evolution of storage and data management?
ROBERT SZABÓ-ROWE: The decline of physical storage solutions like tape and vaults began after the 2011 tsunami in Japan, which severely impacted one of the main tape manufacturers. This event was a real wake-up call, pushing broadcasters to shift from traditional tape-based storage to more modern solutions like discs and local storage. Many companies still rely on on-premises storage systems today, but the real game changer has been the emergence of cloud storage.
The cloud offers broadcasters and content companies an attractive alternative because it provides scalability, flexibility and cost efficiency. Instead of buying and maintaining large physical storage systems, they can now store their assets in cloud data centres that allow them to pay only for what they use.
Additionally, cloud solutions enable replication across different regions, which is crucial for safeguarding content and ensuring global accessibility. While some organisations still operate hybrid models, with both on-site and cloud storage, many are moving towards a full cloud strategy. This shift has also pushed other workflows into the cloud, as extracting content from the cloud can sometimes be complex. Overall, the evolution from physical storage to cloud-based solutions has revolutionised the M&E industry, providing more efficient, scalable and cost-effective ways to manage data.
ERIC TANG: Traditionally, storage has been mostly on-premises. While functional, it’s inefficient and has significant drawbacks, such as limited access speed and the need for extensive physical space and maintenance. As digital technology advanced, the industry began transitioning to digital storage solutions. Although these digital systems offered advantages compared to tape-based storage, they still presented challenges such as complexity in management and scalability.
The advent of cloud storage marked a major shift by offering remote, scalable solutions that alleviated many of the physical constraints of earlier methods. However, cloud storage brought its own issues, particularly around high costs of moving files to the cloud and vendor pricing.
Recent innovations in decentralised storage technologies, leveraging distributed networks, have emerged as a game changer. These solutions combine the convenience of cloud storage with substantial cost savings, potentially reducing expenses by 60-70%. This evolution reflects the ongoing goal for more efficient, scalable and cost-effective storage and data management solutions in the M&E industry.
SAM PETERSON: The rise of digital technology marked a significant shift in storage solutions. Firstly, digital tapes and hard drives started to replace traditional physical formats. These storage media solved a lot of the previous problems as they were more accessible, had higher capacity and were durable, but they still had limitations.
The real transformation came with adopting cloud storage, which offers scalable, secure and cost-effective solutions. Cloud-based systems allow for easy access, sharing and management of large media files, vastly improving efficiency and collaboration across the industry. In addition, it also abstracts the physical from the logical storage and can simplify the ways in which we interface with storage.
JON FINEGOLD: What we’re seeing is the majority of media companies using a wide array of storage types including both file and object storage, on-premises and public cloud storage, across locations and more and more are employing multiple cloud providers. Different workflows have different requirements and so maintaining flexibility and agility is key. For example, we see a lot of post-production workflows using high-performance on-premises file storage where quick access is key – and many distribution workflows moving to the cloud where elasticity and scalability are critical. It continues to be a moving target.
During the pandemic, we saw a huge surge in cloud storage being deployed out of necessity and now we’re seeing many workloads being moved back to on-premises to better manage costs. On our platform – which moves petabytes of high-value media every day – we have seen a 134% increase in cloud storage and a 304% increase in the use of on-premises storage. On-premises is actually growing faster than cloud.
Why is cloud΄s impact so important?
ERIC TANG: Prior to cloud technology, managing large volumes of data required significant on-premises infrastructure, including servers, storage devices and maintenance staff. This set-up was not only expensive but also cumbersome, with scalability issues that made it difficult to keep pace with growing data-storage requirements.
Cloud storage changed the status quo by providing a scalable alternative that allowed companies to store and access data without the need for extensive physical infrastructure. It enabled greater operational efficiency and simplicity, as businesses could leverage cloud services to handle large datasets with minimal upfront investment and maintenance costs.
Additionally, cloud solutions improved redundancy and disaster-recovery options, reducing the risk of data loss. This shift not only improved operational efficiency but also allowed for greater flexibility and agility in managing data, which is crucial for the fast-evolving demands of the M&E industry.
ROBERT SZABÓ-ROWE: Cloud storage has had a transformative impact on the M&E industry for several reasons. First and foremost, it is inherently scalable, allowing broadcasters to expand their storage capacity without the limitations of physical infrastructure. This scalability is especially valuable as content sizes increase, and storage needs fluctuate.
Unlike traditional storage systems, where broadcasters had to plan and provision resources well in advance, cloud storage allows them to pay only for what they use. This flexibility enables broadcasters to respond to changing demands in real time, accessing content on-demand without the need for extensive forward planning.
Another significant advantage is the ability to replicate storage solutions across different data centres. This not only enhances the reliability and availability of content but also ensures that broadcasters can quickly recover from potential failures or disasters by accessing their data from multiple locations. Broadcasters are no longer constrained by the geographical limitations of physical storage, so content is more accessible.
Why is a robust storage set-up key for live broadcast scenarios?
SAM PETERSON: In live broadcast scenarios, instant, high-quality broadcast is expected. Therefore, robust storage solutions are critical to ensure reliability and low latency. Real-time data access and seamless integration with broadcasting equipment are essential to avoid disruptions.
Redundant storage set-ups safeguard against data loss and ensure continuity, while high-speed access allows for quick retrieval and editing of content. Such reliability is paramount to maintaining broadcast quality and viewer satisfaction; in the live broadcast industry, there is no room for delay. Therefore, the ability to quickly process and distribute content can significantly impact a broadcaster’s success and credibility.
Additionally, robust storage enables efficient handling of large volumes of data, which is especially important during high-definition broadcasts and multicamera productions. What’s more, a reliable storage system ensures that any technical issues can be swiftly addressed without causing significant delays or loss of content, which again is crucial for upholding the expected low latency and professionalism of live broadcasts.
JON FINEGOLD: We are capturing way more footage today in live broadcasts; the Super Bowl had more than 100 cameras! We’re also capturing in 4K, 8K and UHD, so the amount of data has grown exponentially. That content has value long after the live broadcast, for social media, future broadcasts and other features, so preserving it all and offering accessibility makes it easy for teams to unlock future value of that footage.
It’s a saturated market. Can you highlight one solution that stands out?
JON FINEGOLD: Signiant remains committed to storage independence and our software works with nearly any storage type and vendor. We believe our customers know best how to store their valuable content and use the Signiant Platform to make it easy to find, access and move from any storage to any destination. Open Drives is a partner we have been impressed with due to its high-performance storage and the ability to run software directly on the storage device, offering a strong performance.
ROBERT SZABÓ-ROWE: One standout solution in today’s market is Tata Comms Media’s Media Hub. This is replicated across multiple data-centre regions worldwide, which ensures that content is always available and can be accessed quickly, no matter where you are. This is essential for maintaining consistency and ensuring redundancy.
What sets Media Hub apart is its deep integration with Tata Comms Media’s fibre and IP transmission network. Broadcasters can record directly from this network, which eliminates the need to physically transport content to facilities before pushing it into an archive. Instead, content can be tapped straight from the network into cloud storage, streamlining the entire workflow. Because the cloud storage is media-centric, it optimises content delivery routes, ensuring that there is no contention when it comes to playing out media.
Can the industry match pace with content-hungry audiences?
ERIC TANG: Today’s audiences demand more live and interactive content experiences than ever. As media content increases in scale, video quality and depth of data, storage requirements are expanding rapidly. Agile, high-performance video streaming and storage solutions that can keep up with these demands are a must. Forward-thinking players in the industry are actively responding to this challenge by adopting innovative streaming solutions designed to handle large volumes of content more efficiently and affordably.
These solutions not only offer the scalability needed for growing data demands, but also incorporate cost-effective and environmentally friendly approaches. By favouring a distributed network approach to video transcoding, delivery and storage, a wide range of established and emerging streaming businesses can grow and experiment without taking financial risks.
SAM PETERSON: As audience demand for high-resolution content grows, the industry must invest in high-performance storage solutions if it wants to keep up. This includes leveraging advanced technologies like SSDs (solid-state drive), NVMe (non-volatile memory express) storage and high-bandwidth data-transfer protocols. Furthermore, industry standards and best practices need to be continually updated to keep pace with technological advancements in storage and ensure compatibility across various platforms and devices.
Investing in workforce training to keep up with these rapidly developing technologies will also be vital. Companies need to balance capital expenditure (capex) and operational expenditure (opex) to adapt to any changing demands. For example, expenditure on cloud storage can, in some cases, be classed as opex as it is purchased as needed, which can help balance budgets and remain competitive.
Collaborative efforts with customers to anticipate and plan for future storage needs are crucial for staying ahead in this dynamic landscape. By maintaining a proactive approach and using the right storage tech, the industry is able to match rising audience demands.
What has AI΄s impact been on the storage landscape?
ROBERT SZABÓ-ROWE: AI has brought a new dimension to storage, especially in the area of content categorisation and management. Traditionally, broadcasters have relied on manual tagging to organise their media assets, which can be time-consuming and error-prone. With AI, this process can be automated, allowing broadcasters to tag and categorise vast amounts of content without human intervention. AI can analyse media files at a granular level, identifying content as well as specific elements within each frame, such as moments in a football game or scenes in a movie.
This automated tagging is a game changer because it makes content retrieval much easier and faster. AI can recognise patterns and categorise content with far more accuracy than manual methods, enabling broadcasters to manage their media libraries more effectively. Once content is tagged, AI can also assist in preparing it for various platforms, such as VOD or CDN delivery, without requiring manual intervention. This not only saves time but also reduces the risk of errors.
Furthermore, AI can help optimise storage by analysing usage patterns and predicting future storage needs. By understanding what content is accessed frequently and what is not, AI can reduce costs and improve efficiency.
JON FINEGOLD: With AI in play, we will continue to see more read access to old content so that AI can help identify clips and help media companies monetise their content in new ways. That will require IT to think about storage types differently as AI systems will need to explore the content and the associated metadata to get to its full potential.
What does the future hold for the storage and data management spaces?
SAM PETERSON: The future of storage and data management in M&E will see deeper integration with cloud technologies. Media companies are expected to increasingly migrate their workloads to the cloud to leverage its scalability and accessibility. There will be a growing emphasis on database and asset-management solutions to handle the expanding volume of media content. As remote work becomes more prevalent, cloud-based solutions will play a vital role in enabling seamless collaboration. Advancements in AI and ML will significantly enhance the efficiency of storage solutions, driving the industry towards more innovative and user-centric approaches.
ERIC TANG: The future of storage and data management in the industry is set to be shaped by several key trends, including advancements in AI, the emergence of interactive content formats and the development of new business models driven by these innovations. As AI continues to evolve, it will not only drive the need for more sophisticated and scalable storage solutions, but also influence how data is managed and analysed.
Real-time interactivity is a game changer for video companies that powers huge revenue opportunities but comes with low-latency data- management challenges. Technology needs to move fast to keep up with shifting viewer trends. Storage and data-management strategies will evolve in line with these shifts, and we’ll see more performant, intelligent and specialised storage solutions tailored to the specific technical requirements of emerging use cases.
JON FINEGOLD: We will continue to see a mix of storage types for the foreseeable future with companies using a mix of on-premises and cloud storage and different tiers of storage for different classes of content. The landscape will continue to be complex and tools like the Signiant Platform will become even more important so that users can search across storage types, preview and interact with assets without having to move them and, of course, move them when needed anywhere in the world with speed, reliability and security.
ROBERT SZABÓ-ROWE: The future of storage and data management in the M&E industry is undoubtedly centred around the cloud. As more and more broadcasters move their operations into the cloud, we’re going to see an increasing reliance on cloud-based workflows and processes. The ability to store vast amounts of data in the cloud and then run cloud-based processes to manipulate that data opens up significant new possibilities for broadcasters.
One of the key advantages of cloud storage is the ability to integrate various processes into a single, seamless workflow. Broadcasters can store their content in the cloud, use cloud-based AI tools to manage and categorise that content, and then deliver it to audiences without ever needing to download or move files. This level of integration will become even more important as content production and distribution systems continue to evolve over time.
This feature was first published in the Autumn 2024 issue of FEED.
