How ML-based Data Compression Software Can Cool Down the World

Collectively, data centers are now estimated to have a carbon footprint equivalent to the entire global aviation industry, with a new study characterizing their growing environmental impact as a major concern. Data centers are predicted to consume a fifth of the world's electricity by 2025, making a significant contribution to global pollution. 

Despite notable collective and individual efforts in the green computing space from major players in the network and data center market (including Apple, Google, Facebook, and Amazon), continuing to meet growing demand with growing capacity creates a critical framework; a system operating at such a large scale that small changes inevitably have macroeconomic effects. Data center optimization and efficient network design have become commercially and politically motivated fields of research—but there simply may not be enough margin for adjustment in data center infrastructure to really change the projected outcomes, given the sheer demand and the rate at which it is growing. 

High-quality video takes up a lot of space. One hour of uncompressed 4K footage runs at 741gb, whilst the anticipated equivalent size for 8K video is 7.29 terabytes. Without some form of compression, video would be uneconomical to store and practically impossible to play back on standard consumer devices. Video compression codecs (the word is itself compressed from code/decode) are methods of transcoding these large video files into more manageable forms by discarding information that the viewer—hopefully—won't notice. Traditionally, codecs operate by calculating the difference between set video keyframes instead of presenting every single frame of the video in succession. If the video shows a ten-second shot of a person speaking in front of a grey wall, data will need to be spent on making sure that the moving face is accurately represented, since we are very sensitive to facial information. But less data will be needed to show the unchanging grey wall since it has more limited detail (and, from the point of perceptual video optimization) far less narrative focus. Conventional encoding trades off processing resources against the quality and file size of the encoded video—a factor that assumes greater significance at the global scale of streaming networks such as Netflix, Amazon, and YouTube. Constant Bit Rate (CBR) encoding guarantees a preset quality with minimal initial computing resources. However, it often results either in larger file sizes, which throw higher amounts of data redundantly at scenes that may not need them; or more moderately-sized encodings, which fail to depict complex events and objects (such as high-speed movement or fog) at an acceptable quality. Variable Bit Rate (VBR) encoding solves this problem by examining the full-resolution source material at least once before the encoding pass and assigning more data where the image really needs it. The disadvantage of VBR is that it requires at least double the computing resources to encode, and can cause network problems when streaming because of the unexpected variations in data throughput. The principles behind encoding have changed relatively little since the first digital video codec was proposed for teleconferencing in 1984. And the way that the global multinational streaming companies are addressing the storage and bandwidth crisis is surprisingly conventional, given that the roster includes some of the biggest AI and machine learning players in the world. 

AV1 is reported to offer compression savings somewhere between 20% and 30% and promises to break the internet's dependence on HEVC's proprietary H.265 codec, in spite of having so much legacy core code that it's considered to be susceptible to later patent disputes. AOMedia's wealth of artificial intelligence talent seems to have played a limited role in the development of AV1. In the case of Netflix, machine learning is chiefly used to optimize network distribution by understanding customer and local network node behavior. A great number of connections are dropped or abandoned because streaming providers can't anticipate what the demands from any one customer will be in the next 15 minutes, which can have a cumulatively negative impact on a network the size and scope of Netflix. This does illustrate the extent to which we, as demanding consumers, are part of the problem too. If we were required to book the viewing of a TV episode, online clip or streamed movie by so little as five minutes, we could solve a great deal of the major streaming companies' logistical problems, and ease the network load. With knowledge of the target device environment (i.e. a smart TV or a mobile phone) and enough time to anticipate and enact suitable Content Delivery Network provisioning, many of the current delivery crises could all but disappear. However, since this is an unlikely move in the current consumer environment, it is possible that artificial intelligence could step in to help in a more fundamental way than as an arbiter on busy networks. 

Meanwhile, researchers at the Massachusetts Institute of Technology have proposed a full-fledged encoding system built around a Generative Adversarial Network (GAN), a sub-section of artificial intelligence research which pits two neural networks competitively against each other, and which has gained special traction in image-related fields. The researchers' neural codec architecture (NCode) achieves what they describe as “orders-of-magnitude improvement in image and video compression.” It works in part by offloading some of the decoding via a local, all-purpose database intended to reside on the end-user device, effectively shunting part of the work to the client-side device. Sharing a technological workload with the end user is a principle that blockchain development has brought into recent prominence; but in terms of improving the carbon footprint of video streaming, it's often a false economy. Peer-to-peer torrent streaming only achieves energy efficiency with high seed rates, whilst Bitcoin-derived blockchain schemes (which now include video streaming startups) have been practically designed to burn extra energy, and are doing so at a frightening rate. To a certain extent, even traditional codec development can end up passing the problem of energy consumption downstream. In the case of AV1, Google is the first of the major AOMedia contributors to commit to AV1 hardware acceleration at the user end. Initially this will take place in the browser-based web environment where AV1 will first be deployed, but eventually, the codec will need to be incorporated into consumer devices such as Smart TVs, either through software updates or appliance upgrades. Inevitably it will have its own energy usage signature to contribute, even discounting the landfill issue of last-gen hardware that can't run AV1 content well, or at all.