Joined in 2023 
82 
63 About this deal
GDeflate is a novel lossless data compression standard optimized for high-throughput decompression on the GPU with deflate-like compression ratios. GDeflate saves CPU cycles by offloading costly decompression operations to the GPU, while saving system interconnect bandwidth and on-disk footprint at the same time. GDeflate compression is inherently data-parallel, which enables greater scalability across a wide range of GPU architectures. It is designed to provide significant bandwidth amplification when loading from the fastest NVMe devices, supporting both bulk-loading and fine-grained streaming scenarios.
With Moore’s law ending, we can no longer expect to get “free” performance improvements from serial processors. High-performance systems have long embraced large-scale data parallelism to continue scaling performance for many applications. On the other hand, parallelizing the traditional data compression algorithms has been challenging, due to fundamental serial assumptions “baked” into their design.
Be proactive ahead of the 4th of July
In DirectStorage all the data read by a single request is compressed in isolation. Most games will arrange their data such that a single file on disk contains multiple streams. Even if the game wants to use a single file for a texture, that is likely to contain multiple streams. For example, a single texture file may have: In general, various data types, such as texture and geometry, compress at different ratios. This can cause some variation in GPU decompression execution performance.
File compression (and decompression) remains a compute-heavy workload that benefits from parallelism, and here the GPU and its faster memory help greatly. Once the relevant assets are committed to video-memory, the remaining data from the asset containers are purged from video memory to make room for the rest of the game's memory load. Microsoft in its tech demo example showed how a 3D scene's assets were loaded in 0.8 seconds with DirectStorage 1.1, compared to 2.36 seconds without it. This is just a synthetic example, you can imagine the impact on much larger AAA games that take dozens of seconds to load levels, even with NVMe SSDs. The hold-up here is not the storage device, but the CPU trying to decompress relevant assets. What Insomniac was probably trying to convey is they could load whole levels instantly, while in game, without the use of speed bumps. But even then, open world games have basically solved this problem without the need for super fast storage. The technology this game boasts has always confused me. It simply looks like a more advanced version of portals or non-contiguous zone transitions found in older game engines. I know I've seen this sort of shifting in UT2004 and HL2 maps, although a much more primitive version of it based off scripted events or static boundaries. No DirectStorage required.The difference I believe, is how they're filling memory. Older techniques would essentially have two levels in memory at any given time. That way you could switch immediately, by just teleporting the player, at the cost of cutting your maximum level size in half. Insomniac appears to be switching between full-scale 10-12gb levels on a console with only 16gb of memory. Which means they only have one level in memory at a time. Existing fixed-function implementations of DEFLATE can also be easily adapted to support GDeflate for improved compatibility and performance. Two-level parallelism There may be some cases where GPU decompression isn’t desirable. To globally disable GPU decompression, set the DisableGpuDecompression field in DSTORAGE_CONFIGURATION to TRUE. Games may wish to provide this option to users as a performance setting, in case they are running on a system that maybe has a significantly faster CPU than GPU.When released, we will provide an update to the DirectStorage SDK that provides everything developers need to get started with GPU decompression. Included with the SDK will be:
Great Deal 
New Comment