Beyond RGB: A new image file format efficiently stores invisible light data

Imagine working with special cameras that capture light your eyes can't even see—ultraviolet rays that cause sunburn or infrared heat signatures that reveal hidden writing. Or perhaps using specialized cameras sensitive enough to distinguish subtle color variations in paint that look just right under specific lighting. Scientists and engineers do this every day—and the resulting data files are so large, they're drowning in it.

A new compression format called Spectral JPEG XL might finally solve this growing problem in scientific visualization and computer graphics. Researchers Alban Fichet and Christoph Peters of Intel Corporation detailed the format in a recent paper published in the Journal of Computer Graphics Techniques (JCGT). It tackles a serious bottleneck for industries working with these specialized images. These spectral files can contain 30, 100, or more data points per pixel, causing file sizes to balloon into multi-gigabyte territory—making them unwieldy to store and analyze.

When we think of digital images, we typically imagine files that store just three colors: red, green, and blue (RGB). This works well for everyday photos, but capturing the true color and behavior of light requires much more detail. Spectral images aim for this higher fidelity by recording light's intensity not just in broad RGB categories, but across dozens or even hundreds of narrow, specific wavelength bands. This detailed information primarily spans the visible spectrum and often extends into near-infrared and near-ultraviolet regions crucial for simulating how materials interact with light accurately.

Unlike standard RGB images with their three channels, these files store information across numerous channels, each representing the intensity of light within a very specific, narrow band of wavelengths. The paper discusses working with spectral images containing 31 distinct channels and even shows examples with as many as 81 spectral bands.

These channels often need to capture a much wider range of brightness values than typical photos. To handle this, spectral images frequently use high-precision formats like 16-bit or 32-bit floating-point numbers for each channel, enabling High Dynamic Range (HDR) data capture. This is a far cry from standard 8-bit images and is key for accurately representing things like the intense brightness of light sources alongside darker scene elements.

Exploring a world beyond RGB

Why would anyone need this level of wavelength detail in an image? There are many reasons. Car manufacturers want to predict exactly how paint will look under different lighting. Scientists use spectral imaging to identify materials by their unique light signatures. And rendering specialists need it to accurately simulate real-world optical effects like dispersion (rainbows from prisms, for example) and fluorescence.

For instance, past Ars Technica coverage has highlighted how astronomers analyzed spectral emission lines from a gamma-ray burst to identify chemicals in the explosion, how physicists reconstructed original colors in pioneering 19th-century photographs, and how multispectral imaging revealed hidden, centuries-old text and annotations on medieval manuscripts like the Voynich Manuscript, sometimes even uncovering the identities of past readers or scribes through faint surface etchings.

The current standard format for storing this kind of data, OpenEXR, wasn't designed with these massive spectral requirements in mind. Even with built-in lossless compression methods like ZIP, the files remain unwieldy for practical work as these methods struggle with the large number of spectral channels.

Spectral JPEG XL utilizes a technique used with human-visible images, a math trick called a discrete cosine transform (DCT), to make these massive files smaller. Instead of storing the exact light intensity at every single wavelength (which creates huge files), it transforms this information into a different form.

Think of it like this: When you look at a rainbow's color transition, you don't need to record every possible wavelength to understand what you see. The DCT works by converting these smooth wavelength patterns into a set of wave-like patterns (frequency coefficients) that, when added together, re-create the original spectral information.

To borrow an analogy from music, it's roughly similar to how MP3 compression works for audio—it selectively preserves frequencies humans most readily perceive, discarding subtle details most listeners won’t miss. Similarly, Spectral JPEG XL preserves the key patterns that define how light interacts with materials, discarding the less significant details.