Do Not Let Kodak Shelve This

As part of my experimentation with VAE and 16bit per image channels I stumbled upon the fact that VAE was able to normalize to 0-65k per channel instead of 0-255.

What does this mean?

Currently nearly every camera has a proprietary RAW format, and before OpenCV those formats could be hard to edit.

Now we can simply take the RAW and convert it to a 48bit PNG, and edit in true color.

What is the discovery I don't want them to shelve?

What if I told you I could reduce the size of the image from 15-150MB down to 600KB to 1.5MB with minimal loss? You would be correct to assume I was full of it. And in fact I still don't 100% believe it myself. So I keep running test and the results are the same.

SDXL VAE can convert a 16bit image to Latent Space and reconstruct with minor loss.

The attached zip is a simple GUI to demonstrate this.

This matters as 100 Raw Photos average to about 4GB in Raw format and 8GB in 42bit PNG.

What are the downsides?

The initial conversion of a RAW to Latent would require a fair amount of processing power to happen in a few seconds (CPU). But it is feasible for a camera.

The conversion of Latent back to image takes far more processing power and needs to be done by GPU for efficiency.

Biggest use cases

Professional grade cameras like IMAX still use 120mm film do to the limitation of capturing 32bit* per channel color, at 4k this is something around 6GB per second or 100MB per frame.

We could use latent space conversion in real-time with a Blackwell series GPU or possible IPEX CPU/GPU

This could allow 4k 16bit per channel or 32bit per channel images to be stored at a savings of 90-95%.

This space savings could offset the computation cost.

AI Study Use Case

This is the reconstructed difference of the original 11.1 Megabyte 48bit PNG compared to the 612KB reconstruction (11.3 Megabytes reconstructed)

This difference could be key to feature extraction.

*96Bit CMYK has not been studied but you could likely normalize to those values.

About the author: I have 25+ years in 3D/Photography/Arts and now 6k working hours in ML/AI, I do not have a math degree.

I do suspect it is not possible mathematically to compress an image from 11MB to 600kb without losses. However if you compare the reconstruction with a .jpg of similar size the latent is fair superior.