Running into the same issue with some 2D imaging preview extension I am working on. The image file size is barely 8MB and without much compression. Meanwhile some other larger files work. It seems the drawing part could be taking a lot of memory.
The imaging files are not getting smaller as time goes by (in general). When was this 100MB limit set? several Apple hardware generations ago?
I know we have a limit in Apple's design and I am not here to push it. I wish to see if there are improvements in core graphics area that can make the "draw" process taking much less memory, and also a graceful exit code, with a friendly message on the non-preview page so the user knows the extension has failed due to 100MB limit.
The current behavior is a crash on the extension, and a blank screen, neither the developer nor the user knows what is going on, unless you have the debug console hooked up. And do I need to remind you how hard it is to debug the extensions already.