I figured out why GIFs had a noticeable delay when processing. Issue has been fixed and gifs should process as fast as any other file again.
good news thanks
It was actually kind of funny why. Basically GraphicsMagick was being really slow to "identify" the image. Identify gets information like the size, resolution, format, color, etc. For GIFs it was doing an identify on every frame so it would take normal image identify time * number of frames. I previously changed this with thumbnailing because it tries to do the same thing (makes a thumbnail of every frame) but it never occured to me that identify would have the same problem. Thankfully the GM wrapper I am using makes it easy, I literally just changed the filename to have an array index on it like filename.gif which means to just take the first frame.
Nice work. Good thing it was such a simple fix tbh.