It looks like this has been resolved in iOS 17.4. Not sure which release fixed it, but I just tested it on my device with iOS 17.4 and am not seeing the stutter anymore.

This still remains an issue