I had a similar experience, https://developer.apple.com/forums/thread/715488. The way i fixed for myself was not to do anything, let the user manually select on the UnHold and then configure the audio session when performSetHeldCallAction is called.

Scenario 3 I have figured out that if performSetHeldCallAction with onHold is true, does not respond with action fulfill (or action fail) within roughly 5 seconds then performEndCallAction is called. This makes sense when incoming PSTN call needs to be accepted asap. For us as of now the stopping of audio and performing the hold operation does go beyond 5 seconds.