DTLS Handshake Fails When App Is in Background – Is This an iOS Limitation?

Hello, We are facing an issue with performing a DTLS handshake when our iOS application is in the background. Our app (Vocera Collaboration Suite – VCS) uses secure DTLS-encrypted communication for incoming VoIP calls. Problem Summary: When the app is in the background and a VoIP PushKit notification arrives, we attempt to establish a DTLS handshake over our existing socket. However, the handshake consistently fails unless the app is already in the foreground. Once the app is foregrounded, the same DTLS handshake logic succeeds immediately. Key Questions: Is performing a DTLS handshake while the app is in the background technically supported by iOS? Or is this an OS-level limitation by design? If not supported, what is the Apple-recommended alternative to establish secure DTLS communication for VoIP flows without bringing the app to the foreground? Any guidance or clarification from Apple engineers or anyone who has solved a similar problem would be greatly appreciated. Thank you.

Answered by DTS Engineer in 867773022

The general rule for networking in the background on iOS is that everything works as long as your process is running. Once your process gets suspended, you start running into edge cases.

TN2277 Networking and Multitasking talks about this idea, although it’s somewhat out of date. Specifically, I now talk about connections being defuncted rather than using the older socket resource reclaim terminology, because Network framework supports a user-space networking stack that doesn’t involve sockets.

However, it sounds like you’re using BSD Sockets directly, in which case TN2277 is still as relevant as it ever was. Specifically:

we attempt to establish a DTLS handshake over our existing socket

It’s likely that this socket was defuncted when your app was suspended in the background, and thus the issue isn’t with DTLS per se, but rather with networking proper.

If you open a network connection in this situation, does that work?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

The general rule for networking in the background on iOS is that everything works as long as your process is running. Once your process gets suspended, you start running into edge cases.

TN2277 Networking and Multitasking talks about this idea, although it’s somewhat out of date. Specifically, I now talk about connections being defuncted rather than using the older socket resource reclaim terminology, because Network framework supports a user-space networking stack that doesn’t involve sockets.

However, it sounds like you’re using BSD Sockets directly, in which case TN2277 is still as relevant as it ever was. Specifically:

we attempt to establish a DTLS handshake over our existing socket

It’s likely that this socket was defuncted when your app was suspended in the background, and thus the issue isn’t with DTLS per se, but rather with networking proper.

If you open a network connection in this situation, does that work?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Problem Summary: When the app is in the background and a VoIP PushKit notification arrives, we attempt to establish a DTLS handshake over our existing socket.

Adding one thing to what Quinn suggested, the first thing I would actually look at here is making sure you understand exactly what actually "happened" when this failed.

In particular, one issue I've seen multiple times is assuming that an issue like this was caused by a network/API problem, when what actually happened was simply that your app suspended before it actually "did" any useful work. This happens in VoIP apps because the age (introduced in iOS 4) of the VoIP background category means that it's tended to use much longer wake times (~30s) than it really "should".

Putting that in concrete terms, the system has three different “VoIP Push" systems (PKPushRegistry, NEAppPushManager, PTChannelManager) which use a very similar architecture, but which actually have different app suspension behavior. That difference shouldn't matter because we've "always" told apps that they should not rely on the system keeping them awake, but PKPushRegistry wake time has always been long enough that it was easy to ignore that issue.

In any case, the solution here is the same for all cases— call UIApplication.beginBackgroundTask(withName:...) in the delegate callback and end that task once other activity (CallKit call start, AudioSession activate, etc.) takes over and/or you've "finished" whatever you want to do. That guarantees your app ~30s of background run time, which is more than enough time to handle whatever you need to, regardless of what our APIs do.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Thank you, Quinn and Kevin, for the detailed inputs. To clarify what we’ve already tested:

  1. Socket Defuncting (Quinn’s point)

We checked whether the UDP socket becomes defunct when the app goes to the background. The same socket can still send packets after the PushKit wake. Outbound DTLS handshake packets are sent successfully. However, no inbound handshake responses are received until the app moves to the foreground. We also tried opening a new socket during PushKit wake and observed the same result. So the issue doesn’t appear to be the socket itself, but rather background network delivery behavior.

  1. App Suspension Timing (Kevin’s point)

To avoid premature suspension, we call: beginBackgroundTask(withName:) immediately inside the PushKit callback. Background task remains active (~30 seconds). DTLS handshake attempts run within this window. App is not being suspended before the handshake runs. Yet the handshake only completes once the app is foregrounded. This suggests the runtime is available, but inbound DTLS packets are not delivered during background state.

  1. Additional tests we tried

Partial handshake before background Keeping DTLS/RTP engine alive briefly Creating a fresh socket Forcing handshake from a secondary thread Service Extension path (not viable—no DTLS + port sharing not allowed) All lead to the same outcome: DTLS handshake only succeeds once the app is foregrounded.

The only DTLS APIs on iOS are those in Network framework. But you’re not using that, right? Rather, you’re using BSD Sockets?

Presuming that, there are at least two layers between the system delivering a UDP packet to the socket buffer and you receiving a DTLS message. Specifically:

  • You have to implement some sort of mechanism to learn about data arriving on the socket. You can do this using select, kqueues, Dispatch sources, and so on.
  • You have to implement DTLS on top of UDP.

A failure at either of these levels could result in the problem you’re seeing. Given that, my advice is that you try to set up a test case that eliminates them.

First, use an RVI packet trace to ensure that the packet was actually delivered to the iOS device.

IMPORTANT It’s possible that this RVI setup might mask the issue. If that’s the case, lemme know, because it would be an interesting data point in and of itself.

Next, get DTLS out of the picture. Just for testing purposes, set up a simple UDP echo server and ping it on the VoIP wake. Presumably you’ll see both the outbound request and the inbound response packets. You can confirm that with RVI. Can you read that inbound response packet from the socket?

Don’t use your BSD Sockets abstraction layer here. Rather, implement this test code directly on top of sockets, using a blocking receive from a dedicated threat. That isolates your code from any issues that might be being caused by the data read notification mechanism in your real code.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

DTLS Handshake Fails When App Is in Background – Is This an iOS Limitation?
 
 
Q