NWConnectionGroup with Both Datagram and Non-datagram streams

The older NWConnectionGroup API doesn’t really model datagrams correctly. In the latest API, NetworkConnection, the modelling makes more sense, where there’s a single subconnection for datagrams.

Are you able to use the new API? If so, I encourage you to try that. But if you have to stick with the old API, most commonly this is because you want to support systems prior to xyzOS 26, lemme know and I’ll dig into that.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

I have migrated my QUIC code to iOS26, the new datagram API works sometimes in debug build. However, when I tried release build, datagram never worked.

The failure in debug mode usually happened in first few test runs (between my iPhone 15 pro and an iPhone simulator on Intel MacBook Pro) after computer restart. In that case, I can see status on both side are ready and successful data sent on one side but no single piece of data received on the other side. At the same time, I can see data streaming through the QUICStream smoothly.

The QUICDatagram API looks simple to use, just send and messages, or did I misunderstand the API?

Differences like this are usually the result of optimisation changes. For example, your app relies an undefined behaviour, the exact behaviour you get depends on the optimisation level, and thus things work in Debug builds but fail in Release builds.

However, as a first step I recommend that you make sure that this isn’t a code signing issue. I explain how to do that in Isolating Code Signing Problems from Build Problems.

Assuming that the problem follows the build configuration (Debug vs Release) rather than the code signing setup (Development vs Distribution) then you need to look at how you’re using the API. The most common problem I see in situations like this is that folks create a connection and fail to maintain a strong reference to that connection. The Release build then optimises the reference away, which causes the connection to close.

The best way to avoid such problems is to use structured concurrency. I have some examples of that in TN3213 Moving from Multipeer Connectivity to Network framework. So, rather than create a connection and then work with it, you call withNetworkConnection(…) and it creates the connection and passes it to your closure. That way the connection is guaranteed to exist for the duration.

Similarly for the listener and, if you’re using one, the browser.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thank you! I should release run first! After several sessions, I noticed that CMVideoFormatDescriptionGetHEVCParameterSetAtIndex in the following snippet returns -12712 only in release build. After the removal of the nonfunctional check, for the first time I can see the streamed video in the release build. Also thank you for telling me TN3213, that is informative.

extension CMSampleBuffer {
  func getHEVCParameterSets() throws -> [HEVCNALUnit]? {
    guard let formatDescription else {
      return nil
    }
    var count: Int = 0
    try ensureSuccess(
      osStatus: CMVideoFormatDescriptionGetHEVCParameterSetAtIndex(
        formatDescription,
        parameterSetIndex: 0,
        parameterSetPointerOut: nil,
        parameterSetSizeOut: nil,
        parameterSetCountOut: &count,
        nalUnitHeaderLengthOut: nil
      )
    )
    let nalus = try Array(0..<count).map {
      try getHEVCParameterSet(at: $0, from: formatDescription)
    }

    // Ensure if the format description can be rebuilt
    let datas = nalus.map { $0.bytes }
    let sizes = datas.map { $0.count }
    let pointer = datas.map {
      $0.withUnsafeBufferPointer { $0 }.baseAddress!
    }
    var formatDescriptionOut: CMFormatDescription?
    try ensureSuccess(
      osStatus: CMVideoFormatDescriptionCreateFromHEVCParameterSets(  // <-- Return error (-12712) here
        allocator: kCFAllocatorDefault,
        parameterSetCount: sizes.count,
        parameterSetPointers: pointer,
        parameterSetSizes: sizes,
        nalUnitHeaderLength: 4,
        extensions: nil,
        formatDescriptionOut: &formatDescriptionOut
      )
    )
    assert(formatDescriptionOut != nil)
    //End of check

    return nalus
  }

  //....
}

Just to be clear, I’m in no way an expert on Core Media, but this looks like a Swift issue and I can certainly help with that.

It’s hard to be 100% sure, because I’m not sure what the type of the bytes property of HEVCNALUnit is, but this looks like undefined behaviour. Specifically, this:

let pointer = datas.map {
  $0.withUnsafeBufferPointer { $0 }.baseAddress!
}

is very suspicious because the closure you pass to withUnsafeBufferPointer(…) escapes its buffer parameter. In general, such buffers are only valid inside the closure itself, as explained in the docs:

The pointer passed as an argument to body is valid only during the execution of withUnsafeBufferPointer(_:). Do not store or return the pointer for later use.

This is exactly the sort of thing that tends to work in Debug builds but fail in Release builds, when the optimiser kicks in.

For a single buffer you can move the work to inside the closure. For example, this code ensures has both the call to memchr and the interpretation of its result is within the closure:

a.withUnsafeBufferPointer { buf in
    let base = buf.baseAddress!
    if let cursor = memchr(base, 44, buf.count) {
        print(UnsafePointer(cursor.assumingMemoryBound(to: UInt8.self)) - base)
        // -> 2
    } else {
        print("not found")
    }
}

However, you’re dealing with an array of values, and that makes things harder. There are a couple of approaches you can take here:

• Manually allocate the memory.
• Engage with Swift’s recently-introduced Span type.

If you can reply back here with the type of the bytes property, I should be able to give you more specific advice.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

It’s better to reply as a reply, rather than in the comments; see Quinn’s Top Ten DevForums Tips for this and other titbits.

You wrote:

The type of bytes is [UInt8].

OK.

What’s your deployment target here?

The reason I ask is that I can think of two ways to approach this problem:

• Manually allocate the memory.
• Engage with Swift’s recently-introduced Span type.

The issue with Span is that it’s really bleeding edge. For example, the span property requires appleOS 26 or later. Thus, if you need to support earlier systems there’s no point looking at Span, and that means you have to copy the properties.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

The target is AppleOS 26 and later. That means I can take a look at Span. I was trying to convert a variable of [[UInt8]] to UnsafePointer<UnsafePointer<UInt8>> so I can pass it to parameterSetPointers of

func CMVideoFormatDescriptionCreateFromHEVCParameterSets(
    allocator: CFAllocator?,
    parameterSetCount: Int,
    parameterSetPointers: UnsafePointer<UnsafePointer<UInt8>>,
    parameterSetSizes: UnsafePointer<Int>,
    nalUnitHeaderLength NALUnitHeaderLength: Int32,
    extensions: CFDictionary?,
    formatDescriptionOut: UnsafeMutablePointer<CMFormatDescription?>
) -> OSStatus

Sorry about that.

It’s better to reply as a reply, rather than in the comments; see Quinn’s Top Ten DevForums Tips for this and other titbits.

I repeat the last reply here so others can see:

The type of bytes is [UInt8]. Thank you for bringing more details. I have to confess I didn't check the docs before use. My purpose was to satisfy the argument type: parameterSetPointers: UnsafePointer<UnsafePointer<UInt8>>.

The target is AppleOS 26 and later.

Oh, hey, that wasn’t the answer I was expecting!

Unfortunately I still don’t think Span will help. You’re dealing with multiple blocks of memory, so you need multiple Span values. There isn’t a good way to manage that on current systems. Specifically, Span is a non-escapable type and there’s no way to put those in an array.

So this works:

let chunk: [UInt8] = …
let chunkSpan = chunk.span

but this fails:

let chunks: [[UInt8]] = …
var chunkSpans: [Span<UInt8>] = []
             // ^ Type 'Span<UInt8>' does not conform to protocol 'Escapable'
for chunk in chunks {
    let chunkSpan = chunk.span
    chunkSpans.append(chunkSpan)
}

Now, it’s possible to build an array-like type that does support non-escapable elements, and indeed the Swift team is currently discussing such things over on Swift Evolution, but:

• There’s nothing you can use right now.
• Doing it yourself is a serious challenge.

The easiest way out of this predicament is to copy the values:

let chunks: [[UInt8]] = …
let chunkDatas = chunks.map { (Data($0) as NSData) }
let chunkPointers = chunkDatas.map { $0.bytes.assumingMemoryBound(to: UInt8.self) }
defer {
    withExtendedLifetime(chunkPointers) { _ in }
}
… work with `chunkPointers`, of type `[UnsafePointer<UInt8>]` …

This works because NSData.bytes is guaranteed to remain valid while the NSData itself is valid, and all the NSData objects are stashed in chunkDatas, which is kept around via the withExtendedLifetime(…).

The problem with this approach it that it ends up copying all the chunk data. If the chunk data is small and this call is made infrequently then those copies don’t matter. If either of those is not true then you have to take a step back and look at where the chunks come from.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Thank you Quinn, copying data works! I have tested.

Sorry for my sloppy answer. The piece of code are in a Swift library that is intended to be run on iOS 26, iPadOS 26 and MacOS 26 and later. But I currently only run them on iPhone/iPad.

Oh, hey, that wasn’t the answer I was expecting!

The chunks are format descriptions of HEVC buffer, more specifically, VPS, SPS and PPS. I searched web and learned the sizes are typically less than 500 bytes. That manipulation runs when encountering I-frame, usually once in a second.

I realized that my intuitions borrowed from C pointer misled me. I should check out manual memory management and your links.

I wish I could give your helps credits again! These teach me a lot. And I am pretty sure right now the original Accepted Answer truly solves my original problem. I will start a new post if I encounter problems. Thank you!

No, thank you. This has given me a great excuse to play with Span (-:

And after not finding a way to achieve this goal using Span, I asked internally as to whether it’s currently possible. It turns out that it’s not.

The driving force behind Span is lifetime annotations. These are still somewhat experimental, so there’s no official docs yet. Rather, you can read about them in swift/docs/ReferenceGuides/LifetimeAnnotation.md.

It seems that we don’t yet have a syntax to express the idea of a collection whose lifetime constraint is determined by the lifetime constraint of its values, and hence there’s no way to create an array of Span values.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"