Post

Replies

Boosts

Views

Activity

Reply to DNS Proxy Provider remains active after app uninstall | iOS
Thank you for your response, it was incredibly helpful in organizing my approach to flow lifecycle management. I also wanted to share that the main issue appeared to stem from how the flow was being opened. Initially, I was using: try await flow.open(withLocalFlowEndpoint: flow.localFlowEndpoint) After updating to: try await flow.open(withLocalFlowEndpoint: nil) …the same code began working as expected on iOS versions > 18.4.1. Really appreciate all your help throughout the investigation!
May ’25
Reply to DNS Proxy Provider remains active after app uninstall | iOS
Thank you so much for the clarification! Just to make sure I fully understand: you're suggesting that the correct approach is to continuously loop over readDatagrams() until it either returns an error like NEAppProxyFlowError.aborted or no datagrams. Within each iteration, I should add a new Task (or handle them concurrently) for processing the datagrams, while keeping the overall flow open and tracking task state if needed. Is that correct?
May ’25
Reply to DNS Proxy Provider remains active after app uninstall | iOS
I'm just following up on the issue I previously reported. I've simplified the implementation by removing most of the added logic, all DNS proxy functionality is now contained within a single class. However, I'm still experiencing issues with NEDNSProxyProvider on iOS 18.4.1 and iOS 15.5 Beta. The same code runs without any problems on iOS 18.3.2 and earlier. Please let me know if any additional information would help troubleshoot this further. import NetworkExtension import OSLog private extension DispatchQueue { static let datagramConnection = DispatchQueue(label: "mwe.dns-proxy.datagram-connection") } class DNSProxyProvider: NEDNSProxyProvider { override func startProxy(options:[String: Any]? = nil, completionHandler: @escaping (Error?) -> Void) { Logger.traffic.info("NEDNSProxyProvider started") completionHandler(nil) } override func stopProxy(with reason: NEProviderStopReason, completionHandler: @escaping () -> Void) { Logger.traffic.error("NEDNSProxyProvider stopped with reason: \(reason.rawValue, privacy: .public)") completionHandler() } override func handleNewFlow(_ flow: NEAppProxyFlow) -> Bool { Logger.statistics.info("[NEDNSProxyProvider] - Received \(flow.debugDescription, privacy: .public)") switch flow { case let udpFlow as NEAppProxyUDPFlow: Task { [weak self] in await self?.handleNewUDPFlow(udpFlow) } return true default: return false } } private func handleNewUDPFlow(_ flow: NEAppProxyUDPFlow) async { defer { flow.close() Logger.traffic.info("[UDP] Flow closed") } do { try await flow.open(withLocalFlowEndpoint: flow.localFlowEndpoint) Logger.traffic.info("[UDP] Opened \(flow, privacy: .public)") } catch { Logger.traffic.error("[UDP] Failed to open: \(error.localizedDescription, privacy: .public)") return } let (datagrams, error) = await flow.readDatagrams() if let error = error { Logger.traffic.error("[UDP] Failed to read: \(error.localizedDescription, privacy: .public)") return } guard let datagrams, !datagrams.isEmpty else { Logger.traffic.info("[UDP] No datagrams") return } do { let resolvedDatagrams = try await datagrams.parallelMap(parallelism: datagrams.count) { [weak self] datagram in guard let self = self else { throw NSError(domain: "UDP", code: 1, userInfo: [NSLocalizedDescriptionKey: "Self is nil"]) } let resolvedQuery: Data = await self.forwardDNSQuery(datagram.0) ?? datagram.0.nxdomainData return (resolvedQuery, datagram.1) } try await flow.writeDatagrams(resolvedDatagrams) } catch { Logger.traffic.error("[UDP] Failed to write: \(error.localizedDescription, privacy: .public)") return } } func forwardDNSQuery(_ data: Data) async -> Data? { await withCheckedContinuation { continuation in let connection = NWConnection(host: "1.1.1.3", port: 53, using: .udp) connection.start(queue: .global()) connection.send(content: data, completion: .contentProcessed { error in if error != nil { connection.cancel() continuation.resume(returning: nil) return } connection.receive(minimumIncompleteLength: 1, maximumLength: 512) { content, _, _, _ in connection.cancel() continuation.resume(returning: content) } }) } } } extension NEAppProxyFlow { public func close(_ error: Error? = nil) { self.closeReadWithError(error) self.closeWriteWithError(error) } } extension Collection { func parallelMap<T: Sendable>( parallelism: Int = 2, _ transform: @escaping (Element) async throws -> T ) async rethrows -> [T] { guard !isEmpty else { return [] } let count = count return try await withThrowingTaskGroup(of: (Int, T?).self, returning: [T].self) { group in var buffer: [T?] = [T?](repeating: nil, count: count) var i = self.startIndex var submitted = 0 func submitNext() async throws { guard i != self.endIndex else { return } group.addTask { [submitted, i] in do { let value = try await transform(self[i]) return (submitted, value) } catch { return (submitted, nil) } } submitted += 1 formIndex(after: &i) } for _ in 0 ..< parallelism { try await submitNext() } var completedCount = 0 while let (index, taskResult) = try await group.next() { buffer[index] = taskResult completedCount += 1 try Task.checkCancellation() if completedCount < count { try await submitNext() } } let result = buffer.compactMap { $0 } if result.count != count { throw NSError( domain: "ParallelismError", code: 5, userInfo: [NSLocalizedDescriptionKey: "parallelMap transformation failed due to invalid result count"] ) } return result } } } extension Data { var nxdomainData: Data { let deafultPacket = Data( [0x00, 0x00, 0x81, 0x83, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00] ) guard self.count >= 12 else { return deafultPacket } var response = Data() response.append(self.prefix(2)) response.append(contentsOf: [0x81, 0x83]) response.append(contentsOf: [0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]) if let optStartIndex = self[12...].firstIndex(of: 0x00) { response.append(self[12...optStartIndex + 4]) // Question section } else { response.append(self[12...]) // No OPT record } return response } }
May ’25
Reply to DNS Proxy Provider remains active after app uninstall | iOS
I also sent another email to the Apple Developer Technical Support team requesting an escalation of the issue. Additionally, I attached a ZIP file containing a minimal working example. Just in case, I’ll also duplicate the MWE link here along with the steps to reproduce the problem. MWE GitHub repository I'm really hoping for Apple’s support on this, as the issue is quite critical for my use case.
May ’25
Reply to DNS Proxy Provider remains active after app uninstall | iOS
Thank you for the clarification, and apologies for the confusion in my earlier message. [quote='837751022, DTS Engineer, /thread/772657?answerId=837751022#837751022'] No. I’m not referring to the behaviour of your DNS proxy, but rather the behaviour of the DNS client whose flows it’s proxying. DNS clients [1] generally use one of three models: [/quote] I now understand that you're referring to the DNS client's behaviour, not the proxy’s behaviour. I’ve experimented with scenarios corresponding to both models (1) and (2), but unfortunately, the issue persists regardless of the DNS client's pattern. iOS version discrepancy What's particularly strange is that the exact same DNS Proxy build works correctly on a device running iOS 18.3.2, but consistently experiences connectivity issues on devices running iOS 18.4.1 and newer with the DNS Proxy Provider enabled. Duplicate flow detection I’ve tested for flow duplication by saving NEFilterFlow instances and comparing them using Unmanaged.passUnretained(flow).toOpaque() and ObjectIdentifier(flow). I haven’t observed any pointer duplication, which I assume rules out flow reuse in this context, but please let me know if I’m misunderstanding something. Here’s an example of multiple flows observed with the same sourceAppSigningIdentifier and bytes, but different local ports: UDP com.apple.mobilesafari[{length = 20, bytes = 0xdaae418eeea18e9e9759e2fef96b0c1caf4869fc}] local port 63117 interface en0(bound) UDP com.apple.mobilesafari[{length = 20, bytes = 0xdaae418eeea18e9e9759e2fef96b0c1caf4869fc}] local port 58507 interface en0(bound) UDP com.apple.mobilesafari[{length = 20, bytes = 0xdaae418eeea18e9e9759e2fef96b0c1caf4869fc}] local port 53307 interface en0(bound) UDP com.apple.mobilesafari[{length = 20, bytes = 0xdaae418eeea18e9e9759e2fef96b0c1caf4869fc}] local port 65345 interface en0(bound) UDP com.apple.mobilesafari[{length = 20, bytes = 0xdaae418eeea18e9e9759e2fef96b0c1caf4869fc}] local port 50995 interface en0(bound) It appears the bytes field is what's triggering duplication when I place flows into a Set, even though the actual objects differ. Case escalation I also received an email from Apple Developer Technical Support referencing this thread and recommending I follow up here. I’ll attach a minimal working example (MWE) to my email reply. Is there a way to escalate this case (ID: 13455454)? This issue is very critical, and currently making our main app just unusable, and I’d like to begin exploring possible workarounds for the latest iOS releases as soon as possible
May ’25
Reply to DNS Proxy Provider remains active after app uninstall | iOS
Thank you for your response! If by "one-socket-per-query model" you mean creating a new NWConnection for each forwarded packet versus reusing a single connection for multiple queries, I’ve tried both approaches. Unfortunately, neither made a noticeable difference in behaviour (the issue of device loosing connectivity remains). Based on logs, it appears the flow might not be leaving the device, each domain seems to resolve correctly, and I assume writeDatagrams sends the response back to the flow successfully. Note: I’ve also simplified the MWE to use only DNSProxyProvider and DNSProxyManager, no MDM or configuration profiles, so that I could test on unsupervised devices. On another iPhone (iPhone 14 Pro running iOS 18.3.2), the DNS Proxy works as expected, with no connectivity delays or hangs. [quote='837587022, DTS Engineer, /thread/772657?answerId=837587022#837587022'] Please clarify what you mean by “same flow”: The same NEAppProxyFlow object? Different NEAppProxyFlow objects with the same flow information? [/quote] Yes, I believe I’m seeing the same NEAppProxyFlow object multiple times. To confirm this, I collected the flow instances in a Set, in order to log unique flows received by handleNewFlow. Logs showed multiple duplicates, which led me to suspect that the provider is being asked to handle the exact same flow repeatedly. iOS 18.5 Beta 4 Testing I updated device that was previously on iOS 18.4.1. However, testing shows no improvement, the device still loses connectivity after some time.
Apr ’25
Reply to DNS Proxy Provider remains active after app uninstall | iOS
Thank you for your response! I’d like to clarify a few things based on my latest testing — this does not appear to be a browser-specific issue. When I use ISC Dig (a third-party DNS query tool) to send single queries, they work fine. However, when I requery the same domain more than once, the flow just hangs, showing the same issue. I’ve also shifted from in-app logging to reviewing system logs, and I frequently see this log entry: error 12:27:14.772619-0400 symptomsd COSMCtrl applyPolicyDelta unexpected absence of policy on appRecord dns-proxy.mwe.Domain-Traffic-Utility bg time + grace 2025-04-26 12:32:06.573 -0400 now 2025-04-26 12:27:14.772 -0400 This log is from the MWE I plan to submit to Code Level Support. Another thing I’ve noticed is that NEDNSProxyProvider is receiving the same flow multiple times in handleNewFlow. My understanding is that this method should be triggered only when a new flow arrives — so seeing it called multiple times for what appears to be the same flow seems strange. I'm not sure if this behavior is expected, but it seemed odd to me. I’ve tried several approaches to resolve the issue, but none have worked so far. I will attach some of the code here for handleNewFlow, maybe I am just not seeing something here handleNewFlow handling override func handleNewFlow(_ flow: NEAppProxyFlow) -> Bool { let flowDescription = flow.debugDescription let appIdentifier = flow.metaData.sourceAppSigningIdentifier Logger.statistics.info("[NEDNSProxyProvider] - Received \(flow.debugDescription, privacy: .public)") switch flow { case let udpFlow as NEAppProxyUDPFlow: return handleNewUDPFlow(udpFlow) case is NEAppProxyTCPFlow: Logger.traffic.info("[NEDNSProxyProvider | PassThrough] - Declining TCP flow: \(flowDescription, privacy: .public) from \(appIdentifier, privacy: .public)") return false default: Logger.traffic.info("[NEDNSProxyProvider | PassThrough] - Declining unknown flow type: \(flow, privacy: .public) from \(appIdentifier, privacy: .public)") return false } } private func handleNewUDPFlow(_ flow: NEAppProxyUDPFlow) -> Bool { Task { [weak self] in await self?.handleNewUDPFlow(flow) } return true } private func handleNewUDPFlow(_ flow: NEAppProxyUDPFlow) async { let flowID = ObjectIdentifier(flow) guard handledFlows.insert(flowID).inserted else { Logger.traffic.error("Already handled this flow \(flow.debugDescription, privacy: .public)") return } do { try await flow.open() } catch { let flowOpenErrorMessage = "[NEDNSProxyProvider | UDP] - Did fail to open NEAppProxyUDPFlow \(flow): \(error.localizedDescription)" Logger.traffic.error("\(flowOpenErrorMessage, privacy: .public)") flow.close(error) return } do { let datagrams = try await flow.readDatagrams() let sourceApp = flow.metaData.sourceAppSigningIdentifier let results = await datagrams.parallelMap(parallelism: ProcessInfo.processInfo.activeProcessorCount) { do { let connection = try DatagramConnection($0, sourceApp: sourceApp) return try await connection.transferData() } catch { return Datagram(packet: $0.packet.nxdomainData,endpoint: $0.endpoint) } } try await flow.writeDatagrams(results) flow.close() } catch { let flowWriteErrorMessage = "[NEDNSProxyProvider | UDP] - Did fail to handle NEAppProxyUDPFlow \(flow): \(error.localizedDescription)" Logger.traffic.error("\(flowWriteErrorMessage, privacy: .public)") flow.close(error) return } } NEAppProxyUDPFlow + NEAppProxyFlow extension NEAppProxyUDPFlow { func readDatagrams() async throws -> [Datagram] { try await withCheckedThrowingContinuation { [weak self] (promise: CheckedContinuation<[Datagram], Error>) in guard let self else { promise.resume(throwing: NSError.unknown(thrownBy: Self.self, description: "NEAppProxyUDPFlow.readDatagrams failed with nil reference to self")) return } self.readDatagrams { datagrams, endpoints, error in if let error { promise.resume(throwing: error) } else if let datagrams, let endpoints { let datagrams = zip(datagrams, endpoints).compactMap { try? Datagram.init(packet: $0, endpoint: $1, connectionType: .udp) } promise.resume(returning: datagrams) } else { promise.resume(throwing: NSError.unknown(thrownBy: Self.self, description: "NEAppProxyUDPFlow.readDatagrams received no data and no error")) } } } } func writeDatagrams(_ datagrams: [Datagram]) async throws { try await withCheckedThrowingContinuation { [weak self] (promise: CheckedContinuation<Void, Error>) in guard let self else { promise.resume(throwing: NSError.unknown(thrownBy: Self.self, description: "NEAppProxyUDPFlow.writeDatagrams failed with nil reference to self")) return } let (packets, endpoints) = datagrams.reduce(into: ([Data](), [NWEndpoint]())) { $0.0.append($1.packet) $0.1.append($1.endpoint.eraseToEndpoint()) } self.writeDatagrams(packets, sentBy: endpoints) { error in if let error { promise.resume(throwing: error) } else { promise.resume(returning: ()) } } } } } extension NEAppProxyFlow { public func open(withLocalEndpont localEndpoint: NWHostEndpoint? = nil) async throws { try await withCheckedThrowingContinuation { [weak self] (promise: CheckedContinuation<Void, Error>) in guard let self else { promise.resume(throwing: NSError.unknown(thrownBy: Self.self, description: "NEAppProxyFlow.open failed with nil reference to self")) return } self.open(withLocalEndpoint: localEndpoint) { error in if let error { promise.resume(throwing: error) } else { promise.resume(returning: ()) } } } } }
Apr ’25
Reply to DNS Proxy Provider remains active after app uninstall | iOS
Hi, I wanted to provide an update on the ongoing issue. I've updated the MWE to work with the DNSProxyManager and can confirm that configurations are properly removed after the app is uninstalled. However, the connectivity issue still persists. The following logs are being generated: error 12:27:06.960630-0400 symptomsd COSMCtrl _foregroundAppActivity incoming bundle dns-proxy.mwe.Domain-Traffic-Utility has nil supplied UUID, finds existing E0FDAC00-3DA4-30B7-A0D2-751949DE01C4 error 12:27:14.772619-0400 symptomsd COSMCtrl applyPolicyDelta unexpected absence of policy on appRecord dns-proxy.mwe.Domain-Traffic-Utility bg time + grace 2025-04-26 12:32:06.573 -0400 now 2025-04-26 12:27:14.772 -0400 The general connectivity seems fine, as I'm able to download apps and existing apps can fetch data from the internet without issue. However, when I try using a browser, it stops working. DNS resolution appears to work fine when I use other services to send DNS queries (e.g. ISC Dig, Network Tools). I also tried setting the minimum OS version in the MWE project to iOS 18 and rewrote the NEAppProxyFlow handling logic using the new API, but the result is the same. I'm quite desperate at this point. I understand that it seems like improper flow handling, but I have two devices running different iOS versions. The device running iOS 18 (previous version) works fine, while the one on the latest iOS 18 version has this issue. Plus, in the MWE, I removed all custom filtering and am simply forwarding packets to the system resolver using NWConnection with a local endpoint for UDP flows. There isn't much in the setup that could go wrong. I'd be very grateful for any troubleshooting ideas or suggestions to work around this. Thank you in advance!
Apr ’25
Reply to DNS Proxy Provider remains active after app uninstall | iOS
I also stopped tracking logs for the app itself and logged everything related to the app. And I think I found something interesting Just to keep in mind: after multiple configuration profile reinstalls, I was able to properly start the app. DNS works fine, as I use ISC Dig to send single requests, logs suggest that the flow is handled properly, and the resolution is fairly quick. The moment you attempt to load something in the browser, it hangs for a while and then returns NSURLErrorTimedOut. After that if you attempt to remove the app, extension continues to "run". Here are some of the system logs that I observed: error 15:01:10.972070-0400 symptomsd NSTAT_MSG_TYPE_SRC_REMOVED received reports drop, source ref 5772 source NWStatsUDPSource DNS Proxy Extension attributed app_bundle_id pid 1184 epid 1184 uuid 48916D33-88D0-3B38-94CE-226BBF7D555C euuid 48916D33-88D0-3B38-94CE-226BBF7D555C fuuid 7FE6B8FB-553D-4DD0-B981-2B9D2F1AC283 started 2025-04-25 15:01:09.921 -0400 error 15:02:11.976891-0400 symptomsd COSMCtrl applyPolicyDelta unexpected absence of policy on appRecord app_bundle_id bg time + grace 2025-04-25 15:06:07.069 -0400 now 2025-04-25 15:02:11.973 -0400 I understand that it looks like possible internal crash in the extension, however, app worked totally fine before the update. No issues like this were observed
Apr ’25
Reply to DNS Proxy Provider remains active after app uninstall | iOS
This issue was previously resolved by updating the ProviderIdentifier in the .mobileconfig file to match the app’s bundle identifier instead of the extension’s (i.e., using app_bundle instead of app_bundle.DNS-Proxy-Extension). However, after updating to iOS 18.4.1, the issue has reoccurred As far as I understand, OS matches provided ID in the configuration file with app's bundle and then attempts to start the filter provider. How is it possible for extension to run after app's deletion?
Apr ’25
Reply to Safari with Prevent Cross-Site Tracking enabled bypasses NEDNSProxyProvider
I found that the issue here is that iOS switches to another cache policy when there's no response from the DNS proxy returnCacheDataElseLoad: Use existing cache data, regardless or age or expiration date, loading from originating source only if there is no cached data. In normal cases it uses: useProtocolCachePolicy: Use the caching logic defined in the protocol implementation, if any, for a particular URL load request. For future reference, one of the ways to handle this is verify cache policy for intercepted browser flows within your Content Filter Providers. This way, you could add additional logic for flow filtering or remediation
Topic: Community SubTopic: Apple Developers Tags:
Dec ’24
Reply to Safari with Prevent Cross-Site Tracking enabled bypasses NEDNSProxyProvider
After some testing, it appears that the main issue is related to browser caching. When a connection was previously established, the browser uses its private cache to load the resource on subsequent launches, bypassing the NEDNSProxyProvider Is there a way to handle this behaviour without explicitly blocking socket/browser flows in NEFilterDataProvider? The app is deployed via MDM, so if a solution involves a configuration payload for supervised devices, that would be a suitable option as well. Thank you!
Topic: Community SubTopic: Apple Developers Tags:
Dec ’24
Reply to A server with the specified hostname could not be found exception
It's been a while, but I wanted to give a quick update for anyone who might encounter a similar issue. In my case, the problem had two parts: Parsing DNS Packets in a Custom DNS Proxy Provider If you're using a custom DoH resolver in your DNS Proxy Provider, it's important to parse the entire DNS packet, not just the requested domain. I recommend using the wire format with the application/dns-message MIME type and sending the complete packet to your provider for resolution. Managing NWConnection Lifecycles When using NWConnection for handling connections with local or remote resolvers, ensure proper lifecycle management for each instance of NWConnection. Use stateUpdateHandler to monitor the connection's state and release connections appropriately after they are completed. A common issues is forgetting to release resources for completed connections, leading to memory leaks that can be a bit hard to detect. Some resources suggest setting stateUpdateHandler = nil and then calling .cancel() on the connection. However, simply calling .cancel() is quite enough, as it automatically releases all associated blocks and handlers. Here's a snippet from the documentation to clarify: /// Cancel the connection and release all associated handlers. /// /// Cancel is asynchronous. The last callback will be to the `stateUpdateHandler` with the `.cancelled` state. /// After that, all handlers are released to break retain cycles. /// Subsequent calls to `cancel()` are ignored. final public func cancel() In my case, these two relatively small issues were causing significant and inconsistent issues within the app. Thanks to Quinn for all the help!
Nov ’24