And is tcpflow.hash alway unique? Means if I use that value to distinguish different tcpflow, will corrupt happens?

flowHash.txt So, each flow that is generated by a client website, that is also claimed by the proxy, will have a specific hash to it that is unique because it conforms to the NSObjectProtocol. Thanks Matt for the reply. Can you help confirm that each flow coming in has an unique flow.hash value by refreshing the same website? I put a trace inside override func handleNewFlow(_ flow: NEAppProxyFlow) -> Bool and printout flow.hash, and it seems unique each print. This is under BigSur 11.4. But I do remember the same trace under Catalina had some flow.hash printed out of the same value. Not sure is it something changed? Thanks in advance for any suggestion. Richard

Thanks for sharing. Do you have any update on the status of r. 70246597, r. 66312716 or r. 67923914? I cannot find any result form https://openradar.appspot.com/ When (beginning from which macOS version) can this uninstall sysExt issue be resolve? I tried Monterey, and it is of the same issue. I see questions like this a lot so I want to clarify one thing: The SystemExtensions framework is intended to be used by a GUI app to load and unload system extensions in response to user actions. It’s not meant to be called from a command-line tool, regardless of how that command-line tool is packaged. Calling it from a command-line tool is unsupported and may yield weird results. I recently filed a bug to get this clarified in the framework’s documentation (r. 70246597). We also have a bug on file to add more options for installing and uninstalling system extensions (r. 66312716), although if you have specific requirements in that space it wouldn’t hurt to file your own bug describing those requirements. Finally, a lot of folks who ask about this are primarily focused on deploying to managed environments, and such environments can do more with system extensions using MDM. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@apple.com" Uninstall is a major hole in the sysex story right now )-: For example, you can currently install a sysex on a managed Mac but not uninstall it. The sysex team is aware of these problems (r. 67923914) but I’ve no info to share as to when they might be resolved. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Multiline BlockQuote

Code Block swift let settings = NETransparentProxyNetworkSettings(tunnelRemoteAddress: "x.x.x.x") settings.includedNetworkRules = [ NENetworkRule(remoteNetwork: NWHostEndpoint(hostname: "0.0.0.0", port: "443"), remotePrefix: 0, localNetwork: nil, localPrefix: 0, protocol:.TCP, direction: .outbound) ] settings.excludedNetworkRules = [ NENetworkRule(remoteNetwork: NWHostEndpoint(hostname: "apple.com", port: "0"), remotePrefix: 0, localNetwork: nil, localPrefix: 0, protocol:.TCP, direction: .outbound) ] And see all *.apple.com traffic go direct as normal, while all port 443 traffic outside of .apple.com went to the proxy. That means no difference between with and without that exclude rule :) Is that right?

But with that configuration, I still set traffic to apple.com 443 as mentioned: 🤪FZ$--hostName: www.apple.com remoteEp: 23.202.173.153:443 appId: com.google.Chrome.helper fzmacappproxy 18:22:57.007704+1000 🤪FZ$--hostName: supportmetrics.apple.com remoteEp: 17.137.160.100:443 appId: com.google.Chrome.helper fzmacappproxy 18:23:47.128264+1000 🤪FZ$--hostName: developer.apple.com remoteEp: 17.253.67.203:443 appId: com.google.Chrome.helper fzmacappproxy 18:23:47.485234+1000

Submitted a TSI for this (Follow-up: 784462640) Thanks and regards Richard

started.txt Thanks Matt for the suggestion. Tried some lsof traces with below command when started and when network failed after several hours. sudo lsof -p $(pgrep fzmacappproxy) | wc -l sudo lsof -p $(pgrep fzmacappproxy) -iTCP -iUDP -n -P | wc -l sudo lsof -p $(pgrep fzmacappproxy) sudo lsof -p $(pgrep fzmacappproxy) -iTCP -iUDP -n -P So far, didn't see too much difference. failed.txt

Minimised it to only passing through udp packets, still see the same issue. Seen from the lsof log, still no too much clue threre. started.txt failed.txt

Thanks Matt for the suggestion. Didn't see any ECONNREFUSED or ENOPROTOOPT as searched from the console log. Will double confirm anyway. Regards Richard

Double confirmed that didn't find the ECONNREFUSED or ENOPROTOOPT you mentioned. Some other test cases done: Use only dnsproxy and pipe through all dns requests Same failure after a while Use NWConnection rather than NWUDPSession for appproxy udp pipe though. Same failure after some time. Do without redirect QUIC packets inside appproxy Same failure after some time. Is it a Monterey bug since nothing wrong of this kind found on BigSur?

Fired a TSI: Follow-up: 785557535

bug report as below: It is FB9751370 - macOS. https://feedbackassistant.apple.com/feedback/9751370

Thanks for the feedback. Very interesting. I am trying to trace it down in this regard. Also, I have uploaded in the bug report my appproxyprovider.swift, udp process part. Thanks in advance to take a review. Regards Richard

Hi there, Not sure whether you managed to reproduce the said issue with the shrink down project I shared from bug report channel. Just to update you some new findings from my end: With the same test case: doing dns request with my dns.sh script with only udp pipethrough, but based on Big Sur 11.5.2, I can see the same memory leak: 10 hours results in >300MB memory consumption by the system extension although it begins only with around 20MB. I cannot trace further back as Big Sur 11.5.2 is the last Big Sur installer available from web. I wonder anything ongoing in this regard. Processes: 461 total, 2 running, 459 sleeping, 1737 threads                                   08:20:48 Load Avg: 2.54, 2.06, 1.88 CPU usage: 10.97% user, 5.90% sys, 83.11% idle  SharedLibs: 464M resident, 81M data, 66M linkedit. MemRegions: 64289 total, 4344M resident, 227M private, 2069M shared. PhysMem: 16G used (3092M wired), 47M unused. VM: 2487G vsize, 2321M framework vsize, 0(0) swapins, 0(0) swapouts. Networks: packets: 9547216/12G in, 6532190/610M out. Disks: 2223114/65G read, 2284134/46G written. PID  COMMAND   %CPU TIME   #TH #WQ #POR MEM  PURG CMPR PGRP PPID STATE  BOOSTS  %CPU_ME %CPU_OTHRS UID FAULTS COW   93043 com.familyzo 0.4 14:39.27 21  6  73  318M+ 0B  0B  93043 1  sleeping *0[1]   0.01479 0.00000  0  94013+ 1074

Hi there, Regarding the thread https://developer.apple.com/forums/thread/62310 for profiling system extension, I guess it is the content as quote below. There are some questions as I tried: I have the Info.plist file as below not sure whether it is right. <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> ... <key>MallocStackLogging</key> <string>1</string> <key>MallocStackLoggingNoCompact</key> <string>1</string> </dict> </plist> When I start xcode to attach to the system extension process, it says the process is running as root. So I use sudo /Applications/Xcode.app/Contents/MacOS/Xcode and managed to attach the system extension process, but then it seems not functional properly: dns request cannot pass through anymore (as I pass through all udp process). Anything wrong with that? I tried also sudo open /Applications/Xcode.app but again it cannot attach to system extension process. In case with all that profiling successful, I am not sure whether it can tell where the memory leak happens. Thanks in advance for any suggestion. Regards Richard Thanks! I’ve recently been helping a developer with this in the context of a DTS tech support incident, and I came up with a technique that, while not a full workaround, can help a lot. Here’s the highlights: You can set environment variables in your provider by modifying its Info.plist . For example, adding an entry like this: XPCService EnvironmentVariables FOO bar will set the FOO environment variable to bar . That opens up the possibility of using memory management features enabled by environment variables, for example, those documented in the malloc man page. Of those, MallocStackLoggingNoCompact is the heavy hitter. Once you have MallocStackLoggingNoCompact set, Xcode’s memory graph feature becomes super useful. To wit: Start your provider in the normal way. Attach to it with Xcode (Debug > Attach to Process). In the debug pane, click the memory graph button. You can explore the memory graph interactively with Xcode, but you can also export it to disk (File > Export Memory Graph). The resulting .memgraph file can be passed to a variety of command-line tools for memory analysis, including heap , leaks , and malloc_history (all of which have their own man page). You can learn more about memory graphs in WWDC 2018 Session 416 iOS Memory Deep Dive. Share and Enjoy — Quinn “The Eskimo!” Apple Developer Relations, Developer Technical Support, Core OS/Hardware let myEmail = "eskimo" + "1" + "@apple.com"