I've done some further research into a possibility of building and linking a custom kext against the kasan kernel. I've put the following CFLAGS into my Xcode project based on the MakeInc.def Apple's own kasan kernels are built with. OTHER_CFLAGS = -DKASAN_TBI=1 -DKASAN_SCALE=4 -fsanitize=kernel-hwaddress -fsanitize-ignorelist=$(SRCROOT)/../../../../xnu/san/memory/kasan-denylist-arm64 -mllvm -hwasan-recover=0 -mllvm -hwasan-mapping-offset=0xf000000000000000 -mllvm -hwasan-instrument-atomics=1 -mllvm -hwasan-instrument-stack=1 -mllvm -hwasan-generate-tags-with-calls=1 -mllvm -hwasan-instrument-with-calls=0 -mllvm -hwasan-use-short-granules=0 -mllvm -hwasan-memory-access-callback-prefix="__asan_" Xcode emitted the following error error: unsupported option '-fsanitize=kernel-hwaddress' for target 'arm64-apple-macos26.0'. According to this post, https://discourse.llvm.org/t/hw-address-sanitizer-on-arm64-apple-darwin/76153/5, on the llvm support forum, the Xcode toolchain doesn't support kernel address sanitisation: Yes, they have it in XNU. But if HWASan is enabled, it’s not compilable. By default, the clang from XCode is used, so it’s Apple’s own. Tried with apple’s fork of llvm-project which is actively maintained but it’s also unable to compile for target arm64-apple-darwin. If want to experiment, here is a useful all-in-one script that does building, all you need to change is the KERNEL_CONFIG to KASAN. Do I understand it correctly that any further attempts to build and link my custom kext with address sanitisation enabled are futile and should be abandoned?

I've managed to boot a KASAN kernel on Apple Silicon running macOS 15.7.4 build 24G517 by following the instructions, https://kernelshaman.blogspot.com/2021/02/building-xnu-for-macos-112-intel-apple.html, you referneced in a reply above. I was also able to build my custom KEXT with these CFLAGS='-fsanitize=address -mllvm -asan-globals=0' as done by the XNU unit tests: ifeq ($(BUILD_ASAN),1) BUILD_SANITIZERS = 1 # compile XNU with asan # TODO: enable globals instrumentation and write a proper ignorelist for problematic global vars XNU_CFLAGS_EXTRA += -fsanitize=address -mllvm -asan-globals=0 # make mocks code aware of sanitizers runtime being linked SANITIZERS_FLAGS += -D__BUILDING_WITH_ASAN__=1 endif # BUILD_ASAN Attempting to build a boot collection with my asan-instrumented KEXT results in a bunch of undefined asan symbols errors emitted. How do I build my KEXT against a KASAN kernel so I can debug the UAFs it has? Thanks.

You're right about the kernel clearing the thread state I set up via those debug registers. One place this happens in is the machine_thread_set_state function where the monitor debug system control register gets cleared and the debug watchpoint control registers and the debug watchpoint value registers get reset. Thanks for looking into this. I won't be filing a bug for this. I've instrumented the memory allocation call sites with diagnostics to output the allocation size and the data type to match that against the info on the memory that got corrupted by a UAF given in the kernel panic log. I'm now endeavouring to manually track down the code paths that lead to the UAFs.

I've been exploring the possibility of debugging UAFs by setting up hardware watchpoints via the CPU debug registers on Apple Silicon. In particular, I've set up the MDE and KDE bits in the MDSCR_EL1 register, enabled the relevant bits in the DBGWCR register to track both load and store operations, and set up the DBGWVR register to track memory addresses of interest. I have a consistent UAF reproducer, and my custom KEXT I've instrumented via inline assembly doesn't seem to detect memory IO after it's been free'd. So I was wondering, if it's down to me not being aware of some other CPU registers that may need setting up or is it because the system enforces some restrictions on 3rd party KEXTs attempting to manipulate those debug registers. Your clarifying this would be greatly appreciated.

Thanks very much for looking into this for me. I'll take a closer look at what I'm doing with the values stored in the RB tree in my filesystem. I might need to resort to interprocess synchronisation to guard the critical sections in that code.

Based on the symbolicated backtraces, there are no references to where the memory is being used after being freed, or where it's being freed. There's just this reference to where it's being allocated: ['atos', '-o', '/Users/user/panic/Kernel_Debug_Kit_15.6_build_24G84.dmg_extracted/KDK.pkg/Payload/System/Library/Kernels/kernel.release.t8103.dSYM/Contents/Resources/DWARF/kernel.release.t8103', '-arch', 'arm64e', '-textExecAddress', '0xfffffe0025c50000', '0xfffffe0025cae7c0', '0xfffffe0026558328', '0xfffffe002655e730', '0xfffffe0025d30f08', '0xfffffe0025cc21c0', '0xfffffe002620afe4', '0xfffffe00243f717c', '0xfffffe00243f7110', '0xfffffe00243df12c', '0xfffffe00243df610', '0xfffffe0024408bc0', '0xfffffe0025ee60cc', '0xfffffe0025ecdf08', '0xfffffe0025eceee4', '0xfffffe002635e9b8', '0xfffffe0025e0fff0', '0xfffffe0025c57b88'] panic_trap_to_debugger (in kernel.release.t8103) (debug.c:1400) kalloc_type_distribute_budget (in kernel.release.t8103) (kalloc.c:1343) kmem_init (in kernel.release.t8103) (vm_kern.c:4584) zone_early_scramble_rr (in kernel.release.t8103) (zalloc.c:3220) kalloc_ext (in kernel.release.t8103) (kalloc.c:2520) mprotect (in kernel.release.t8103) (kern_mman.c:1311) 0xfffffe00243f717c 0xfffffe00243f7110 0xfffffe00243df12c 0xfffffe00243df610 0xfffffe0024408bc0 vn_pathconf (in kernel.release.t8103) (vfs_vnops.c:1875) openat_dprotected_internal (in kernel.release.t8103) (vfs_syscalls.c:5127) mknod (in kernel.release.t8103) (vfs_syscalls.c:5469) sendsig (in kernel.release.t8103) (unix_signal.c:626) sleh_synchronous (in kernel.release.t8103) (sleh.c:1267) fleh_synchronous (in kernel.release.t8103) + 24 The unsymbolicated memory addresses come from my KEXT where memory is allocated via OSMalloc(). The object being allocated ends up being stored in an RB tree. I'd like to take you up on your offer to take a look at the full panic log for me if I may. Please find the full panic log attached to this ticket https://feedbackassistant.apple.com/feedback/21231833.

Hello Kevin, I put together a Python script to do a full symbolication based on your instructions. I'm having trouble computing addresses of functions given in the actual stack frames. First off, the symbolication of the frame of the thread that panicked does succeed as the function addresses come precomputed by the system: % ./symbolicate.py TEXT_EXEC 0xfffffe00072d4000 TEXT 0xfffffe0007004000 ktext_exec_base 0xfffffe002c900000 load_address: 0xfffffe002c630000 panicked_thread_faddrs: ['0xfffffe002c95d93c', '0xfffffe002cacd124', '0xfffffe002cacb31c', '0xfffffe002c903b88', '0xfffffe002c95dc08', '0xfffffe002d264898', '0xfffffe002d2700f8', '0xfffffe002caccf80', '0xfffffe002cacb490', '0xfffffe002c903b88', '0xfffffe002cb7f820', '0xfffffe002cba8538', '0xfffffe002cb8fd00', '0xfffffe002cb905c0', '0xfffffe002d05eb5c', '0xfffffe002cacb3a4', '0xfffffe002c903b88', '0x19d82a2b0'] ['atos', '-o', 'Kernel_Debug_Kit_26_build_25A353.dmg_extracted/KDK.pkg/Payload/System/Library/Kernels/kernel.release.t8103.dSYM/Contents/Resources/DWARF/kernel.release.t8103', '-arch', 'arm64e', '-l', '0xfffffe002c630000', '0xfffffe002c95d93c', '0xfffffe002cacd124', '0xfffffe002cacb31c', '0xfffffe002c903b88', '0xfffffe002c95dc08', '0xfffffe002d264898', '0xfffffe002d2700f8', '0xfffffe002caccf80', '0xfffffe002cacb490', '0xfffffe002c903b88', '0xfffffe002cb7f820', '0xfffffe002cba8538', '0xfffffe002cb8fd00', '0xfffffe002cb905c0', '0xfffffe002d05eb5c', '0xfffffe002cacb3a4', '0xfffffe002c903b88', '0x19d82a2b0'] handle_debugger_trap (in kernel.release.t8103) (debug.c:1863) handle_uncategorized (in kernel.release.t8103) (sleh.c:1818) sleh_synchronous (in kernel.release.t8103) (sleh.c:1698) fleh_synchronous (in kernel.release.t8103) + 24 DebuggerTrapWithState (in kernel.release.t8103) (debug.c:830) Assert (in kernel.release.t8103) (debug.c:841) sleh_synchronous_sp1 (in kernel.release.t8103) (sleh.c:1191) handle_kernel_abort (in kernel.release.t8103) (sleh.c:3960) sleh_synchronous (in kernel.release.t8103) (sleh.c:1698) fleh_synchronous (in kernel.release.t8103) + 24 vn_create (in kernel.release.t8103) (vfs_subr.c:8079) vn_open_auth (in kernel.release.t8103) (vfs_vnops.c:483) open1 (in kernel.release.t8103) (vfs_syscalls.c:0) open_extended (in kernel.release.t8103) (vfs_syscalls.c:5273) unix_syscall (in kernel.release.t8103) (systemcalls.c:181) sleh_synchronous (in kernel.release.t8103) (sleh.c:1484) fleh_synchronous (in kernel.release.t8103) + 24 0x19d82a2b0 Here's the output generated for the very first kernel frame that fails to symbolicate: kernelFrames: tid:110 IOServiceTerminateThread UUID: 8502a040-9cf9-35f5-b8a2-84b0e48d379e [1, 622608] funcaddr: 0xfffffe0008774000+0x98010 -> 0xfffffe000880c010 [1, 617372] funcaddr: 0xfffffe0008774000+0x96b9c -> 0xfffffe000880ab9c [1, 509108] funcaddr: 0xfffffe0008774000+0x7c4b4 -> 0xfffffe00087f04b4 [1, 8496472] funcaddr: 0xfffffe0008774000+0x81a558 -> 0xfffffe0008f8e558 [1, 53004] funcaddr: 0xfffffe0008774000+0xcf0c -> 0xfffffe0008780f0c ['atos', '-o', 'Kernel_Debug_Kit_26_build_25A353.dmg_extracted/KDK.pkg/Payload/System/Library/Kernels/kernel.release.t8103.dSYM/Contents/Resources/DWARF/kernel.release.t8103', '-arch', 'arm64e', '-l', '0xfffffe002c630000', '0xfffffe000880c010', '0xfffffe000880ab9c', '0xfffffe00087f04b4', '0xfffffe0008f8e558', '0xfffffe0008780f0c'] 0xfffffe000880c010 0xfffffe000880ab9c 0xfffffe00087f04b4 0xfffffe0008f8e558 0xfffffe0008780f0c UUID: 8502a040-9cf9-35f5-b8a2-84b0e48d379e is that of the kernel as referenced in the panic log Kernel UUID: 8502A040-9CF9-35F5-B8A2-84B0E48D379E and given in the binaryImages lookup table located at index 1: "binaryImages": [ [ "fbe15ad4-ea36-3c07-81be-460a8240c1d4", 18446741874887413328, "T" ], [ "8502a040-9cf9-35f5-b8a2-84b0e48d379e", 18446741874828328960, "T" ], Function addresses are computed as follows: faddr=binaryImages[loadaddr]+kernelFrames[offset]. Given the diagnostics above [1, 622608] funcaddr: 0xfffffe0008774000+0x98010 -> 0xfffffe000880c010, where 0xfffffe0008774000 is the load address of the kernel 18446741874828328960, and 0x98010 is the offset from the load address 622608, atos(1) fails to perform symbolication. Your second suggestion to use the value of the offset+ the load address of 0 doesn't succeed either. Your clarifying how function addresses given in kernel stack frames are to be computed so they result in a successful symbolication would be greatly appreciated.

Thanks for the link. I was able to build a bootable Kext Collection as instructed in the post you referenced. I was then able to boot into a KASAN instrumented kernel on my Apple Silicon machine. On reproducing a kernel panic via a UAF I got the following symbolication, which I didn't find useful in identifying the source of a UAF in my KEXT. % symbolicateKernelPanicBacktrace.sh ~/2025-09-24-114045.kernel.core.kasan.myfs.uninstrumented.log /System/Volumes/Data/Library/Developer/KDKs/KDK_12.5.1_21G83.kdk/System/Library/Kernels/kernel.kasan.t8101 ASCII text panic(cpu 2 caller 0xfffffe0024926790): KASan: UaF of quarantined object 0xfffffe167506f880 handle_debugger_trap (in kernel.kasan.t8101) (debug.c:1431) kdp_trap (in kernel.kasan.t8101) (kdp_machdep.c:363) sleh_synchronous (in kernel.kasan.t8101) (sleh.c:854) fleh_synchronous (in kernel.kasan.t8101) + 40 DebuggerTrapWithState (in kernel.kasan.t8101) (debug.c:662) panic_trap_to_debugger (in kernel.kasan.t8101) (debug.c:1074) Assert (in kernel.kasan.t8101) (debug.c:688) ubsan_json_init.cold.1 (in kernel.kasan.t8101) (ubsan.c:0) asan.module_ctor (in kernel.kasan.t8101) + 0 kasan_crash_report (in kernel.kasan.t8101) (kasan-report.c:136) kasan_violation (in kernel.kasan.t8101) (kasan-report.c:0) kasan_free_internal (in kernel.kasan.t8101) (kasan-classic.c:815) kasan_free (in kernel.kasan.t8101) (kasan-classic.c:843) kfree_zone (in kernel.kasan.t8101) (kalloc.c:2416) kfree_ext (in kernel.kasan.t8101) (kalloc.c:0) IOFree_internal (in kernel.kasan.t8101) (IOLib.cpp:360) __asan_global_.str.102 (in kernel.kasan.t8101) + 8 nx_netif_na_txsync (in kernel.kasan.t8101) (nx_netif.c:1682) netif_ring_tx_refill (in kernel.kasan.t8101) (nx_netif.c:4025) nx_netif_na_txsync (in kernel.kasan.t8101) (nx_netif.c:1708) netif_transmit (in kernel.kasan.t8101) (nx_netif.c:3770) nx_netif_host_output (in kernel.kasan.t8101) (nx_netif_host.c:0) dlil_output (in kernel.kasan.t8101) (dlil.c:6776) ip_output_list (in kernel.kasan.t8101) (ip_output.c:1626) tcp_ip_output (in kernel.kasan.t8101) (tcp_output.c:0) tcp_output (in kernel.kasan.t8101) (tcp_output.c:2713) tcp_input (in kernel.kasan.t8101) (tcp_input.c:0) ip_proto_dispatch_in (in kernel.kasan.t8101) (ip_input.c:0) ip_input (in kernel.kasan.t8101) (ip_input.c:0) proto_input (in kernel.kasan.t8101) (kpi_protocol.c:0) ether_inet_input (in kernel.kasan.t8101) (ether_inet_pr_module.c:221) dlil_ifproto_input (in kernel.kasan.t8101) (dlil.c:5696) dlil_input_packet_list_common (in kernel.kasan.t8101) (dlil.c:6121) dlil_input_thread_cont (in kernel.kasan.t8101) (dlil.c:3169) Call_continuation (in kernel.kasan.t8101) + 216 I also was able to instrument my KEXT as described in the Pishi project. But the instrument_kext.py Ghidra script ended up garbling the LR register in my KC by overwriting the two most significant bytes of the address: panic(cpu 7 caller 0xfffffe002d0984c0): Kernel data abort. at pc 0xfffffe002d46a9b0, lr 0xc8b2fe002d46a9ac (saved state: 0xfffffe3d227eed70) x0: 0x0000000000447ed0 x1: 0xfffffe0030f6d1c0 x2: 0x0000000000000000 x3: 0xfffffe1017381410 x4: 0x00000000000000fd x5: 0x0000000000000000 x6: 0xfffffe002b81606c x7: 0xfffffe3d227ee980 x8: 0x0000000000000000 x9: 0x64627135a6d70010 x10: 0x0000000000000005 x11: 0xfffffe1014d87e40 x12: 0xfffffe1014d7c000 x13: 0x0000000000000000 x14: 0x0000000000000000 x15: 0x0000000000000008 x16: 0x0000020077cba83c x17: 0xfffffe0030259920 x18: 0x0000000000000000 x19: 0x0000000000000000 x20: 0xfffffe167f157890 x21: 0xfffffe167f15617c x22: 0x00000000e00002bd x23: 0x0000000000000000 x24: 0x000000000014b4dc x25: 0x0000000000000000 x26: 0xfffffe167e137840 x27: 0x0000000000000000 x28: 0x0000000000000000 fp: 0xfffffe3d227ef140 lr: 0xc8b2fe002d46a9ac sp: 0xfffffe3d227ef0c0 pc: 0xfffffe002d46a9b0 cpsr: 0x60401208 esr: 0x96000006 far: 0x0000000000000000 Debugger message: panic Memory ID: 0x6 OS release type: User OS version: 21G83 Kernel version: Darwin Kernel Version 21.6.0: Wed Aug 10 14:28:18 PDT 2022; root:xnu_kasan-8020.141.5~2/KASAN_ARM64_T8101 This has just about exhausted my options in being able to locate the source of UAFs in my KEXT. If you have any other practical advice to offer, it would be greatly appreciated. How do kernel devs at Apple debug UAFs?

Have you tried testing on Intel, either on real hardware or running in a VM? I haven't. I don't have access to an Intel-based Mac at the moment. And I don't have a VM readily available. The following write-up[1] on instrumenting KEXTs claims that being able to load a KASAN kernel doesn't mean that one's own KEXT would get instrumented as well. Is that an accurate statement? [1] https://r00tkitsmm.github.io/fuzzing/2025/04/10/Pishi2.html How reproducible is the issue? One issue is only reproducible when mounting my filesystem on an M4 Max CPU, but not on M1 or M2 machines I tried this on. Another UAF issue is reproducible universally though which manifests itself when attempting to open multiple PDFs at the same time.

Thanks for your feedback and suggestions. I double checked which vnode_t is returned by my vnop_lookup. It is indeed the one that references both the original file and the hardlink. vnop_lookup: cat-1235 vnop_lookup: -> lookuprpc(/hlf.txt) lookuprpc: cat-1235 lookuprpc: lookup successful entry exists lookuprpc: -> cache_lookup(/hlf.txt) lookuprpc: <- cache_lookup(/hlf.txt) -> -1 ;VFS_CACHE_HIT vnop_lookup: <- vp fffffe2a45cf6b60 /hlf.txt The vnop_open call that comes immediately after the previous vnop_lookup call is for the parent directory, not the file being returned by the previous lookup: vnop_open: zsh-570 vnop_open: vnode_isdir( root) -> 1 With no open vnop_open calls made afterwards. Here's a similar backtrace for the original file being looked up. vnop_lookup: cat-1236 vnop_lookup: -> lookuprpc(/f.txt) lookuprpc: cat-1236 lookuprpc: lookup successful entry exists lookuprpc: -> cache_lookup(/f.txt) lookuprpc: <- cache_lookup(/f.txt) -> -1 ;VFS_CACHE_HIT vnop_lookup: <- vp fffffe2a45cf6b60 /f.txt If my vnop_lookup returns the correct vnode_t for the file being looked up, what causes ENOENT to be returned to open(2) in userspace? How do we track it down?

Thanks for clarifying that point. Hopefully, Xcode will soon come invested with the symbolication capabilities on Apple Silicon which will certainly be much appreciated. I have closed FB17497530. Thanks.

If you haven't already, please file a bug asking for us to provide a tool/tools for this process and then post the bug number back here. That's very kind of you to suggest that. Please see FB17733912. Please advise if I should close my previous bug report, FB17497530, which you've already looked into.

Further diagnostics have shown that ubc_msync(UBC_PUSHDIRTY|UBC_SYNC|UBC_INVALIDATE) was being called, for the file being written, through the VNOP_GETATTR path on each change of the file size while cluster_write() of the same file was in progress. The change I made now calls ubc_msync() only if there's no cluster_write() in progress when getting file attributes. To test the patch I extracted the zip file a few times in a row with the system no longer crashing and the following metrics being reported by vm_stat(1): % vm_stat 30 | awk '/Statistics/; !/Statistics/{print $3,"|",$13}' Mach Virtual Memory Statistics: (page size of 16384 bytes) specul | file-backed 165368 | 653350 65896 | 888301 66724 | 920760 65872 | 907261 63935 | 898648 67795 | 885978 63295 | 871778 59696 | 863204 3309 | 807539 188881 | 859100 60996 | 1018163 63041 | 1015598 59276 | 1013489 57392 | 1013524 60394 | 1013068 56646 | 1009342 634 | 953419 333949 | 1003401 59119 | 1132207 60277 | 1131932 56650 | 1128616 59147 | 1124974 56620 | 1124759 57165 | 1123408 55714 | 1122087 Thanks very much for all your help. Are you at liberty to disclose how you obtained the backtraces from the full panic long I'd supplied?

I was able to collect some diagnostics which revealed an interesting pattern of things that were happening prior to the kernel panic occurring. Depending on the amount of memory available, one or more files are able to get extracted from the zip file before the panic occurs. Calls being made cluster_write() gets called repeatedly until all file data is stored in VM. Then VNOP_PAGEOUT gets called. I thought that VNOP_PAGEOUT was only called for mmapped files. But there are no VNOP_MMAP/VNOP_MNOMAP calls being made for the files being extracted. The current VNOP_PAGEOUT implementation ends up calling err_pageout() for files that didn't get tagged as being memory mapped by VNOP_MMAP. After a number of VNOP_PAGEOUT calls, VNOP_STRATEGY is called which commits file data to remote storage via a do_write() rpc. This pattern is repeated until the system ultimately runs out of VM with kernel panic ensuing. My intuition is telling me that those VNOP_PAGEOUT calls are being made for a reason. Possibly, to free up the memory pages used up by the cluster API. I tried calling cluster_pageout() from VNOP_PAGEOUT despite VNOP_MMAP never being called. But that resulted in VNOP_STRATEGY being called through two separate paths, VNOP_PAGEOUT and cluster API, which resulted in I/O stalling. Any further pointers would be much appreciated.

Your description of the write operation logic my filesystem implements is fairly close. I thought I'd feel in a few blanks though. write() called. VNOP_WRITE gets called. Input data gets copied from user space into kernel space. Input data equal to the size of uio_resid() gets split up into smaller chunks and is iteratively transmitted over network until there's no more input data left with sock_send(MSG_DONTWAIT) returning the number of the bytes written in sentlen, which is equal to the value of uio_resid() of the chunk passed into sock_send(). VNOP_WRITE returns once all input data has been transmitted. write() returns. Not sure about the 'driver finishes uploading data at some point' part though. Or were you referring to the TCP stack queueing up the input data after sock_send(MSG_DONTWAIT) returns for subsequent transmission? At some point you need to decide to NOT return from "VNOP_WRITE" and instead block Without first understanding what's using up all that VM, I can't see how such a point may be determined. How would VNOP_WRITE block? For some reason, IOLog() data doesn't wind up in the output of log show|stream on macOS 15.5 running on M4 Max. I've added debug print statements to output buffer sizes and memory (de)allocation for uio_t buffers used to store the chunks of input data. I'll run the zip file extraction while collecting the diagnostics via log stream on an M2 Mac where logging is working and see if that reveals anything of interest.