Hi Kevin, first off, thanks a lot for your pointers, and yes, macOS now has a DEXT-based iSCSI initiator for NAS. There's still a bit left to do, but the connection holds and traffic runs at full tilt. What more could you want. And as promised, here are some numbers: Cache-independent RAW iSCSI measurements These are cache-independent RAW whole-disk measurements against a real 4.29 TB NAS-backed iSCSI LUN, RAW and unformatted target. Every run completed deterministic read-back verification with zero data mismatches, so the VERIFY column is a full end-to-end integrity pass over the whole transfer. Notes: Host queue depth was 256 for every run. The two 5G tests used sixteen 1 MiB requests, so the effective queue depth there was 16. Both 5G connections went straight to the NAS over independent mobile networks, no VPN. Throughput on those two rows is set by the radio path. What matters is that the initiator carries complete, read-back-verified iSCSI transfers over a live mobile link, the groundwork for our iSCSI-over-TLS layer in the initiator. The 400 GiB endurance run stayed at 111.592 MiB/s WRITE and 111.537 MiB/s READ, WRITE, READ and VERIFY each about 61 minutes, with a bounded DEXT proxy count throughout and no port-table growth. This is all running on Intel now, and running well on older Intel hardware specifically mattered to us: for something like this it has to stay performant on aging systems, not just the latest ones. From next week we move to Apple Silicon (M5 Pro) and see how throughput holds up there. The initiator itself is not fully done either, getting a DEXT driver cleanly embedded inside an application turned out to be a real challenge. If anything else turns up during beta testing, I'll post it here, though I doubt it will. One last thing, and I mean it. Thanks for the past few weeks. You revisited your own conclusions the moment the facts moved, and you went out of your way more than once, the entitlement request on our behalf being the clearest example. That kind of engagement is rare and made this a real pleasure. So thank you, Kevin. Enjoy your weekend, and until the next posting. :) Torsten

Hi Kevin, yes, I mean the initiator perspective, real iSCSI READ/WRITE over the network. Right now it is more annoying than technically critical. The important part is the DEXT does what it should, at least that is how it looks. I completely agree with you. If WRITE runs but READ does not run cleanly, this is almost certainly a code issue, not a framework issue. My personal guess is that it comes down to a wrong or missing request ID, READs and WRITEs getting in each other's way during lookup. But that is only a theory. We will see who is right in the end. The bets are on. 😉 Nonetheless, thank you for your continued support so far. That is not something I take for granted. Thank you again! I promise you, once it runs, you will be the first to know. I will send you valid numbers then. I am curious myself. Best regards, Torsten

Hi Kevin, yes. At the API level the immediate cause was the lifetime of the IOBufferMemoryDescriptor returned by UserGetDataBuffer in our DEXT. Our explicit asynchronous retain and release accounting was already balanced, so the real work was to identify which proxy owned the growing Mach rights and to find the point where retirement was safe for deferred READ and WRITE tasks. Your observation that UserGetDataBuffer may have been intended primarily as a debugging aid turned out to be directly relevant. Under sustained I/O, the imported IOBufferMemoryDescriptor proxy and its lazily cached IOMemoryMap consumed Mach send rights per task until the DEXT approached port-table exhaustion. We isolated it in stages. lsmp snapshots around bounded I/O showed a strict 1:1 growth of IOBufferMemoryDescriptor and IOMemoryMap proxies. A purge-free 32 MiB write added 8,213 of each. They survived task completion, unmount, target removal and UserClient close. Only terminating the DEXT cleared the port table. Static analysis of the DriverKit runtime explained the pair: IOBufferMemoryDescriptor::GetAddressRange lazily calls CreateMapping and caches the IOMemoryMap on the descriptor. At refcount zero, IOBufferMemoryDescriptor::free() releases that map and the destruction path returns both Mach send rights. So the unit to retire is the descriptor proxy itself. Driving it to destruction releases the cached map with it. The safe retirement point came from the completion ordering in your open-source IOSCSIParallelFamily. SCSIParallelTask holds the task buffer in fDextDataBuffer. CompleteParallelTask releases that field before it calls CompleteSCSITask. Only after the return from ParallelTaskCompletion or BundledParallelTaskCompletion has the kernel-side SCSI task released its data-buffer ownership. Although it seems that this source is an older snapshot, we treated it as the lifetime model and confirmed the retirement point empirically against the current runtime. We treat the contract as empirical, derived from the SCSI-family ordering and confirmed against the runtime. The implementation now gives every successful UserGetDataBuffer import exactly one owner. We carry a single durable asynchronous hold through the single-task or bundled state and retire the proxy exactly once after framework completion returns, on a serial completion queue outside the data-path lock. With your ownership clarification, the expected retain count there is two. We release both, 2 to 1 to 0. Zero destroys the descriptor, releases its cached map and returns both send rights. The same exactly-once rule covers normal, inline, rejected, failed and bundled completion. If the count differs from the expected two, we release only our own hold and flag it in an anomaly counter. The result is port-flat under the load that previously exhausted the DEXT, with a 512 MiB write and cold readback byte-exact, GDBRETIRE unexpected=0, a stable PID and clean teardown and reconnect. The full lsmp captures and lifecycle analysis are attached to the last FA commit. The same work also updates our earlier read of the page-bound path. With direct raw-device I/O and cache bypass, WRITE now reaches about 111.7 MB/s over 2.4 GHz Wi-Fi. READ is currently held near 25 MB/s, about a 4.5x gap. WRITE shows the page-bound path can carry strong throughput, so the open limitation is now isolated to READ. We still have a small READ gap to close. I am fairly confident it sits on the iSCSI side. iSCSI is famously particular and rewards a clean implementation. With WRITE already at full speed, I read this as ours to solve, most likely in our own iSCSI handling. I would be surprised if DEXT or a framework issue were the cause. A clean DMA path would admittedly make it easier, but I am optimistic. We will see where it lands. One note on our own numbers, as a fair cautionary tale. Before we switched to raw-device I/O on the iSCSI LUN, we ran a test by copying a 4 GB file to the NAS over iSCSI via 2.4 GHz Wi-Fi. That test reported about 2.5 GB/s, completing in around 1.6 seconds, which is physically impossible over Wi-Fi. Two explanations were available: either we had quietly broken a few laws of physics and should start drafting a Nobel acceptance speech, or something went wrong with macOS, caching and/or RAM. We reluctantly chose option two and moved the harness to direct raw READ and WRITE on the iSCSI block device, bypassing the file system cache entirely. ;-) To find the remaining READ ceiling, we are wiring an end-to-end I/O flight recorder that timestamps and correlates each READ across SCSI task arrival, UserGetDataBuffer, ring publication, iSCSI Data-In reception, payload assembly, DEXT copy-back and framework completion. That should show whether the ceiling comes from buffer preparation, network scheduling, Data-In assembly, copy-back serialization or completion latency. Cross the fingers! Best regards, Torsten

Million times thanks to you!!! We finally got it & it's running! Hi Kevin, your last reply gave us the direction we needed. We have now validated a local, implementation-specific mitigation for the per-task proxy accumulation reported in FB24326124 In our current SCSIControllerDriverKit build, the DEXT port table remains essentially flat under the same page-sized workload that previously exhausted it and terminated the DEXT. The current validation result on our Intel test system is: 512 MiB of random data written to the mounted LUN unmount and remount before verification SHA-256 read back from the device matched the source DEXT PID remained unchanged throughout the test process port count was 111 before the transfer, 111 after WRITE and 116 after READ READ-back throughput was 219.4 MiB/s over WLAN The current WRITE timing is still affected by caching, so we are repeating that measurement with a cache-controlled test before publishing a number. The same build also completed clean unmount, target removal, UserClient close, reconnect and subsequent I/O. We are now extending the validation to longer soak tests and a substantially higher task-capacity configuration. The full lifecycle analysis, DriverKit runtime evidence and raw lsmp captures are already attached to FB24326124 I am keeping the public summary at the result level while the implementation remains under active validation. This gives us a viable interim path on the current public DriverKit surface. The underlying reports remain relevant: FB24326124 covers the per-task proxy lifecycle. FB23814092 covers the single-page software-I/O limitation. FB23814013 covers the missing CPU-accessible BlockStorage request descriptor. The current mitigation addresses the immediate stability blocker. It does not remove the additional per-page work imposed by the existing SCSI path, so an Apple-side fix would still provide a direct performance benefit without requiring another architectural change on our side. Right now this is a breakthrough. It works, but the full test matrix on Intel and Apple Silicon still has to be completed. At this stage it already looks like a more than viable solution until Apple has the final fixes in place. If the next soak tests remain clean, we will take a Developer ID build through notarization and begin a small, controlled external beta. Notarization would let us distribute broadly, but this driver writes at block level to live LUNs, so we will keep the initial rollout deliberately narrow and widen it only as field results allow. Now my final question to you: If any further SCSIControllerDriverKit bugs come up, including during the public beta, would it be fine with you if we post them to this thread? Thank you again to you and the SCSI team for continuing to investigate this. Best regards & enjoy your weekend!! Torsten

Hi Kevin, thanks, that matches what we ended up building: a tunable engine (I/O size, slot count) with single-page transfers and large slot counts, close to the single-page / massive-slot stress test you described. I filed FB24326124 with the full SDK, IIG, runtime-disassembly and lsmp evidence. Running that engine surfaced a second issue in the same per-task buffer path, beyond the UserGetDataBuffer overhead you flagged as the next bottleneck: a lifetime issue. For each data-bearing READ or WRITE SCSI task, the current path creates this proxy pair in the DEXT: UserGetDataBuffer -> one borrowed IOBufferMemoryDescriptor proxy GetAddressRange -> DriverKit CreateMapping -> one cached IOMemoryMap proxy The generated IIG reply carries one object, the IOBufferMemoryDescriptor. Static disassembly of the DriverKit runtime shows IOBufferMemoryDescriptor::GetAddressRange lazily calling IOMemoryDescriptor::CreateMapping and caching the returned IOMemoryMap on the descriptor. The normal destruction chain is present, but only runs at refcount 0: IOBufferMemoryDescriptor::free -> release cached IOMemoryMap -> OSObjectFree -> mach_port_deallocate The proxies remain live after final task completion. A purge-free 32 MiB write added 8,213 IOBufferMemoryDescriptor proxies and 8,213 IOMemoryMap proxies. The DEXT's own counter later reported: acquired=21061 released=21061 earlyDrop=0 gap=0 At the same point lsmp still showed 21,125 live proxies of each class. A teardown probe kept the same DEXT PID through volume unmount, target removal and UserClient close but the counts remained at 21,125 per class. DEXT process termination cleared the process port table. Because the DMA path fragments the workload into predominantly 4 KiB parents, the count rises quickly. In repeated runs the DEXT failed with its port count in the observed 260,000 to 290,000 range. Reconnecting the iSCSI session left operation unavailable in our test environment and only rebooting the Mac restored it. Two connections to your earlier notes: This is essentially the single-page / massive-slot engine you suggested as a stress test. It currently drives the DEXT into port-table exhaustion rather than only running slowly. Your read-path idea of parallel UserGetDataBuffer calls behind the SCSITask-table lock would leave the per-call lifetime unchanged. Any throughput gain would therefore bring the same port ceiling closer in time. Retiring the imported descriptor proxy once per task at a final state would let the existing destruction path reclaim both send rights without changing the public API. Does the software-I/O path you outlined already include that retirement, or would it be a separate lifecycle correction that could also be delivered in a macOS 26 software update for Intel? Thank you again to you and the SCSI team for continuing to look into the software-backed path. Best regards & enjoy your Weekend!!! Torsten

Kevin, thank you very much for the detailed response and for taking this back to the SCSI team. The confirmation that the single-page limitation is a bug rather than an inherent limitation of SCSIControllerDriverKit is probably the most important result of this investigation for us. Even though the current limitation means that we cannot move forward with a production implementation at the moment, having the issue clearly identified and having a potential path to address it is a huge step forward. The software I/O approach you described makes a lot of sense for our use case. In particular, the possibility of handling this within IOUserSCSIParallelInterfaceController, without requiring a DriverKit API or ABI change, sounds like a very clean solution for software-backed SCSI controllers. Your suggestion regarding the bundled architecture and a very high maximum task count is also very helpful. As I understand it, this would allow us to reduce the delivery overhead and task pile-up caused by the page-sized requests by keeping substantially more work in flight and delivering tasks in bundles. We already have a highly parallel data path behind the DEXT, so this is something we can test without fundamentally changing the architecture. We see this primarily as a useful proof of technology. It may demonstrate how far the existing path can be pushed despite the page-sized task limitation and we will certainly test it. For a production iSCSI initiator, however, we would not consider this workaround a viable foundation. Bundling can reduce delivery and scheduling overhead, but it does not remove the underlying fragmentation. A 1 MiB I/O still becomes 256 separate 4 KiB SCSI tasks. At sustained storage throughput, that means processing very large numbers of tasks, completions and associated bookkeeping for I/O that would naturally be represented by far fewer larger operations. Even if sufficient parallelism makes the throughput look reasonable, the CPU cost, latency characteristics and scaling behavior would remain concerns for a commercial storage product. That is why the bug fix makes such a fundamental difference for us. It is not simply another performance optimization. It determines whether SCSIControllerDriverKit can provide a clean production data path for this type of software-backed controller rather than requiring us to optimize around artificial I/O fragmentation. Given that the potential software I/O solution may be contained within IOUserSCSIParallelInterfaceController, without requiring a DriverKit API or ABI change, may I ask two final planning questions? I completely understand that you cannot provide a roadmap or commit to a release date. From an engineering perspective, is this something you would reasonably expect could be addressed on a near-term macOS software update timescale, or should we plan for the current single-page limitation to remain for the foreseeable future? Also, since macOS Tahoe 26 is the final major macOS release supporting Intel Macs, could a fix of this kind potentially still reach macOS 26 and therefore Intel Macs, or is it more realistic to expect that the corrected software I/O path would only become available on Apple silicon? The timing distinction is particularly useful for us. If the underlying issue has a realistic prospect of being addressed, we would much rather avoid investing significant engineering effort into optimizing a proof-of-technology path around a limitation that may disappear. Regardless of the timing, thank you again to you and the entire SCSI team for taking the time to investigate this. We really appreciate the depth of the technical feedback and the effort that has gone into understanding this use case. Although we cannot take the production implementation much further with the current limitation, having the limitation confirmed as a bug and having a potential path forward is a fantastic outcome for us. We hope that the underlying issue can be addressed in the near future and we are very much looking forward to testing the proper software I/O path if and when it becomes available. Best regards, Torsten

Kevin, regarding your statement that SCSIControllerDriverKit is not intended for virtual or software-based controllers, I believe there might be a small additive path that makes it work without opening the DMA security wall. After reviewing Apple's public kernel source repository, the public SCSIControllerDriverKit interfaces and the behavior of our software-backed SCSI prototype, we believe there may be a relatively small architectural change that would support software storage controllers without opening or weakening Apple's DMA security boundary. We are aware that granting developers direct access to DMA is not a direction Apple wants to take. That's the reason for this "workaround idea". Core Idea Apple does not need to open the DMA security wall. It can keep that wall fully closed for software-backed controllers and still provide the missing native storage path. We currently call this the No-DMA Solution. By "No-DMA" we mean the data-path contract presented to the DriverKit developer. A software-backed controller does not program a physical DMA engine. It therefore does not require a DMA address, IOVA, IOMapper or access to a device-specific IOMMU domain. The framework would continue to own and enforce all memory authorization. Current Task Path Our understanding of the current task path is approximately: ProcessParallelTask → PrepareForDMA → GenerateIOVMSegments → validate segment geometry → UserProcessParallelTask → UserGetDataBuffer Once UserProcessParallelTask() is reached, the existing UserGetDataBuffer() path provides the IOBufferMemoryDescriptor that a CPU-based software controller needs. The problem is that a request whose original buffer produces more than one DMA/IOVM segment can fail before the DEXT callback is delivered. The driver therefore never gets the opportunity to use the CPU-accessible buffer. Apple's public interface already defines UserGetDataBuffer() as returning an IOBufferMemoryDescriptor for the task. Proposed Change The proposed change is an explicit, opt-in software-backend capability. The following code is only illustrative pseudocode. We do not know the internal class names or the most appropriate public API shape. 1. Add a Controller Capability // Illustrative API only. constexpr uint64_t kSCSIControllerOptionSoftwareBackend = 0x00000001ULL; The existing default would remain the current hardware-DMA behavior: virtual uint64_t GetControllerOptions() { return 0; } A software-backed controller would opt in explicitly: uint64_t GetControllerOptions() override { return kSCSIControllerOptionSoftwareBackend; } The same capability could alternatively be reported as an optional key through the existing UserReportHBAConstraints() contract if that fits the framework ABI better. 2. Add One Task-Admission Branch Conceptually, the framework-side change could be: if (controller->GetControllerOptions() & kSCSIControllerOptionSoftwareBackend) { /* * Keep the original task buffer under framework ownership. * * Establish the request-scoped CPU-accessible buffer used by * UserGetDataBuffer(). * * Do not prepare the task for a physical DMA engine. * Do not generate an IOVA for the software controller. * Do not reject the task based on DMA segment count. */ PrepareTaskBufferForCPUAccess(task); DispatchUserProcessParallelTaskOrBundle(task); } else { /* * Existing hardware-controller path, unchanged. */ PrepareForDMA(task); GenerateIOVMSegments(task); ValidateControllerDMALimits(task); DispatchUserProcessParallelTaskOrBundle(task); } 3. Reuse the Existing CPU-Buffer Path The software controller would continue using the public interfaces that already exist: IOBufferMemoryDescriptor *buffer = nullptr; IOAddressSegment range = {}; UserGetDataBuffer( targetID, controllerTaskIdentifier, &buffer ); buffer->GetAddressRange(&range); /* * CPU-based software processing: * iSCSI, NBD, encryption, compression, cloud storage, etc. */ ParallelTaskCompletion(...); Essential Contract The exact implementation may differ internally. The essential contract is only: Software-backed controller selected → no hardware-DMA preparation → no DMA segment-count admission gate → normal task delivery → complete request range available through UserGetDataBuffer → existing completion and cancellation semantics retained Existing hardware controllers would remain on the current path because they would not set the capability. For software-backed controllers, multi-page and page-straddling requests could then be delivered up to the controller's reported maximum transfer size without making physical DMA segment count part of the controller contract. Security Model The security model would remain narrowly request-scoped: no raw DMA address exposed no IOVA reverse mapping no public IOMapper access no physical-address access no access to another device's IOMMU domain The DEXT would receive only the buffer already authorized for the current task, with its range, direction and lifetime bound to that task. Access would end with completion, cancellation or teardown. Architectural Distinction Architecturally, this would create a clean distinction: Hardware-backed controller: request buffer → DMA preparation → IOMapper / IOVA → physical device Software-backed controller: request buffer → request-scoped CPU access → software transport or processing Immediate Use Case: iSCSI Initiator For our iSCSI initiator the resulting path would be: macOS SCSI task → UserProcessParallelTask → UserGetDataBuffer → software iSCSI transport → ParallelTaskCompletion This would remove the current dependency on one DMA/IOVM segment while leaving Apple's DMA isolation fully intact. (Tadaaaa - sounds like a great Jackpot) Although iSCSI is our immediate use case, the same capability would also support software-defined block storage, NBD, cloud-backed disks, encryption, compression, deduplication and virtual test controllers. Scope Assessment We obviously do not know whether the internal framework is structured exactly as the pseudocode suggests, or whether the existing UserGetDataBuffer() backing buffer currently depends on part of the DMA-preparation path. We also do not expect the API names above to be adopted literally. The architectural change nevertheless appears local and additive: one opt-in controller capability one task-admission distinction one request-buffer lifetime guarantee If this approach proves successful for SCSIControllerDriverKit, a similar concept could likely be applied to BlockStorageDeviceDriverKit as well though from our perspective, that would require a significantly larger effort. With DriverKit 27 currently in beta, we thought this might be a useful time to raise the idea & it's the easiest solution. ;) Best regards, Torsten

Hi Kevin, thanks again for the detailed explanation and for requesting the SCSIControllerDriverKit entitlement. Your last reply makes the architectural situation much clearer. In particular, the point that neither BlockStorageDeviceDriverKit nor SCSIControllerDriverKit was originally designed with software-backed I/O in mind explains very well why we keep ending up at the same boundary from two different directions. I’m now trying to make a practical product decision and would appreciate your technical read. For a pure software iSCSI initiator, BlockStorageDeviceDriverKit still seems like the conceptually natural fit: there is no PCI/Thunderbolt device and no hardware DMA target. The block-device model itself already gives us what we need. The missing piece is a request-scoped, CPU-accessible memory descriptor for DoAsyncReadWrite instead of only dmaAddr. From the outside, this looks like a relatively contained addition to the existing BlockStorage I/O path, particularly since the underlying I/O path already has the memory descriptor needed to establish the DMA mapping. I fully appreciate that the implementation, ABI, security and validation work on your side may make it considerably less trivial than it appears from the API boundary. SCSIControllerDriverKit works functionally, which is an important result, but the current page-bound path is not viable for our product. Even if that limitation is fixed first, SCSI remains an adaptation of a hardware-controller interface to a software-backed device rather than the natural abstraction for it. So we are effectively choosing between three paths: • continue investing in SCSI and wait for the larger-I/O fix you mentioned; • wait for a proper CPU-accessible BlockStorage I/O path; • ship a classic KEXT, which on Apple Silicon means requiring Reduced Security. The third option is the one I would most like to avoid. Apart from the security implications, asking users to lower their Mac’s security policy simply to access an iSCSI LUN is a poor installation experience. At the same time, we need to make a realistic engineering decision. If there is a realistic prospect of BlockStorage gaining such a path in a future macOS release, investing significant effort into a new KEXT may make very little sense. I understand that you cannot provide a roadmap or promise a timeframe; even a qualitative indication of whether this is something worth designing around would be extremely helpful. What I’m really asking is for your technical intuition: given what you now know about this use case and the current implementation, is BlockStorage support for software-backed I/O something you could realistically see being addressed in a future macOS release, or should we plan on the assumption that SCSI is the only DriverKit path for the foreseeable future, with a KEXT as the fallback if its performance limitations cannot be resolved? If additional measurements, a minimal reproducer or anything else attached to FB23814013 would help make the case internally, I’m very happy to provide it. Thanks again, Torsten

Hi Kevin, thanks, that helps. Let me answer your last post directly first, because we now have concrete data for both the SCSI and BlockStorage paths. Regarding the ring size: agreed. We were deliberately conservative because we initially treated the shared region more like scarce driver memory. Your clarification changes that assumption. A substantially larger ring is clearly possible and may improve batching and headroom. However, our measurements also support your other point: increasing the ring would not fundamentally remove the bottleneck while the original SCSI requests remain page-sized. Using normal mounted-volume I/O on the same ~152 MiB file on our Intel test system, we measured: Initial request-by-request WRITE: 0.32–0.36 MB/s Shared-memory ring WRITE: 2.5–2.9 MB/s Shared-memory ring READ: 5.8–8.0 MB/s 1 GbE payload ceiling: ~110 MB/s The shared ring improved WRITE by roughly 7–9x. But almost all framework requests remained 4 KB: WRITE: 4 KB 37,238 16 KB 18 64 KB 0 READ: 4 KB 38,315 16 KB 2 64 KB 22 Our application-side coalescer reduced 92,026 original requests to 91,876 wire commands, only about 0.16%. By the time requests reach us they are already separate active SCSI tasks, so a larger ring can keep more requests in flight but cannot remove the per-task framework lifecycle. That gives the scale you asked for: on Intel, the current single-page behavior costs us roughly one to two orders of magnitude compared with normal 1 GbE NAS throughput. QD64, multiple ITTs, ImmediateData, larger Data-In PDUs and the shared ring all work, but they cannot compensate for paying the full SCSI/DriverKit lifecycle for almost every 4 KB request. On Apple Silicon the 16 KB page size reduces the request count by 4x, but the same limitation remains. For a NAS initiator, that is still not product-viable. So I agree that SCSIControllerDriverKit is probably the most useful path to fix first. If the kernel starts issuing larger requests while UserGetDataBuffer remains unchanged, our SCSI DEXT should require little architectural change. There is, however, one new result from the BlockStorage path that may be relevant. Our IOUserBlockStorageDevice implementation is essentially complete apart from one very small but critical missing bridge. We already have dynamic geometry from the real iSCSI session, RMB=0 / native fixed-disk presentation, /dev/diskN, publish/unpublish, READ/WRITE/FLUSH orchestration, queue depth, shared-memory App/DEXT transport, real iSCSI READ/WRITE, completion handling and PR / single-writer arbitration. The remaining problem is specifically this callback: DoAsyncReadWrite( bool isRead, uint32_t requestID, uint64_t dmaAddr, uint64_t size, uint64_t lba, uint64_t numOfBlocks, IOUserStorageOptions options) DriverKit 25.5 documents dmaAddr only as: DMA address of the data buffer Unfortunately, the documentation does not explain how a software-backed BlockStorage driver is supposed to access the data behind that DMA address from CPU context. We initially tested whether dmaAddr might be CPU-dereferenceable inside the DEXT. Runtime proved that assumption wrong. READ completion repeatedly crashed the DEXT at: memcpy(reinterpret_cast<void *>(dmaAddr), ...) with: EXC_BAD_ACCESS KERN_INVALID_ADDRESS After repeated IOUserServer crashes, macOS eventually panicked with: Driver IOUserServer(com.aviontex.iscsi...) has crashed too many times (reason 2:11) We stopped testing and inspected the actual Xcode 26.5 / DriverKit 25.5 SDK instead of making further assumptions. We checked IOUserBlockStorageDevice.iig, the generated header, the private StartDev interface, exported BlockStorageDeviceDriverKit symbols, the generic IODMACommand API and the wider DriverKit headers. We could not find any public equivalent of: GetDataBuffer(...) GetDMACommand(...) dmaAddr -> IOMemoryDescriptor dmaAddr -> IODMACommand Map/Resolve/LookupDMAAddress(...) This is particularly noticeable because other DriverKit families expose descriptors explicitly when CPU-side access is intended. DoAsyncUnmap() in the same BlockStorage class receives an IOMemoryDescriptor *, and SCSIControllerDriverKit explicitly provides UserGetDataBuffer(). IODMACommand::PerformOperation() initially looked promising. DriverKit documents it as a way to perform CPU access to a prepared DMA mapping, for example to/from a driver-allocated bounce buffer. But that method operates on a specific prepared IODMACommand instance. IOUserBlockStorageDevice::DoAsyncReadWrite() gives us only the resulting DMA address, not the IODMACommand or IOMemoryDescriptor that owns that mapping. Interestingly, our older experimental BlockStorage implementation had already identified exactly this gap. It contained a proposed: dmaAddr -> avx_descriptorForDMA() -> IOMemoryDescriptor path, but avx_descriptorForDMA() was deliberately left as a stub returning nullptr until a real BlockStorage-family API could be identified. So the BlockStorage path is now in a rather frustrating position: from our side it is almost finished, and the only missing connection is: framework-created DMA mapping | dmaAddr | ??? | CPU-accessible request bytes | shared ring | iSCSI transport We searched the public DriverKit 25.5 SDK for that bridge and cannot find one. So the remaining question is now very narrow: Is this absence intentional? Is IOUserBlockStorageDevice designed on the assumption that dmaAddr is handed to DMA-capable hardware, with no supported CPU-access path for a software-backed device? Or is there an intended BlockStorage-specific mechanism to access the already-prepared mapping that is not exposed or documented in the public SDK? If such a mechanism exists, the BlockStorage path may genuinely be only one small missing API connection away from working and would avoid the single-page SCSI limitation entirely. If it does not exist, then the picture is finally clear: SCSIControllerDriverKit: CPU-accessible request descriptor available, but I/O is currently page-bound. BlockStorageDeviceDriverKit: the desired block-I/O model is available, but READ/WRITE exposes only a DMA address with no documented CPU-accessible descriptor path. In that case, your comment that SCSIControllerDriverKit is the path most likely to be fixed first makes complete sense, and we would freeze the BlockStorage work rather than build another unsupported workaround around the DMA contract. Thanks again for helping us narrow this down. At this point the remaining BlockStorage issue is no longer a large architectural problem on our side, but literally this one missing DMA-to-CPU access bridge. Best regards, Torsten

Hi @lazarro! That’s exactly our goal. We want the native iSCSI path to be really fast. From my side it no longer feels like a question of if Apple will provide the missing descriptor support, but rather when. I’m hoping they play along because without that change this path simply won’t become what it needs to be. I’m keeping my fingers crossed for all of us ;) I don’t expect Kevin to be able to answer the timing question here in the forum anytime soon. We will probably both have to wait a bit and it might unfortunately take some more time. Best regards, Torsten

Hi Kevin, We now have the measurements we promised. My previous post was still based on calculations and on the assumption that the single-page SCSI path might remain viable if per-request overhead stayed low enough. On our Intel test system, that assumption does not hold. Functional result SCSIControllerDriverKit works functionally for a native software-backed iSCSI device. We validated an RMB=0 fixed disk backed by a real 4.29 TB Synology LUN with 512-byte blocks, native IOMedia and APFS mounting, verified READ and WRITE traffic, clean target creation and removal, QD64, bundled task intake, multiple iSCSI ITTs, 64 KB Data-In PDUs and ImmediateData. This is a mounted native macOS disk carrying real file I/O, not an INQUIRY-only probe. Measured performance The same approximately 152 MiB file was used throughout. Path / direction Elapsed time Effective rate Initial request-by-request path, WRITE 7-8 minutes 0.32-0.36 MB/s Shared-memory ring, WRITE 52.5-62 seconds 2.5-2.9 MB/s Shared-memory ring, READ 19.8-26.3 seconds 5.8-8.0 MB/s 1 GbE payload ceiling, context only ~1.4 seconds ~110 MB/s The ring improved representative WRITE performance by roughly 7-9x. That is a real gain, but a result of only a few MB/s is still not viable for a LAN-connected NAS block device. Why we built the ring The initial request-by-request path took seven to eight minutes for 152 MiB, so we built a substantial workaround to determine whether the App/DEXT handoff was the primary bottleneck. The DEXT now creates and shares a roughly 16 MB IOBufferMemoryDescriptor containing a request queue, a completion queue, 64 request slots, 64 completion slots and 64 payload slots of 256 KB each. The path supports QD64, bundled DriverKit intake, multiple ITTs, out-of-order completion, doorbells, completion kicks and ImmediateData. The data path is effectively: SCSI task -> UserGetDataBuffer -> request mapping -> shared staging slot -> userspace iSCSI -> completion ring -> READ copy-back -> individual framework completion This removed the old payload-sized UserClient transport and the QD1 bottleneck. It did not remove the framework lifecycle of each original SCSI task. What remained Despite advertising 256 KB through Block Limits VPD, representative epochs were still almost entirely 4 KB tasks: Task size WRITE READ exact 4 KB 37,238 38,315 exact 16 KB 18 2 exact 64 KB 0 22 exact 128/256 KB 0 0 other small 10 25 An application-side coalescer was byte-correct, but 92,026 original requests became 91,876 wire commands, a reduction of only 0.16%. The requests were already individually active rather than accumulating as a mergeable batch. The limiting granularity therefore originates above the ring. The single-page restriction is the blocker Every workaround still pays one complete framework lifecycle per page-sized task: callback, UserGetDataBuffer, descriptor and mapping ownership, data movement and individual completion. QD64 can overlap these lifecycles; it cannot remove them. For a 152 MiB transfer, the arithmetic is: Request size Request count 4 KB 38,912 16 KB 9,728 64 KB 2,432 256 KB 608 We have not yet measured Apple Silicon, but its 16 KB page does not change the verdict. Even assuming ideal 16 KB tasks, the same file still requires almost ten thousand complete framework lifecycles. A larger system page reduces the count but does not remove the page-bound architecture. Measuring Silicon would refine the number, not the conclusion, because the constraint we need removed is single-page, not 4 KB specifically. What we actually need A self-created IOBufferMemoryDescriptor only describes DEXT-owned staging memory. It is not the descriptor of the current DoAsyncReadWrite request, and it does not make dmaAddr a documented CPU-accessible pointer. For a software-backed block device, we need the request-scoped memory object: a documented, CPU-accessible, multi-page descriptor with defined length, direction, mapping, synchronization, ownership and asynchronous lifetime. DoAsyncUnmap already carries an IOMemoryDescriptor in the same IOUserBlockStorageDevice class, while DoAsyncReadWrite exposes only dmaAddr. Such an API would not guarantee line-rate performance. It would remove the artificial requirement that a software network block device be forced through an IODMACommand/DART-dependent single-page workaround. FSKit We considered the FSKit/raw-DiskImage approach seriously, but for a native iSCSI initiator it is not an equivalent solution. The device it produces is a raw disk image, not a SCSI device. It exposes disk-image block semantics, so the behavior our initiator implements at the SCSI transport level has nowhere to live: Persistent Reservations, sense data, Unit Attention, proper SCSI error reporting, task management and task ordering. FSKit models a filesystem, and the disk-image indirection repurposes it as a byte-backing store, so the result is a block device layered over a file rather than a native one. It is also fragile as a product foundation. The block device exists only while the FSKit mount and the hdiutil attachment stay alive, so an extension crash, an app update or an unclean teardown takes the device, and anything mounted on it, with it. For byte movement it may well be faster than the current single-page SCSI path, and we are not disputing that. It is a different, non-native architecture that discards the SCSI device model our product is built on. SCSIControllerDriverKit is therefore functionally viable for this use case, but the current page-bound request path is not product-viable for us. The shared ring makes the workaround substantially better; it cannot remove the reason the workaround exists. Without the single-page restriction, performance is an engineering problem. With it, performance is an API-architecture problem. Does this measured result match your technical view of the remaining constraint? If it does, we are left waiting for a kernel fix: a request-scoped, multi-page descriptor. For a software device the clean home for it is BlockStorageDeviceDriverKit, which sidesteps IODMACommand entirely; without a multi-page path there or in SCSIControllerDriverKit, every path we have stays pinned to one page per request, 4 KB on Intel and 16 KB on Apple Silicon. Such a change would ship only in a future macOS, which puts Intel-based Macs permanently outside this feature. That is a trade-off we can live with. Best regards, Torsten

We ran the whole thing through on paper first. Everything below is calculation and assumption, not measurement. The real numbers come once we measure, but the calculations already let us make a call. Short version: we build on SCSI and validate it. FSKit stays as a last resort. Extending BlockStorage with a descriptor path looks like the clean fix. 1. SCSI DEXT On the single-page limit we did the math instead of guessing. Assuming 16 KB per request on Apple Silicon, a 20 GB transfer comes to 1,310,720 requests (on Intel at 4 KB it would be 5,242,880). Whether that becomes a problem depends on per-request latency. Working it through, in theory it looks like this: Latency per request 20 GB @ 16 KB Effective throughput 10 us 13 s 1.5 GB/s 50 us 66 s 305 MB/s 100 us 131 s 153 MB/s 140 us 183 s 110 MB/s <- 1 GbE line 200 us 262 s 76 MB/s 500 us 11 min 31 MB/s If those numbers hold, the single-page limit stays acceptable up to roughly 140 us per request, which keeps us at or above the 1 GbE ceiling of about 110 MB/s. Our target market of SOHO and home NAS runs mostly on 1 GbE and Wi-Fi anyway, where real-world throughput sits at or below that line. And if there is a way to lift the single-page constraint, that only improves the picture. 2. FSKit This is the option you raised, and on raw speed you are probably right: it avoids the DMA path completely. We took the suggestion seriously and ran the numbers. Since FSKit has no fixed transfer size like the single-page SCSI path, the only thing we can really compare is request count. Assuming cluster-sized I/O, a 20 GB transfer looks like this: 20 GB transfer, requests by I/O size (assumption) SCSI single-page 16 KB 1,310,720 requests FSKit 64 KB 327,680 requests (4x fewer) FSKit 128 KB 163,840 requests (8x fewer) FSKit 1 MB 20,480 requests (64x fewer) On request count FSKit in theory clearly wins. The trade-off is higher per-request overhead through the VFS layer, so which approach is faster in the end we cannot say without measuring both. What makes FSKit unattractive for us is not throughput, but what it costs at the layer above it. It hands us file offsets instead of SCSI semantics. Reservation handling, sense data, command ordering and error semantics would all have to be rebuilt on top of a layer that no longer speaks SCSI. And more important, our own in-house iSCSI protocol extensions (iSCSI-over-TLS or iSCSI-via-Remote) are built on native iSCSI. A SCSI or BlockStorage DEXT keeps that native layer underneath them. FSKit replaces it, and we cannot yet say what that does to those extensions, but building on a file abstraction instead of the native wire is a risk we would rather not take. If nothing else works we would have to, but it would set our development back considerably. 3. Extending BlockStorage with a descriptor path This would be additive and entitlement-gated. Leaving architecture, design, conception, testing and review aside completely and looking purely at the implementation effort, our estimate is less than a week of native code with no impact, since it is an extension and not a change. The reasoning: DoAsyncUnmap already carries an IOMemoryDescriptor across the DriverKit boundary in the same class, while DoAsyncReadWrite carries only a bare address one method away. So the plumbing to pass a descriptor already exists in the family. The descriptor also has to stay alive in the request until CompleteIO regardless, so it looks reachable via request ID. More broadly, this looks like a gap and not a missing capability. Other DriverKit families hand the dext a real IOMemoryDescriptor for their data path. Even SCSIController does, through UserGetDataBuffer, which is exactly what we are relying on. BlockStorage is the one that does not: its DoAsyncReadWrite passes only a bare address, with no descriptor equivalent anywhere in the class. And storage is exactly the area where getting the bytes wrong costs data. Our conclusion Of the three paths, only the descriptor extension actually solves this rather than working around it. SCSI seems to work but stays capped. FSKit works but breaks the native semantics. The descriptor path is the only one that removes the underlying reason this device class still needs a kext. Laid out plainly, this is why it looks like the right call to us, and why we think it is the sensible one for the platform too: Apple users finally get a native iSCSI solution. The missing piece is only the descriptor path in the family. It makes the DMA problem moot instead of fixing it. No IODMACommand, no segments, no DART, no single-page limit, and no impact on the general DMA path that every driver depends on. It moves a whole device class off kexts. iSCSI on macOS means a kext today, and on Apple Silicon that means Reduced Security. The descriptor path removes the technical reason for that, for everyone, not just us. It strengthens platform security. Nobody has to weaken their machine to reach their own storage. It gives people a reason to move to Apple Silicon. A fix lands from macOS 27 onward, which is Apple Silicon only, so native high-throughput iSCSI becomes a capability of the newer machines rather than a reason to stay on a kext. Realistically a change like this would take maybe six to nine months through the normal release cycle. That would put it in macOS 27 or later. Older Intel machines would keep running on the single-page SCSI path on Tahoe, without regression. Nobody would lose a working setup. That is a bigger outcome than one product, and that is why we keep coming back to it. We are building on SCSI now and will test whether it holds up. If it is good enough for SOHO and home NAS traffic, we can live with it, and none of this blocks us. We would of course like to know how long something like this might take, but we know that is not a question you can answer. The one thing we are asking for is your technical read. Does this match how you see the constraints today? And is there a fundamental blocker in this approach that we are missing? Best regards, Torsten

Hi Kevin, Both bugs are filed: FB23814013 FB23814092 If you need more detail on either, or want anything in a different form, just say the word and I will add it. Filing them was the easy part. We spent most of yesterday testing, so the data was already sitting there. Thanks again for requesting SCSIControllerDriverKit on our behalf. That was not something I expected and it is appreciated. We will keep testing and give you an update once we know more, probably sometime next week. Best regards & enjoy your weekend!!! Torsten

Hi Kevin, I'm afraid I have to disagree with one detail. Please give me two seconds to enjoy the moment. You know, the Big Five for Life: getting married, having children, planting a tree, building a house and correcting an Apple DTS engineer. :-) All right, moment over. Seriously, though: thank you for revisiting this so openly, investigating the alternatives and trying to move the underlying issues forward internally. Your revised overall conclusion appears to be correct: SCSIControllerDriverKit is viable for this use case, at least on the system we have now tested. One detail may need clarification, though: the descriptor API path. In the public DriverKit 25.5 headers, UserMapHBAData does not return a memory descriptor. Its only output is the unique controller task identifier: UserMapHBAData( uint32_t *uniqueTaskID ); The descriptor itself is retrieved later through: UserGetDataBuffer( targetID, controllerTaskID, &buffer ); when called from UserProcessParallelTask, using the request's fControllerTaskIdentifier. Perhaps this is the mechanism you meant. UserMapHBAData establishes the task identity, and UserGetDataBuffer subsequently retrieves the task's IOBufferMemoryDescriptor. As we are still testing, we would like to keep the conclusions narrow and clearly distinguish between what we have measured and what remains open. Current test configuration Hardware: Intel Mac macOS: 26.5.2 Xcode: 26.5 DriverKit SDK: 25.5 IOClass: IOUserSCSIParallelInterfaceController IOProviderClass: IOUserResources There is no PCI or Thunderbolt provider. The signed DEXT does not request the PCI transport entitlement. The extension was installed and activated successfully. The controller became registered, matched and active. The following lifecycle and reporting steps completed: UserInitializeController UserReportHBAConstraints UserStartController UserInitializeTargetForID The framework then submitted real SCSI commands through UserProcessParallelTask. Inside those callbacks: UserGetDataBuffer -> kIOReturnSuccess -> non-null IOBufferMemoryDescriptor GetAddressRange -> kIOReturnSuccess -> nonzero DEXT-local address -> expected transfer length We initially performed a descriptor-only probe without touching the buffer. That succeeded. We then performed a strictly bounded standard INQUIRY write using the address returned by GetAddressRange. The observed sequence was: INQUIRY allocation length 6 -> wrote 6 bytes -> completed GOOD INQUIRY allocation length 36 -> wrote 36 bytes -> completed GOOD macOS consumed the initial six bytes and subsequently requested the complete 36-byte INQUIRY response. The resulting IORegistry properties contained the exact data written by the DEXT: Vendor: AVIONTEX Product: iSCSI4NAS VIRT Revision: 0001 The native storage stack then advanced through: IOSCSIParallelInterfaceDevice IOSCSITargetDevice IOSCSIHierarchicalLogicalUnit IOSCSIPeripheralDeviceType00 IOBlockStorageServices IOBlockStorageDriver It is now issuing subsequent discovery commands, including READ CAPACITY(10). There were no panics and no DEXT crashes. The DriverKit crash counter remained at zero. For this specific configuration, we have therefore demonstrated: IOUserResources -> IOUserSCSIParallelInterfaceController -> UserProcessParallelTask -> UserGetDataBuffer -> IOBufferMemoryDescriptor -> GetAddressRange -> bounded byte write -> successful SCSI completion -> data consumed and interpreted by the macOS storage stack This confirms your revised conclusion that the SCSI path is possible today. It also shows that the original fBufferIOVMAddr limitation does not prevent the DEXT from accessing the request data, because UserGetDataBuffer provides a separate descriptor-based path. We still do not use or dereference fBufferIOVMAddr. Single-page limitation Your warning about the single-page DMA limitation remains open. Our successful transfers were only: 6 bytes 36 bytes These results therefore neither confirm nor disprove a single-page limitation. We have not yet tested a cross-page request, Scatter/Gather I/O or real READ/WRITE commands. Could you clarify the exact configuration Apple currently recommends to enforce the single-page restriction? In particular, should a virtual HBA use: maximum segment count read = 1 maximum segment count write = 1 maximum segment byte count read = runtime system page size maximum segment byte count write = runtime system page size maxTransferSize = runtime system page size Should any additional alignment constraint be applied? And should "one page" always use the runtime system page size rather than a fixed 4096-byte value, particularly when validating the same implementation across Intel and Apple Silicon? BlockStorageDeviceDriverKit and FSKit We also agree with your assessment of BlockStorageDeviceDriverKit. Its current DoAsyncReadWrite interface exposes only the DMA address and does not provide an equivalent to UserGetDataBuffer. We will file the two requested Feedback Assistant reports: A descriptor-only read/write path for BlockStorageDeviceDriverKit A documented descriptor-only, multi-page I/O path for SCSIControllerDriverKit We will post both feedback numbers here once they have been submitted. We will also retain the FSKit/raw-DiskImage design as a fallback and potential performance comparison. For now, however, the SCSI path has progressed far enough that we would like to finish validating it before changing architectures. Scope For completeness, the results above currently apply only to: Intel macOS 26.5.2 DriverKit 25.5 development signing Developer Mode SIP disabled The following remain separate validation steps: Apple Silicon SIP-enabled standard security distribution provisioning multi-page I/O real READ/WRITE traffic long-running stability This is an early status report, not a final result. We will keep testing and keep you posted as we go, including the feedback numbers once both reports are filed. One more thing, and I mean it. When we started, we did not see the SCSI option at all. In hindsight we were probably too fixated on BlockStorageDeviceDriverKit, because from where we sit that is the more logical family for this product. It took both the entitlement process and your pointers to send us back to the SCSI DEXT and look at it properly. So thank you again. Your correction and the pointer back toward the SCSI path appear to have saved the architecture.

Thanks Kevin, that was fast and clear, and I appreciate you taking the time to look at it. I really hope you are wrong, and not just for selfish reasons. If neither storage family can carry a network backed block device, then there is no DEXT path to iSCSI on macOS at all. Not for us, not for anyone. It stays a kext, and on Apple Silicon that means telling users to drop to Reduced Security to reach their own storage. Nobody should have to weaken their machine to mount a disk. As a product that is simply not shippable, so for us that road ends there. The selfish reason is easier to explain: three years of my life are in this thing. So you can imagine I read your reply twice, went for a walk, and read it a third time. Right now the outcome is either "we built something that cannot exist" or "the DTS engineer is wrong about one detail". I know which one I am betting on, though I admit the odds are not in my favour. No argument from me either way, you know this stack better than I ever will. But I would rather find out than debate it. We are going to test this thoroughly now and see what actually happens, and as soon as I know more I will come back with the details. Thanks again.