Hello everyone,
I've been chasing a nasty silent data corruption bug in the macOS SMB client (smbfs.kext) and wanted to share what I found, in case someone else has hit it and in the hope that someone from the SMB team sees it.
What happens
Under concurrent writes with repeated reopens, the client can regress its cached file size (np->n_size) to an earlier, smaller value - behind data it has already written and flushed to the server. The next write then treats the already-written range as a hole, zero-fills it via IO_HEADZEROFILL, and sends the zeros to the server, right on top of the correct bytes it transmitted moments earlier. No write(2) fails and nothing is logged - the file just quietly comes back with a chunk-aligned run of zeros in the middle, at the correct overall length.
Environment
macOS 26.5 (Darwin 25.5.0), Apple silicon (16 KiB VM pages), SMB 2.1
Sources referenced: SMB client 538.121.1, xnu 12377.121.6
How to reproduce
Mount an SMB 2.1(2.0.2 has the same issue as well) share.
Have several threads write the same files in 8 KiB chunks, each chunk via its own open/lseek/write/close (so the file is reopened constantly as it fills), while the files are concurrently resolved by name (stat / directory enumeration).
Read the files back through a cache-cold path (second mount, or F_NOCACHE) and compare.
Roughly 1 file in several hundred came back corrupted for me. The core of the write pattern:
CHUNK = 8192 # 2 chunks per 16 KiB page
def write_chunk(path, data, offset):
fd = os.open(path, os.O_CREAT | os.O_RDWR) # own handle per chunk
try:
os.lseek(fd, offset, os.SEEK_SET)
os.write(fd, data)
finally:
os.close(fd)
# per file: content = os.urandom(random.randint(265000, 300000));
# chunks written in batches of 8 threads, joined between batches;
# each file written twice from the same buffer: NAME, then NAME.copy
In my runs the corruption always landed on the second (.copy) write.
One caveat: I reproduced this against a third-party SMB server, not against macOS File Sharing (smbd), and I don't expect it to reproduce against smbd directly. The stale size arrives via reopen-via-lookup (smbfs_update_size <- smbfs_nget <- smbfs_vnop_lookup) on a freshly instantiated vnode, whose n_sizetime lets the freshness guard pass. smbd instead reopens via vnop_compound_open -> smbfs_attr_cacheenter (warm vnode; the guard rejects it) - the same stale-size candidates occur, they just all get rejected. The server merely steers the client onto the vulnerable path; the bug itself is entirely client-side.
What I observed
I captured the kernel side with dtrace fbt probes on smbfs_setsize() / smbfs_update_size() (os_log drops events under this load). Timeline for one corrupted file, correlating pcap and dtrace (dtrace has whole-second resolution, marked ".x"):
[pcap] = network packet capture of the SMB traffic between client and server
[dtrace] = kernel-side trace of the smbfs size-update functions; timestamps only have whole-second resolution, so ".x" marks an unknown sub-second time
:39.778 [pcap] — client sends WRITE off=32768 len=32768 with the correct data, covering [40960:65536).
:39.777–.860 [pcap] — throughout, the server's CREATE/CLOSE responses report a strictly monotonic EOF: 0, 8192, 40960, 65536, ... 288255.
:39.x [dtrace] — on a reopen, smbfs_update_size applies EOF 40960 (a superseded value), regressing n_size from 65536 to 40960.
:39.x [dtrace] — the next write starts past the regressed size, so zero_head_off = 40960 and IO_HEADZEROFILL is set.
:39.804 [pcap] — client sends WRITE off=32768 len=57344, ALL ZEROS over [40960:65536), on top of the correct data it sent 26 ms earlier.
End result: the file is 288255 bytes (correct length) with 24 KiB of zeros at [40960:65536) - three consecutive 8 KiB chunks, i.e. 1.5 x 16 KiB VM pages. Worth stressing: the server's own responses reported a strictly monotonic EOF the whole time, so the regression to 40960 was purely the client applying a superseded value.
Expected, obviously: the file reads back byte-for-byte identical to what was written.
Where I think the bug is
From reading the smbfs and xnu sources, three things combine:
np->n_size isn't consistently synchronized - read under the node lock only (smbfs_vnops.c:7329/7387/7391) but written under f_clusterWriteLock (:7411) and by smbfs_vnop_strategy under the cluster lock, so the reader deciding the zero-fill has no ordering guarantee. Possible fix: read it once under f_clusterWriteLock in smbfs_vnop_write so the snapshot, extend, and zero_head_off stay consistent.
The freshness guard checks the wrong thing - smbfs_update_size's reqtime <= n_sizetime guard validates the reply's request time, not whether the value is still current, so a superseded (smaller) size applied later still passes and calls smbfs_setsize(smaller). Possible fix: never shrink n_size from fa_size while the vnode has dirty pages or in-flight writes beyond that size.
The zero-fill is destructive - zero_head_off = np->n_size (smbfs_vnops.c:7391) feeds IO_HEADZEROFILL, and cluster_write zeros [n_size, uio_offset) without checking whether the UBC already holds those pages as valid/dirty (vfs_cluster.c), then flushes the zeros to the server. A defensive check there would neutralize the corruption regardless of cause.
Has anyone else seen silent zero-runs in files written over SMB under concurrent access?
Thanks!
