NSFileProviderPartialContentFetching and high-latency API calls

Question

Stef120569 OP

Created 2w

Replies 2

Boosts 0

Participants 3

I am adding NSFileProviderPartialContentFetching support to an existing NSFileProviderReplicatedExtension. My backend has a high "Time To First Byte" latency (approx. 3 seconds) but reasonable throughput once the connection is established.

I am observing a critical behavior difference between Partial Content Fetching and standard Materialization that causes sequential reads (e.g., dd, Finder copies, Adobe apps) to fail with timeouts.

The Scenario: I have a 2.8 GB file. I attempt to read it sequentially using dd.

**Baseline (Working): Partial Fetching Disabled **

I do not conform to NSFileProviderPartialContentFetching.
The system triggers fetchContents(for:version:request:completionHandler:).
My extension takes 3 seconds to connect, then streams the entire 2.8 GB file.
Result: Success. The OS waits patiently for the entire download (minutes) without timing out, then dd reads the file instantly from the local disk.

**The Issue: Partial Fetching Enabled **

I add conformance to NSFileProviderPartialContentFetching.
The system requests small, aligned chunks (e.g., 16KB or 128KB).
My extension fetches the requested range. This takes ~3 seconds due to network latency.
The first few chunks succeed, but shortly after, the operation fails with Operation timed out.
It appears the VFS kernel watchdog treats these repeated 3-second delays during read() syscalls as a stalled drive and kills the operation.

**My Questions: **

Is there a documented timeout limit for fetchPartialContents completion handlers? It seems strictly enforced (similar to a local disk I/O timeout) compared to the lenient timeout for full materialization.
Is NSFileProviderPartialContentFetching inherently unsuitable for high-latency backends (e.g., cold storage, slow handshakes), or is there a mechanism to signal "progress" to the kernel to reset the I/O watchdog during a slow partial fetch?
Does the system treat partial fetching as "Online/Direct I/O" (blocking the user application) whereas full fetch is treated as "Offline/Syncing" (pausing the application), explaining the difference in tolerance?

Any insights into the VFS lifecycle differences between these two modes would be appreciated.

Boost

Answer 1

Engineer OP

Apple

2w

It appears the VFS kernel watchdog treats these repeated 3-second delays during read() syscalls as a stalled drive and kills the operation.

Did you see something in the system log which leads you to believe this?

If you can file a Feedback through Feedback Assistant, and post the number back here, it would be helpful. Installing the FileProvider logging profile prior to filing the Feedback (https://developer.apple.com/bug-reporting/profiles-and-logs/?name=file&platform=macos), and providing timestamps and filenames of your reproduction in the Feedback form, will make the Feedback more actionable.

Thanks!

0

Answer 2

DTS Engineer OP

Apple

2w

I'll also add that, given this dynamic:

My backend has a high "Time To First Byte" latency (approx. 3 seconds) but reasonable throughput once the connection is established.

...you might want to consider adding logic that would use the initial "NSFileProviderPartialContentFetching" call to trigger a full download, instead of only downloading EXACTLY what we specifically request.

More broadly, I think you should probably treat NSFileProviderPartialContentFetching as a tool that helps get data to the calling application more quickly, NOT as a firm API contract that you will ONLY retrieve the specific byte range it requested. For large files, your startup latency numbers are high enough that if you REALLY stick to specific byte range retrieval, you're going to end up with absurdly poor performance. Case in point, my back of the envelope map shows that downloading a ~2 GB file at ~128KB every 3s is going to take ~13 hours. Similarly, if you can fetch the "full" file with minimal additional time/bandwidth, I think it's totally reasonable to complete the file "just in case".

If you read through the documentation for fetchPartialContents(for:version:request:minimalRange:aligningTo:options:completionHandler:), you can actually see "hints" of how this could/should work. You’re allowed to return ranges larger than the requested range (including the entire file), up to and including the entire file. It also specifically recommends returning a cloned file if/when you still need/want that file’s contents:

"After receiving the fileContents, the system clones and deletes the file. Don’t mutate the corresponding file after calling the completion handler. If your extension needs to keep a copy of the content, create a clone of the content and pass the URL of the clone as the completion handler’s fileContents parameter."

That "trick" is what you'd use to service partial content requests while you're still downloading the entire file. That is, you start downloading to a specific file and you respond to content requests by returning clones of that same file as you fill out its contents.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

0