No. That doesn’t buy you any thread safety above and beyond what URLCache already supports. Got you, will not bother with the locks then. Yep, I’ve seen that myself. I believe it comes about because URLCache defers write operations on a background queue and so your subsequente read operation races with that. I suspected this. A better behaviour would be: update memory representation, return updated memory representation for readers and if needed do the writing on a background queue (but that should not affect readers). Again, keep in mind the big picture here: URLCache was designed to meet the needs of an HTTP client, Safari basically, not as a general-purpose caching API. I hear you. There are pros and cons of course. It’s certainly doable but that code is not sufficient. To make this work you need a primitive other than a lock because other threads have to be able to block on the reservation waiting for the ‘lead thread’ to finish the load. I generally hate locking, especially long term... IMHO - that's the recipe for deadlocks... I would make other clients getting some placeholder data (e.g. "loading" image if that's an image) without blocking. Once the data is finally loaded by the lead thread - the corresponding model state is changed and the relevant updates should be sent so that the all interesting parties update their representations (e.g. via SwiftUI's observation machinery, etc).

I use URLCache to not implement caching myself; it does everything I need out of the box, through I admit my caching needs are pretty basic: essentially a key / data store with ability to specify total size, with persistence & encryption, cleaned up by OS either partially or completely if OS needs disk or RAM. If I were to implement caching myself (which I can) that would be quite some chunk of work I'd rather not do, testing and handling edge cases (like sanitising and deleting cache after app crash that left cache in an inconsistent state). Here I follow the "Best code is no code" strategy (or, like in this case the well tested code that is maintained by the system itself). Your example with "waste" + "override" is understandable, although not a show stopper. If I read the warning sentence above properly (which I am not sure I do!) that "concurrent read/write calls themselves to the same resource can crash" (similar to how unprotected read / modify of a dictionary can crash), then this is what I plan doing: extension URLCache {     static let lock = NSLock()     func storeCachedResponse_mt(_ response: CachedURLResponse, for request: URLRequest) {         Self.lock.lock()         defer { Self.lock.unlock() }         storeCachedResponse(response, for: request)     }     func cachedResponse_mt(for request: URLRequest) {         Self.lock.lock()         defer { Self.lock.unlock() }         return cachedResponse(for: request)     }     func removeCachedResponse_mt(for request: URLRequest) {         Self.lock.lock()         defer { Self.lock.unlock() }         removeCachedResponse(for: request)     } } Does it make sense? Under what circumstances can this deadlock? I also found some strange behaviour in a single threaded scenario:     removeCachedResponse(for: request)     cachedResponse(for: request) // still returns data sometimes! This is a single threaded case when no-one else is writing the cache. ditto for "set + get" with get sometimes returning the old value. It feels like "remove"/"set" is somewhat asynchronous.. There is nothing in the docs abut this behaviour. I put some workaround but I am not totally happy about it. Cache reservation you are talking about is probably doable via (pseudocode): func read() { lock() result = get() if result == nil { set(loadingMarker) unlock() loadResource() { data, error in if not error { lock() set(data) unlock() } else { lock() set(nil) // or set(error) unlock() } } } unlock() } where loading / error markers can be stored as a userInfo attribute of a cached entry.

I see, thank you. That it worked under simulator gave me a false hope and I was following the fool's gold for quite some time now :-( I need to generate an image for a remote video URL (without downloading the whole video upfront). Natural way to do this - using AVAssetImageGenerator. The attempt to use thumbnail extension for that purpose (along with my custom scheme involving creating a temporary local file with a custom extension, and passing the original video URL to the thumbnail extension) was done to run AVAssetImageGenerator in a process of its own, as I found that calling AVAssetImageGenerator's copyCGImage / generateCGImagesAsynchronously on a background thread still causes some unwanted activity happening on the main thread 😡. As I can not just spawn a new process directly under iOS (can I?): is there any other app extension suitable for the task of using network + asset image generation code and somehow returning the resulting image to the main app process? is there any other way to mitigate AVAssetImageGenerator unwanted main thread usage issue?

filed the relevant bugs against hasBytesAvailable & JSONObjectWithStream. it's also quite easy to reproduce JSONObjectWithStream's hang with a custom InputStream subclass (details in FB9531840) but it looks like the main culprit is hasBytesAvailable returning false when it should not (FB9531304). fwiw, the open source version at swift-corelibs-foundation is unfinished (according to its status page "jsonObject(with:options:) with streams remains unimplemented") so not surprising it's incorrectly tries to decode a partial json without collecting all the bytes until the eof first.

Quinn, a few days ago i implemented my solution without relying on input streams and JsonSerialiser (in short i had to parse a sequence of JSON structs and realized i can't use JSONDecoder or JSONSerializer out of the box, but have to preprocess the data first to split it into individual json chunks.) what's still puzzling me is this blocking behaviour (even if i don't use it now). i want to get to the bottom of the blocking issue regardless of json serialization as it reveals my lack of understanding of stream API and it may harm me in a different context one day, so please bear with me. can you please comment on these claims? (a forenote: the stream in question is obtained from the url session task using the sample code above. so it is a true asynchronous network stream.) apple's version of JsonSerialization.json(with stream) tries to read until EOF... but fails to do this correctly (and thus blocks), and i think i found why, see 4 and 5 below. open source version of JsonSerialization.json(with stream) doesn't even try to read until EOF (and thus attempts to parse a partially read json data). my "quick & dirty" modification on top of open source version reads until EOF and seems to be working alright. ignore the bad "busy waiting" practice for now, i'm not going to use it in production just trying to simulate the blocking behaviour observed with apple's code. finally when i added the extra hasBytesAvailable check to (3) i can reproduce the blocking (great!), thus i suspect this is what apple code is doing internally that causes the block. the blocking itself is quite interesting phenomenon though. (see ** below) this behaviour of hasBytesAvailable looks like a bug. at the very least if it is clueless for some reason - it should be returning "true" instead of false to be more in-line with the mentioned documentation comment: "hasBytesAvailable... May also return true if a read must be attempted in order to determine the availability of bytes." the documentation comment (and the corresponding implementation) would be more sane if it was: "hasBytesAvailable... MUST return true if a read must be attempted in order to determine the availability of bytes". i mean, with the "MUST" - the code outlined in 5b is "correct" (and is as good as 5a). with the MAY" - the code in 5b is not correct to begin with... because: "yes, hasBytesAvailable MAY return true if it doesn't know the answer - so the subsequent read could be attempted. but at the same time hasBytesAvailable DOES NOT HAVE TO return true (even if it's clueless), so it MAY as well return false, and so the subsequent read would not even have a chance to be running at all" the very idea that "hasBytesAvailable" might not know the answer better than "read" (and thus has to return spurious "true's") deserves some serious justification... i mean, can it not be reimplemented as a wrapper on top of whatever read is doing inside, just without reading? **) putting it here as it screws the numbering. it looks like the code of this form works alright (pseudocode): 5a while !stream.eof { stream.read() // might return 0 bytes } while the code in the form blocks forever: 5b while !stream.eof { if stream.hasBytesAvailable { stream.read() } } as if "hasBytesAvailable" is improperly returning "false", even if the read could have returned some data.

possibly related: hasBytesAvailable true if the receiver has bytes available to read, otherwise false. May also return true if a read must be attempted in order to determine the availability of bytes. the documentation says "may also return true", not "should" nor "must". which i interpret as "even if read must be attempted in order to determine the availability of bytes, "hasByteAvailbale" may still return "false"... which is quite confusing behaviour to me. it would be much less confusing with the "Must return true...". and that "hasBytesAvailable" can't return the answer sometimes (but the "read" can) - is also quite confusing behaviour, in its own.

if i add an extra "hasBytesAvailable" check i can replicate the built-in jsonObject(with: stream) behaviour, and block forever: var status = stream.streamStatus while status != .atEnd && status != .error && status != .closed { repeat { if stream.hasBytesAvailable { // *** THIS CHECK let buffer = try [UInt8](unsafeUninitializedCapacity: 1024) { buf, initializedCount in let bytesRead = stream.read(buf.baseAddress!, maxLength: buf.count) initializedCount = bytesRead guard bytesRead >= 0 else { throw stream.streamError! } } data.append(buffer, count: buffer.count) } } while stream.hasBytesAvailable status = stream.streamStatus } return try jsonObject(with: data, options: opt) i don't completely understand why this version blocks though. the stream is getting populated (and doing so from a different queue for that matter), so i would expect that even if hasBytesAvailable is "false" temporarily and is false for a few subsequent iterations (or maybe a few million iterations) eventually it shall become "true" once the stream is populated with the next chunk of data, so eventually this loop shall progress to the end. but that's not happening.

the Foundation JSON parser is not a streaming parser; you have to present the parser with all the JSON data at once hmm. when i called the method from the link you provided (instead of the built-in "jsonObject" method): extension JSONSerialization { // renamed to "jsonObject2" to avoid collision with the built-in method class func jsonObject2(with stream: InputStream, options opt: ReadingOptions = []) throws -> Any { var data = Data() guard stream.streamStatus == .open || stream.streamStatus == .reading else { fatalError("Stream is not available for reading") } repeat { let buffer = try [UInt8](unsafeUninitializedCapacity: 1024) { buf, initializedCount in let bytesRead = stream.read(buf.baseAddress!, maxLength: buf.count) initializedCount = bytesRead guard bytesRead >= 0 else { throw stream.streamError! } } data.append(buffer, count: buffer.count) } while stream.hasBytesAvailable return try jsonObject(with: data, options: opt) } } it throwed "Unexpected end of file while parsing object."... but that's totally expectable... the loop breaks by getting zero, which indicates that "the end of the buffer was reached". and it doesn't try to read the whole data to the EOF before feeding it to json decoder, which obviously makes json decoder unhappy. then i simply wrapped this "repeat loop" within another loop: var status = stream.streamStatus while status != .atEnd && status != .error && status != .closed { repeat { ... } while stream.hasBytesAvailable status = stream.streamStatus } return try jsonObject(with: data, options: opt) and that actually worked! (although this busy waiting is still not the bestest thing to do.) looks like built-in "jsonObject" is doing something more naughty which causes it to block. bug?

also the CPU goes to 98 - 100% when it blocks.

the sample code below. it normally works fine after the first blocking (i.e. after i kill the app and launch it again). i guess this is some sort of caching, despite of the "reloadIgnoringLocalAndRemoteCacheData" flag. to defeat this caching just change the url ever so slightly (e.g. increase pageSize by 1). the app works normally with small page sizes. note that i used this particular URL just as a guinea pig for the test. import Foundation class C: NSObject { private var session: URLSession! private let someSerialQueue: DispatchQueue private let otherSerialQueue: DispatchQueue private let someOperationQueue: OperationQueue override init() { someSerialQueue = DispatchQueue(label: "someSerialQueue") otherSerialQueue = DispatchQueue(label: "otherSerialQueue") someOperationQueue = OperationQueue() someOperationQueue.maxConcurrentOperationCount = 1 someOperationQueue.underlyingQueue = someSerialQueue super.init() } func test() { let pageSize = 90 let testEndpoint = "https://api.github.com/search/repositories?q=created:%3E2021-08-13&sort=stars&order=desc&accept=application/vnd.github.v3+json&per_page=\(pageSize)" URLCache.shared.removeAllCachedResponses() var request = URLRequest(url: URL(string: testEndpoint)!) request.timeoutInterval = 5 request.httpShouldUsePipelining = true request.cachePolicy = .reloadIgnoringLocalAndRemoteCacheData session = URLSession(configuration: .default, delegate: self, delegateQueue: someOperationQueue) let task = session.dataTask(with: request) task.resume() } } extension C: URLSessionTaskDelegate { func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: Error?) { if let error = error { print("TaskDelegate: didCompleteWithError: \(error)") } else { print("TaskDelegate: didComplete with no error") } } } extension C: URLSessionDataDelegate { func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, willCacheResponse proposedResponse: CachedURLResponse, completionHandler: @escaping (CachedURLResponse?) -> Void) { completionHandler(nil) } func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive response: URLResponse, completionHandler: @escaping (URLSession.ResponseDisposition) -> Void) { let status = (response as! HTTPURLResponse).statusCode print("DataDelegate: didReceive response. status: \(status)") completionHandler(.becomeStream) } func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didBecome streamTask: URLSessionStreamTask) { print("DataDelegate: didBecome streamTask") streamTask.captureStreams() } func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive data: Data) { fatalError("DataDelegate: shall not happen") } } extension C: URLSessionStreamDelegate { func urlSession(_ session: URLSession, streamTask: URLSessionStreamTask, didBecome inputStream: InputStream, outputStream: OutputStream) { print("StreamDelegate: didBecome inputStream / outputStream") inputStream.open() otherSerialQueue.async { print("b4 JSONSerialization.jsonObject") let object = try! JSONSerialization.jsonObject(with: inputStream, options: []) print("after JSONSerialization.jsonObject") let len = "\(object)".count print("got JSON, \(len) bytes") } } } let c = C() c.test() while true { print("run loop run") RunLoop.main.run() }

where is the best place for SwiftUI questions?

[1] That’s not quite true. In dns_sd.h you’ll find DNSServiceNATPortMappingCreate, which may yield useful results in some circumstances. However, it does not work in all circumstances — for example, it generally fails on WWAN — and thus won’t meet your needs. do you mean that DNSServiceNATPortMappingCreate does not return an external IP address on WWAN or that it does not work at all on WWAN (so it is not possible (or reliable) to use PCP / NAT-PMP functionality for a WWAN interface)?

another illustrative example why the ability to get the current queue might be needed. let's assume the project adheres to these rules: rule #1 calls are executed either on the main queue or on one of the queues i've created explicitly. rule #2 there is no sudden queue hopping from call to call, any queue hopping if needed is explicit. } carrying the currentQueue parameter from call to call would be too noisy, imho.