I did file FB 22349771 (space intentionally added to avoid moderation flag).

Interesting. I didn't realize that. However, it doesn't seem to always work. Yea it doesn't always work. I have a fairly solid hack workaround in place to make it work in my app, which was a bit tedious to put together. There are a few rules I discovered through trial and error. Checking for "new custom icon" not "tahoe' custom icon etc. And a few other tricks for dealing with some other issues. I was a bit proud of myself. I discovered a few folder locations where Finder got the icons wrong but my app got it right. Wasn't meant to last though. I see. My workaround still works, except for the fact that folder colors in my app are now wrong. Kind of a big deal. Been working on this update for a while now and I'm finally close to finished..but just when I thought I was out they pull me back in. I mean I guess I could do the drawing and tinting, etc. myself but really that's like another day at least. Then testing. Sorry I'm not vibing yet with the rest of them. Guess I have to decide whether to go down that road or stick my fingers in the dike and pray they don't take like 6 months to fix it. But all is not bad in the world of QL. The Quicklook slow loading of thumbnails for app icons, seems to be fixed. Before those would take like 30 seconds. If I could trade bugs I'd take that one back! Folder color is more important!

I see that too. Luckily, I only use the new QLThumbnailGenerator flow if the url has the "com.apple.icon.folder#S" extended attribute. I'm sure that's an "implementation detail", but still a good idea. How are you getting the icon otherwise? NSWorkspace gets the folder color right but it gets other things wrong. Quicklook thumbnailing gets the other things right, but now it gets the folder color wrong. A non-empty folder results in an icon with paper sticking out on macOS Tahoe but an empty folder doesn't have the paper sticking out. NSWorkspace seems to get everything right on 26.4 (it didn't before otherwise I would have went with it) but only when a custom SF symbol/Emoji is set as far as I can tell. And Finder has its way of being exempt from all of this, natch!

I file FB 22340592 (space intentionally added because posting FB links seems to cause posts to be flagged until approved by a moderator, suddenly).

Not really. That is, you might be able to subclass the control to get the behavior you want; however, I think the work involved would be fairly similar making your own control using our view components. True. Yea it isn't too much work. Just needed to fetch the titles and the icon images in the background. but CFURL does NOT, I briefly thought about this though I wasn't sure how path control grabs the icons (through the url or something else like NSWorkspace) and I wanted to have that in the bg too. Under normal conditions this isn't a big deal but dealing with a url on a slow network it can cause a little hitch, as I discovered!

FB22294400

Also, are you sure the Finder actually copied the file (it may very well be skipping these files). Finder can copy it. I put it directly on the pasteboard in my test. In fact I can copy it too, provided I use one of the workarounds described in the other thread. Looks like this turned out to be exactly the same issue as the one described in the other thread. Interestingly .DS_Store causing problems with copying is infamous enough to have a mention on Wikipedia: .DS_Store files have been known to adversely affect copy operations. If multiple files are selected for file transfer, the copy operation will retroactively cancel all progress upon reaching a (duplicate) .DS_Store file, forcing the user to restart the copy operation from the beginning Not exactly the same issue I described, but similar! Does the file have a data fork? it has, data. I can read the filenames when I view it as 'plain text' Resource fork is less than 300 bytes but that is enough for it to cause copyfile to fail. The total file size is ~12kb. I think almost any file with a resource fork is enough to cause copyfile to choke currently due to a bug. Will get around to filing that bug soon.

Mystery (partially) solved. I just took a sneak peak at the extended attributes of the .DS_Store file and some have a resource fork (probably the ones that were failing to copy I BET!). So appears to be the same failure we are talking about here: https://developer.apple.com/forums/thread/814076 .DS_Stores on the NAS's with resource forks.. maybe that will help you figure out leaky compression

Interesting and really good information to know! IF this is in fact a compressed file, then just copying the resource fork won't actually work. There's an additional UF_COMPRESSED file flag that needs to be set, which then "marks" the file as compressed so that the system knows to retrieve and decompress the file contents when the file itself is opened. If you read through the previous threads, you can see that this flag is what ends up "hiding" the presence of the resource fork from the rest of the system. However, if it was a compressed file then it should never have been written to the NAS device as a compressed file. All of our copy engines would have either automatically decompressed it (this is what happens if you copy the file without "knowing" about file compression) or noticed that the target DIDN'T support compression... and decompressed the file. "Blame" is definitely the critical factor here. The worst-case scenario I'd be concerned about is that the files were already damaged when they hit the NAS and UF_COMPRESSED was stripped. You then copy the file "properly" (replicating the file as it exists on the NAS) and then get blamed for both the bad copy AND damaging the original (since you were the last one to "touch" it). [...] My personal inclination would probably be to use Carbon, as that gets you closest to "do what the Finder does”, which is the benchmark most developers are concerned with. FYI, you can "fix" these files (by setting UF_COMPRESSED), but that is something the user would need to be involved with, not something you can do automatically[1]. In regards to the potential compression situation, I'm assuming Finder copying doesn't involve the user or set UF_COMPRESSED? Or does it? If it just copies and the file, and that file remains 'unusable' then I guess copyfile's behavior (failing) would be better in that case. Now if there is someone out there just hoarding really old files with resource fork, Finder's behavior is obviously superior. All it takes, as you demonstrated, to get copyfile to fail is to feed it a file with a resource fork. And the open source code indicates that it at least tries to copy it. So I guess I just want to copy the data the user feeds me. Whether the file is already broken due to compression leak etc. that's outside my app's bounds. My app isn't a "data repair tool," it isn't in that genre, so I just am going to have to settle for copying whatever I get from the pasteboard. In the case of a file with a resource fork, I guess I'd hope it is a very old file from ancient times and not a broken file. It'd be great if Finder shared the same copy engine with the rest of us, at least whatever issues would be consistent system wide. But I get it..it probably won't ever happen.

I think the idea you're getting at here is that you should be able to point your copy engine at any point in the file system, have it do the "right" thing based only on the information it's provided by copyfile, not external knowledge (like the full source path). [...] That's a lovely idea, but for better or worse, macOS just does not work that way. [...] but if you want to work with the "full" system... Well, I don't think there's any way to do without grappling with a whole bunch of edge cases. Well I did not directly point copyfile at .DS_Store and tell it to do the thing. I started a recursive copy on a folder, which happens to have a .DS_Store file in it. And on macOS just about every folder the user views in Finder will have one of those hidden in there, unfortunately. The folder I was copying was on a NAS, has like 24 videos in it. I'm doing this intentional slow copy to test everything out. So the copy gets like 98% done and then it needs a diaper change on the .DS_Store file! I'd ignore all .DS_Store errors and, potentially, just skip copying them all together. I have no intention of including .DS_Store in the copy. I am ignoring it now. I'm not really sure if they should even exist and probably shouldn't passively travel in a copy (maybe with the one exception when an advanced user is viewing hidden files and actively puts it in the pasteboard). It's okay with me if I can't do the copy on DS_Store, I guess. I generally want my app to do what the user is trying to do as long as it can't hurt them. My main issue is it doesn't report the reason why I can't do the copy. In fact sometimes I can do the copy but similar to the issue in the other thread it seems that when the .DS_Store is on the NAS, copyfile fails and that may have something to do with it. But I can copy .DS_Store that are on the local drive (perhaps because of the clone option I dk). And yes Finder can copy it, even from the NAS. So I guess if I really wanted to copy it I might be able to dust off that old Carbon File Manager (but I actually don't). Mainly my concern is cryptic failures even though I am primarily dealing with user facing files in reasonable user-facing locations. I'm not really trying to travel to the underworld. In any case if my app doesn't have the right privileges to copy .DS_Store or whatever (which are littered all over the file system) a normal error, even one with no recovery options seems appropriate. I really want to know if it failed because of something else and not permissions (perhaps because the way the data is stored). If there is another file I come across that stores data in a similar way...and the user would reasonably WANT to copy that file, that's actually what I'm more concerned about than .DS_Store.

Just read your reply again more carefully as it is not the end of the day and I'm not tired :) . As the thread up above shows, the primary way to bypass the issue "copyfile" is having is to use the "..namedfork/rsrc" convention, Interesting. I was using xattr API. So just skipping the resource fork during copyfile, then running copyfile again over the src and destination (with .namedfork/rsrc convention) seems to work on my dummy test file. Is that expected to be a reliable workaround? I guess if this actually works reliably I wouldn't have to restart copy from scratch or suppress deprecated API warnings. Or maybe there good reason to start the copy from scratch with the Carbon file manager?

Have you filed a bug on this and, if so, what is the bug number? Not yet, but I do have another thread where I describe the same problem in regards to .DS_Store: https://developer.apple.com/forums/thread/819041 I will post the bug number (most likely there) when I carve out a little time for it. Currently if copyfile fails and errno is not set my code falls back to a kind of 'Unknown Error' which I hate but if the user is counting on my app to perform the copy it's kind of a gamble if the operation doesn't succeed to just hope not critical data was lost. A big part of this is that xattrs aren't actually "safe“— the system doesn't have a documented system for defining how they should be handled, and app/copy engine/file format support for them is inconsistent, so storing critical data in them can end up being a good way to lose critical data. Importantly, that kind of data loss isn't really a "bug" in the same way other kinds of data loss would be. The general "point" of xattrs is to allow the attachment of arbitrary metadata, but part of that contract is that it's not really part of the "core" file data. This was always my understanding of xattrs. The resource fork WAS originally intended as a larger-scale data storage location, which meant repurposing it was substantially less disruptive (and required a lot less work) than creating a new "bulk" xattr would have been, particularly in the context of HFS+ (where they were originally architected and introduced). Yea my concern is if these files are really old the user may be keeping them around for a long time because they consider them to be important. So I wouldn't want to be blamed if they copied the file in my app, discard the original, and the copy is corrupted. Like the Carbon framework, resource forks were before my time though. The answer depends on what you're trying to do. Basically what I'm trying to do is avoid failure. If I can reasonably and gracefully handle the issue and not present an error, that is best. At the same time, I feel a responsibility to the user. If I don't show an error, the copy should have worked and I don't want to silently hide true errors. Another less important need (but still kind of important), is to keep the implementation as simple as possible. I don't want to rewrite copyfile as I already got like three billion lines of code in my app and I have tons of other stuff to tackle. I don't see much return on investment doing that. Basically my workaround currently is if I run into a resource fork in the copyfile callback, I set a flag and skip it. Then when copyfile finishes (if successfully) I just copy the resource fork over from source to destination. If that sounds like a bad idea, let me know and maybe I'll just fallback to the Carbon file manager instead.

Sorry for the double post. But just a quick additional question. Am I better off working around the issue by retrying with the Carbon file manager or should I just skip the resource fork in the copyfile callback, then copy it over myself at the end?

I'm experiencing this issue. Copy file works locally for these files but fails when coming from a NAS. Works from Finder in both cases, as previously described above. copyfile does not seem to provide a usable error. The callback is invoked twice and I get : what == COPYFILE_COPY_XATTR stage == COPYFILE_START Then the callback is invoked a second time with: what == COPYFILE_COPY_XATTR stage == COPYFILE_FINISH In both cases my callback function returns COPYFILE_CONTINUE. Then the copyfile function returns -1, errno is not set. Not getting a usable error unfortunately forces kind of jackhammer workarounds. Have you noticed this issue for files without resource forks (but use other xattrs) or is the resource fork the main cause?

That does the thing! Thanks! And, yes, we probably should have documented all of this, so PLEASE file about this and post the bug number back here. I filed FB22260532