It's telling you that your texture descriptor settings are incorrect relative to your other settings. Notice how the number is exactly 1/2. Based on the limited information you wrote, it is expected because you went from BGRA8 to RGBA16, which is twice the size.

I missed your link earlier... I'll take a look tonight, did you first double check that isJitteringEnabled is in fact working as of today in a scenekit project without the rest of that sample project? SceneKit is notorious for having bugs. So you should confirm that first. Then you should try changing Scenekit to rgba16 without the rest of that sample project. If you run into the same error, you'll know that you still need to change the texture settings.

I just checked the project files... It is as I stated in my first reply, you need to ensure the texture settings match, and then you will not get the error regarding the textures. After you change to rgba16 in both places (such as kCVPixelFormatType_64RGBAHalf), you will then be confronted by another texture setting match problem for the texture in the second attachment. Since you are overriding the render pass descriptor for SceneKit, it would be logically assumed that you then have to create and setup that texture on your own. If you spend more time with Metal, these things will be more obvious as they are very normal things. I will leave that to you to complete, but I will say that the entire thing is clunky, extremely inefficient, and I would not personally recommended following this sample code's strategy or even using SceneKit or Swift. The use of AVAssetWriter is also not recommended. These, in their current form today and in the past, are Apple's half-baked attempts at providing convenience with a heavy price. There is little expectation based on Apple's prior results, that the standards of quality will change anytime soon. (Even though I hope for that every day) I would instead recommend avoiding as many Apple frameworks as you can, and becoming more familiar with Metal and C, and get to your goals that way.

Simple answer: You pass your buffer to your kernel, write to your texture, and then render the texture on screen as appropriate for your specific circumstance. If you are performing intensive calculations, then the answer to 'what is the boilerplate for me', would be decided based on the specifics of your situation. You mention that you don't need vertex shaders and primitives, but those are in fact, how you would get the texture onto the screen. And depending on your specific use case, will change how you architect the pipeline in your cpu code - and that part may be large for intense and complex calculations. Aside: I find, the metal developer community feels small, with a minority who are familiar with the entirety of it, and even Apple employees can be found to be unknowledgeable about simple things. (I had to find out the hard way - by building endless tests to measure and assess each thing myself.) Because of this, you are unlikely to find quality sample code anywhere online. Even if you do, the specifics of that sample may be incorrect for you. Frequently, when you want to do one thing in combination with other things, you have to change how you do both. Because of this, when asking for a boilerplate for one thing, it is likely not what you will truly need for the real use case - causing you to either ask a second time for something else, or settle for something significantly inefficient just because that's what you got working. That isn't a great approach. And there are enough options in the possibility space, that I wouldn't be able to predict exactly what applies to you without a very thorough description of everything. The more specific and clear you are about the entirety of your application use case, the more likely you will be to get not just an answer, but one that is the most correct answer for you in that specific circumstance. However, an even better position for you to take, would be to less be concerned with trying to achieve your goal, and more concerned about trying to understand the fundamentals of the system more thoroughly, so that you could design architectures on your own, and know why things need to be a certain way. Then, when you came to ask for help, your questions would be less related to a lack of understanding basic things, and more related to Apple's bugs or the cutting edge algorithm you're trying to implement. If we can foster that level of thoroughness in the developer culture outside Apple, then we might see Apple follow along and adopt a culture of higher standards in their foundation software and hardware.

It might be the distinction between linear/sRGB/gamma. If it is, you could just change your pixel formats, or perform the appropriate equations in the shader.

One part of the reason for this, is that using Swift is ironically slow and inefficient compared to C, and not appropriate if your real goal is performance and maximizing the capability of the hardware. In most all serious scientific applications, games, art tools, etc. where the Metal is to be used, there is a large amount of memory and data that is to be managed, transformed, and passed around to and from the cpu and gpu. Using Metal itself might imply the developer is trying to get the full performance or stability possible at the sacrifice of ease of use or any other benefit the Swift coder might believe they are gaining. So to combine something inefficient like Swift with Metal is a bit of strange practice for a use case where we sincerely are maximizing the potential of the hardware. What exactly is the developer trying to achieve by going only part way? Perhaps they are just following the cultural fads or caught up in novelty, and not measuring, or perhaps they are doing something where they just do not care about the performance consequences. Most of the Swift architecture and syntax choices over the years have reflected strange preferences of the community, that don't align with clear rationality, so I don't think it's something the community can readily engage a logical dialog about, without also confronting the irrationality of many ideas that were already adopted. In any case, the typical lesson here, is to avoid the tendency to try to learn by example - not to learn by reading other people's code, but to come to understand the fundamentals. Then you will understand how to build anything, and clearly see the limits, problems, and incorrect practices of publicly published examples.

Yes. The gpu debugging tools have had numerous issues throughout OS releases. The cause is merely quality assurance problems. - And it infects every corner of the company. The way we can resolve it, is by raising standards, and making it clear to the staff that their current work is not acceptable. If we say, it is not acceptable, and if they truly care and are not disingenuous, then the managers would then be more critical and reject more of the sloppy and half-baked work. It means that the developer culture itself has to become more sincere about it's standards as well - so as to be more thorough before submitting work. A lot of this comes from the way people go about achieving the results - not just the results themselves. Accepting that there are formally right ways to do things, and that the current culture inside and outside the company is wrong, would be the first step we could do today. Beginning to practice the right ways are also things that can and should be done today, and not put off like, "It is a nice idea, but I don't feel like working that hard, and it doesn't matter that much, it is already quite good, so I wont do it that way today." I often hear this and similar patterns, where the teams take attitudes of mind where they pat themselves on the back, and tell each other that it is good, and further validate themselves by positive reviews - (without taking into account the dynamics of reviews in a culture that is trying overly hard to be positive.) As soon as the team says, "We think it is good and we worked very hard" - It is over in that very moment, they have blocked themselves from looking at the core problems and fixing them. What we have to do, is be perfectly clear, that "No, it is not good. It is not meeting the minimum standard of quality", and that we have to provoke them to confront the reality of how bad things actually are, so that they will take them seriously enough to fix them. Because the third party developers often don't complain as clearly, nor do we organize ourselves together to communicate effectively, and because the guys at the top of the company don't care enough, these problems will go on for years until we put our foot down and be more strict about what quality actually means, and how much we actually care to get it.

To extend this a little more... (As a side note: It should go without saying, that even though you can call C from Swift, it is not the same and there is overhead.) I almost always write software tools that try to be as efficient as possible, but they still hit the hardware limits for realtime interaction, and I need it to do more. Because of this, in a real tool, it never makes sense for me to use slower languages and existing frameworks for UI either. The performance gain from rolling custom UI at the same level as the rest of the Metal code is really significant. Especially, if you are using any of the features of Swift that people do like - because those actually come at the highest cost. I write most all my code in raw C, and only use Objective-C for Metal calls, because I am trying to get the most out of the hardware, at the sacrifice of making things convenient for the programmer. It's not like I enjoy this at all, I just accept the suffering in order to get the final tool to be better. I ended up that way, because I wasn't getting as good results the other ways. The initial brainstorming ideas that lead to Swift 1 and playgrounds, reflected concepts like Bret Victors thoughts of interactivity. But the ideas were greatly misunderstood. Ideally: the final compiled machine code, should be the most optimized and stable as it can, because the end result is the first priority the second priority is how we interact with the machine as a developer, and to distinguish features of the text language as distinct from the IDE - Swift's core implementation misunderstood this - even though LLVM's AST format is completely able to implement the proper solutions if the people had thought this through correctly. This was obvious all along, but it goes back to the cultural problems and miscommunication. (After all, I believe C uses the same underlying AST format, does it not? The failure of all other languages to achieve the same level of performance, comes from how people think about the languages/interface, system, and goals) There are also additional moral concerns where performance is actually related to physical energy consumption and environmental damage. If we take the responsibility for the impact of our efficiency choices as developers as a whole, then it means confronting that as well. If millions of coders take the position of thinking it doesn't matter so much (let's take for instance the wide spread adoption of Unity and Unreal), then as a whole, the impact is quite large, so it becomes something we'd have to address collectively as a culture. It's similar to how, if everyone litters only a little it is a big problem in the whole - though each person will say "It's just one".

Link a minimal project that demonstrates this - then I can review it and confirm or deny whether it is your error, or Apple's.

It's more a consequence of the problems of scale: As you increase the scale and number of parts, communication slows down. The fast memory on the chip used for tile memory is smaller. TBDR is about taking advantage of faster memory access with smaller amounts of data that can fit in smaller amounts of physical memory. As far as I am aware, for desktop gpus, there was only one line of Nvidia cards that had something similar. If you run a compute kernel on the M1, you would expect it to be more efficient in memory access in this regard compared to the other gpus. However, in situations that go beyond the M1's storage and processing ability, of course, you would see a point where the overall results would seem like it is underpowered. This is how you know what strategies and use cases are more appropriate for the M1 vs an eGPU. An eGPU is slow in communication no matter how powerful a card you put in there, so it really isn't appropriate for smooth interactive rendering views. It's more appropriate to use eGPUs for non-realtime offscreen rendering. The many other pipeline benefits of TBDR will come into play if your use case is something that really matches what it is best at - but that may often not be the case, and you may still need multiple render passes.

Please send an actual xcode project for analysis. What you uploaded has telling signs that some of this is Apple's fault, and some of it, like #3 should be an expected error. But if you want someone outside or inside Apple to try to help, you'll need to spoon feed everyone a normal Xcode project. On a normal day, they are inadequate to address basic problems and do not test thoroughly, but they are even less inclined to help if you send it in this format.

Have you confirmed that the assembly code, OS, etc are all identical? (And even before this, did you verify with the gpu debugger that everything is identical? Such as the pixel format and size? Did you verify it against a non-Swift implementation?)

No, what you provided is not enough. It is not merely tracking api calls. You should provide a project that compiles to allow them to use the full diagnostic tools that are unavailable with just the binary. If you don't provide them this, then the other party has to write it themselves. Often, this causes several tangent problems to occur during triage, that delays identifying and resolving the true problems. (As opposed to the misconstrued notions of what the problems are thought to be) These things occur unnecessarily, and you can do something about that today. You can go to File -> New Project in Xcode, and make a minimal project that replicates what you are seeing in your main project. I am available to provide a second look on your work today to confirm without doubt the issues, but if you neglect to make the sample project and provide this, it will sit on the shelf further. After you have sent this to me, and I have confirmed it's entirety, we can both submit crystal clear reports, to make the complaint more effective. (Also, in case it isn't obvious, the projects you submit should be in Objective C, not swift or C++, and they genuinely should be the minimum that depicts the bug without dependencies.)

I see, I would also recommend writing a test that does not use a CVPixelBuffer at all. This is because it is important to identify if the culprit is reading from the texture or writing to the buffer. You can do this easily with C inside the Objective-C version. Simply call malloc with the size of your buffer, and pass that pointer to getBytes instead of the CVPixelBuffer. If you find that there is no slow down, you will know for certain that the issue is with CVPixelBuffer's delay and not getBytes. Then you can inform Apple about this in the report. Send me a copy and I will confirm that the test code was written correctly.

That is because blend modes of that kind can only be done manually in your fragment or kernel, not in the pipeline descriptor options. You have to write the math for it yourself and structure the render passes appropriately to perform the compositing. Also fyi, you are going to get banding with that pixel format.