Why are you using a CAMetalLayer if you don't use any feature from Metal? Your example looks like a very uncommon usage to me: in my experience you get red output when the framebuffer isn't even cleared because there is no render pipeline (MTLRenderCommandEncoder) scheduled to render into that drawable CAMetalLayer provides drawable that cannot be written to by default, only rendered into, at least from Metal pipeline perspective. Is your texture storage even shared? I would expect the texture created by CAMetalLayer to have private storage, so you can only write to it through Metal commands.

The only idea that comes to my mind (without knowing the actual transform to do, since it's definitely not uniform) would be to try to use CIPerspectiveTransform.

Your mtkView size is 414x233 points, which is 828x466 pixels with 2x display scaling. By default, the drawable texture size in MTKView follows the view size, which makes sense to render content at the native resolution and perfectly fitting the UIView bounds. So you have two options: force a specific drawable size on MTKView (disable MTKView.autoResizeDrawable and set MTKView.drawableSize to whatever you want). Texture will have the correct size but it may look weird on display. render to a MTLTexture that you create and own (thus with the size that you want) instead of using the one from the drawable

Did you profile to see what was the bottleneck in the CoreImage implementation and where the bottleneck is now?

In Xcode you can define a symbolic breakpoint on -[CAMetalLayerDrawable texture] to break each time this method is called (including the valid calls). This can help you find where this is used in your app.

if the regression calculation in Metal takes too long After how long do you see it failing? Do you also have a requirement to hardcode the threadgroup size? Or did you notice that it performed best with 16x16 ? https://developer.apple.com/documentation/metal/compute_passes/calculating_threadgroup_and_grid_sizes provides sample code for dynamically computing this size. As a side note, although I don't expect this to be the core of your issue, you shouldn't instantiate the compute pipeline state as part of the process() function. This needs to be done only once, not for each process() call. This state should be created beforehand or at least cached.

I expect what you're seeing is your flags being applied to all shaders while they should only be applied to the Core Image ones. https://developer.apple.com/videos/play/wwdc2020/10021/ describes a way to apply the Core Image flags to Core Image shaders only, through custom build rules. Alternatively you can add your CI shaders in a distinct target.

hi, Could you share a full and self-contained example project? ideally zipped or hosted I happen to have almost the same hardware configuration and could give a quick look at it. In the meantime I recommend profiling your program with Instruments (Metal one) to see what happens.

How much work do you queue in advance? Can you have at least 30ms of GPU work queued at any given time for instance?

It looks like you're looking for the simd_float2 type from <simd/simd.h>.