This setup works, tho I get: "frameBufferOnly texture not supported for compute." Update: metalLayer.framebufferOnly = false got rid of this error.

Thank you, that worked. I can now sample the layer: half4 color = layer.sample(float2(x, y));

This works! Also remember to set the info.plist key "CADisableMinimumFrameDurationOnPhone" to true.