For everyone who also has this Problem (capturing ARKit video image and using it in your frag shader turning out too bright): (This example is a part of my IOS app, so aspect ratio and such could/should change for other devices) The conversion vom YCbCr_srgb to RGB conversion is a 2 step operation. First you have to matrix-multiply your color with this YCbCr conversion matrix. constant const float4x4 ycbcrToRGBTransform = float4x4(     +1.0000f, +1.0000f, +1.0000f, +0.0000f,     +0.0000f, -0.3441f, +1.7720f, +0.0000f,     +1.4020f, -0.7141f, +0.0000f, +0.0000f,     -0.7010f, +0.5291f, -0.8860f, +1.0000f ); The second step is to adjust your gamma. So you end up with something like this: fragment float4 myFragmentShader(fragmentInOut in [[ stage_in ]],                                  texture2d<float, access::sample> capturedImageY [[ texture(0) ]],                                  texture2d<float, access::sample> capturedImageCBCR [[ texture(1) ]]) {     constexpr sampler s(mip_filter::linear,                         mag_filter::linear,                         min_filter::linear); // in.displayBounds and in.position is calculated in the vertex shader, 0.615764 is aspect ratio of the video image size     float fx = (((in.displayBounds.x - in.position.x) / in.displayBounds.x) - 0.5) * 0.615764 + 0.5;     float fy = (((in.position.y) / in.displayBounds.y) - 0.5) * 1.0 + 0.5; // Flip them around, since we're in portrait mode here, for full device orientation awareness, you have to add a bit more logic const float2 tp = float2(fy, fx); // Sample the color from the captured video frame const float4 ycbcr = float4(capturedImageY.sample(s, tp).r, capturedImageCBCR.sample(s, tp).rg, 1.0); // apply the YCbCr to RGB transform matrix and apply the gamma correction const float3 rgbColor = pow((ycbcrToRGBTransform * ycbcr).rgb, 2.2); // Now if you return this color, it will look exactly the same as the video background return float4(rgbColor, 1.0); }

Or what could help me too is a conversion matrix for YCbCr (srgb) to RGB (not SRGB). Or a conversion from SRGB to RGB.

Ok, I think my question should be: How do I change the color space in a SwiftUI app (IOS)?

Ok, seems my answer didn't fix this problem at all. It is still too bright. Which texture do you refer to when you talk about the background texture? Color 0? (I use SceneKit/SCNProgram to get my stuff on screen)

Yeah, my bad. It is loading correctly and displaying correctly. But my last question still has issues. I'll continue there.

Spot on, thanks a lot :) import ARKit public class MyARScnView : ARSCNView {     public override var colorPixelFormat: MTLPixelFormat {         get {             return MTLPixelFormat.bgra8Unorm         }     } } fixed my issue