MetalFX FrameInterpolator assertion: `Color texture width mismatch from descriptor` even when all texture sizes match

Graphics & Games Metal
ovenxp OP
Created 1w
Replies 4
Boosts 0
Views 523
Participants 2

I am integrating MetalFX FrameInterpolator into a custom Unity RenderGraph–based render pipeline (C++ native plugin + C# render passes), and I am hitting the following assertion at runtime:

/MetalFXDebugError.h:29: failed assertion `Color texture width mismatch from descriptor'

What makes this confusing is that all input/output textures have the correct width and height, and they exactly match the values specified in the MTLFXFrameInterpolatorDescriptor.

Setup

Input resolution: 1024 x 512

Output resolution: 2048 x 1024

MTLFXTemporalScaler is created first and then passed into MTLFXFrameInterpolator

The TemporalScaler and FrameInterpolator descriptors use the same input/output sizes and formats

All Metal textures:

Have no parentTexture

Are 2D textures

Match the descriptor sizes exactly (verified via logging)

Texture bindings at encode time frameInterpolator.colorTexture = mtlTexColor; // 1024 x 512 frameInterpolator.prevColorTexture = mtlTexPrevColor; // 1024 x 512 frameInterpolator.motionTexture = mtlTexMotion; // 1024 x 512 frameInterpolator.depthTexture = mtlTexDepth; // 1024 x 512 frameInterpolator.uiTexture = mtlTexUI; // 2048 x 1024 frameInterpolator.outputTexture = mtlTexOutput; // 2048 x 1024

All widths/heights are logged and match:

Color : 1024 x 512 (input) PrevColor : 1024 x 512 (input) Motion : 1024 x 512 (input) Depth : 1024 x 512 (input) UI : 2048 x 1024 (output) Output : 2048 x 1024 (output)

The TemporalScaler works correctly on its own. The assertion only occurs when using FrameInterpolator.

Important detail about colorTexture

Originally, colorTexture was copied from BuiltinRenderTextureType.CurrentActive. After reading that this might violate MetalFX semantics, I changed the pipeline so that:

colorTexture now comes from a dedicated private RenderGraph texture

It is not the backbuffer

It is not a drawable

It is not used as a final output

It is created before UI rendering

Despite this, the assertion still occurs.

Question

Can uiTexture for MTLFXFrameInterpolator legally come from a texture copied from BuiltinRenderTextureType.CurrentActive?

More generally:

Are there additional hidden constraints on colorTexture / prevColorTexture (such as Metal usage, storageMode, aliasing, or hazard tracking) that could cause this assertion, even when sizes match?

Does FrameInterpolator require colorTexture and prevColorTexture to be created in a very specific way (e.g. non-aliased, ShaderRead usage, identical Metal resource properties)?

Any clarification on the exact semantic requirements for colorTexture, prevColorTexture, or uiTexture in MetalFX FrameInterpolator would be greatly appreciated.

Replies  4
Boosts  0
Views  523
Participants  2
DTS Engineer OP
Apple
1w

I don't know the answer to this specific question and you might get better answers from the Unity Discussions, but, I'm happy to give this a try in case any of this is useful. Here are some suggestions/things to check" ~

  1. Verify the actual Metal texture properties, not just the Unity RenderTexture properties. Log mtlTexColor.width, mtlTexColor.allocatedWidth, mtlTexColor.bufferOffset and check if any textures are actually texture views (mtlTexColor.parentTexture != nil) instead of textures.
  2. Check and make sure descriptor dimensions match the temporal scaler exactly. When creating the MTLFXTemporalScaler descriptor, verify input/output sizes and when creating the MTLFXFrameInterpolator descriptor with that scaler, use identical dimensions.
  3. In your Unity RenderGraph, create dedicated, non-aliased textures explicitly for MetalFX (use RTHandles with explicit dimensions, not RenderGraph transient textures) and make sure textures are not pooled/reused across frames with different sizes. Consider using MTLStorageModePrivate with explicit creation rather than letting Unity manage the allocation.
  4. Check and make sure prevColorTexture is actually from a previous frame and has consistent dimensions. Verify these dimensions between frames, to check if the descriptor isn't being recreated with different dimensions

Aside from those items, here are some answers to your questions ~

BuiltinRenderTextureType.CurrentActive is probably not the issue since the error specifically mentions "Color texture."

It is quite possible that there are hidden constraints on colorTexture/prevColorTexture related to how Unity creates the underlying Metal resources. The textures must match not just in apparent dimensions but in their underlying Metal resource properties - including allocated width, storage mode, and memory layout.

I expect FrameInterpolator will expect Metal textures with no aliasing, consistent dimensions frame-to-frame, and matching the descriptor exactly at the Metal resource level, not just the Unity RenderTexture level.

The fact that this works with TemporalScaler alone but fails with FrameInterpolator suggests the interpolator has stricter validation, particularly around the temporal consistency of colorTexture vs prevColorTexture dimensions.

0
ovenxp OP
1w

hear add some debug code, MetalFX FrameInterpolator calling code like this: void MetalFXFrameInterpolator::MTLFXInterpolateFrame(const FrameInterpolatorRenderContextInternal& context, id<MTLFXFrameInterpolator> frameInterpolator, id<MTLCommandBuffer> cmdBuffer) { id<MTLTexture> mtlTexOutput = context.idOutputTexture.IsValid() ? QueryMTLTexturePointer(context.idOutputTexture) : GetBackBufferColorMTLTexture(); id<MTLTexture> mtlTexPrevColor = QueryMTLTexturePointer(context.idPrevColorTexture); id<MTLTexture> mtlTexMotion = QueryMTLTexturePointer(context.idMotionTexture); id<MTLTexture> mtlTexColor = QueryMTLTexturePointer(context.idColorTexture); id<MTLTexture> mtlTexDepth = QueryMTLTexturePointer(context.idDepthTexture); id<MTLTexture> mtlTexUI = QueryMTLTexturePointer(context.idUiTexture);

if (mtlTexColor.width != frameInterpolator.inputWidth
    || mtlTexColor.height != frameInterpolator.inputHeight
    || mtlTexPrevColor.width != frameInterpolator.inputWidth
    || mtlTexPrevColor.height != frameInterpolator.inputHeight
    || mtlTexOutput.width != frameInterpolator.outputWidth
    || mtlTexOutput.height != frameInterpolator.outputHeight
    || mtlTexUI.width != frameInterpolator.outputWidth
    || mtlTexUI.height != frameInterpolator.outputHeight)
{
    NSLog(@"ERR tex width mismatch: %lu x %lu  ~  %lu x %lu ", (unsigned long)mtlTexColor.width , (unsigned long)mtlTexColor.height, (unsigned long)frameInterpolator.inputWidth, (unsigned long)frameInterpolator.inputHeight);
//    return;
}
if (mtlTexColor.parentTexture != nil)
{
    NSLog(@"mtlTexColor parentTex %lu x %lu", mtlTexColor.parentTexture.width, mtlTexColor.parentTexture.height);
}
if (mtlTexPrevColor.parentTexture != nil)
{
    NSLog(@"mtlTexPrevColor parentTex %lu x %lu", mtlTexPrevColor.parentTexture.width, mtlTexPrevColor.parentTexture.height);
}
if (mtlTexUI.parentTexture != nil)
{
    NSLog(@"mtlTexUI parentTex %lu x %lu", mtlTexUI.parentTexture.width, mtlTexUI.parentTexture.height);
}
if (mtlTexOutput.parentTexture != nil)
{
    NSLog(@"mtlTexUI parentTex %lu x %lu", mtlTexOutput.parentTexture.width, mtlTexOutput.parentTexture.height);
}

NSLog(@"ERR mtlTexColor gfxFormat: %lu, size: %lu x %lu  ~  %lu x %lu ", (unsigned long)mtlTexColor.pixelFormat , (unsigned long)mtlTexColor.width , (unsigned long)mtlTexColor.height, (unsigned long)frameInterpolator.inputWidth, (unsigned long)frameInterpolator.inputHeight);

NSLog(@"ERR mtlTexPrevColor gfxFormat: %lu, size: %lu x %lu  ~  %lu x %lu ", (unsigned long)mtlTexPrevColor.pixelFormat , (unsigned long)mtlTexPrevColor.width , (unsigned long)mtlTexPrevColor.height, (unsigned long)frameInterpolator.inputWidth, (unsigned long)frameInterpolator.inputHeight);

NSLog(@"ERR mtlTexMotion gfxFormat: %lu, size: %lu x %lu  ~  %lu x %lu ", (unsigned long)mtlTexMotion.pixelFormat , (unsigned long)mtlTexMotion.width , (unsigned long)mtlTexMotion.height, (unsigned long)frameInterpolator.inputWidth, (unsigned long)frameInterpolator.inputHeight);

NSLog(@"ERR mtlTexDepth gfxFormat: %lu, size: %lu x %lu  ~  %lu x %lu ", (unsigned long)mtlTexDepth.pixelFormat , (unsigned long)mtlTexDepth.width , (unsigned long)mtlTexDepth.height, (unsigned long)frameInterpolator.inputWidth, (unsigned long)frameInterpolator.inputHeight);

NSLog(@"ERR mtlTexUI gfxFormat: %lu, size: %lu x %lu  ~  %lu x %lu ", (unsigned long)mtlTexUI.pixelFormat , (unsigned long)mtlTexUI.width , (unsigned long)mtlTexUI.height, (unsigned long)frameInterpolator.outputWidth, (unsigned long)frameInterpolator.outputHeight);

NSLog(@"ERR mtlTexOutput gfxFormat: %lu, size: %lu x %lu  ~  %lu x %lu ", (unsigned long)mtlTexOutput.pixelFormat , (unsigned long)mtlTexOutput.width , (unsigned long)mtlTexOutput.height, (unsigned long)frameInterpolator.outputWidth, (unsigned long)frameInterpolator.outputHeight);

NSLog(@"ERR m_MTLFXTemporalScaler inputSize: %lu x %lu  outputSize: %lu x %lu ", (unsigned long) m_MTLFXTemporalScaler.inputWidth , (unsigned long)m_MTLFXTemporalScaler.inputHeight, (unsigned long) m_MTLFXTemporalScaler.outputWidth , (unsigned long)m_MTLFXTemporalScaler.outputHeight);

NSLog(@"ERR m_MTLFXTemporalScaler colorFormat- %lu depthFormat- %lu  motionFormat- %lu outputFormat- %lu ", m_MTLFXTemporalScaler.colorTextureFormat , m_MTLFXTemporalScaler.depthTextureFormat, m_MTLFXTemporalScaler.motionTextureFormat , m_MTLFXTemporalScaler.outputTextureFormat);

NSLog(@"ERR FrameInterpolator colorFormat- %lu depthFormat- %lu  motionFormat- %lu  outputFormat- %lu  uiFormat- %lu", frameInterpolator.colorTextureFormat , frameInterpolator.depthTextureFormat, frameInterpolator.motionTextureFormat , frameInterpolator.outputTextureFormat, frameInterpolator.uiTextureFormat);

frameInterpolator.uiTexture = mtlTexUI;
frameInterpolator.colorTexture = mtlTexColor;
frameInterpolator.depthTexture = mtlTexDepth;
frameInterpolator.motionTexture = mtlTexMotion;
frameInterpolator.outputTexture = mtlTexOutput;
frameInterpolator.prevColorTexture = mtlTexPrevColor;


frameInterpolator.deltaTime = context.deltaTime;
frameInterpolator.nearPlane = context.nearPlane;
frameInterpolator.farPlane = context.farPlane;
frameInterpolator.fieldOfView = context.fieldOfView;
frameInterpolator.aspectRatio = context.aspectRatio;
frameInterpolator.jitterOffsetX = context.jitterOffsetX;
frameInterpolator.jitterOffsetY = context.jitterOffsetY;
frameInterpolator.motionVectorScaleX = context.motionVectorScaleX;
frameInterpolator.motionVectorScaleY = context.motionVectorScaleY;


frameInterpolator.depthReversed = context.depthReversed;
frameInterpolator.shouldResetHistory = context.shouldResetHistory;
frameInterpolator.uiTextureComposited = context.uiTextureComposited;

m_MTLFXTemporalScaler.reset = context.shouldResetHistory;
m_MTLFXTemporalScaler.colorTexture = mtlTexColor;
m_MTLFXTemporalScaler.outputTexture = m_ScalerOutputTexture;
m_MTLFXTemporalScaler.motionTexture = mtlTexMotion;
m_MTLFXTemporalScaler.depthTexture = mtlTexDepth;
m_MTLFXTemporalScaler.depthReversed = context.depthReversed;
m_MTLFXTemporalScaler.jitterOffsetX = context.jitterOffsetX;
m_MTLFXTemporalScaler.jitterOffsetY = context.jitterOffsetY;
m_MTLFXTemporalScaler.inputContentWidth = mtlTexColor.width;
m_MTLFXTemporalScaler.inputContentHeight = mtlTexColor.height;
m_MTLFXTemporalScaler.motionVectorScaleX = context.motionVectorScaleX;
m_MTLFXTemporalScaler.motionVectorScaleY = context.motionVectorScaleY;


[frameInterpolator encodeToCommandBuffer: cmdBuffer];
frameInterpolator.colorTexture = nil;
frameInterpolator.depthTexture = nil;
frameInterpolator.motionTexture = nil;
frameInterpolator.outputTexture = nil;
frameInterpolator.prevColorTexture = nil;

}

0
ovenxp OP
1w

and the output log is below: [2026-01-22 12:42:00] Compiled shader: Hidden/PostProcessing/ExtractUIHint, subShaderLOD:0, pass: <Unnamed Pass 0>, stage: fragment, codeSize: 1059, keywords

OSLOG-8E9458FC-7294-45FC-B2B2-10E6E69096A7 7 80 L 72 {t:1769056920.688173,tid:0xfd221e,offset:0x4a67d0c} ERR mtlTexColor gfxFormat: 70, size: 1024 x 512 ~ 1024 x 512

OSLOG-8E9458FC-7294-45FC-B2B2-10E6E69096A7 7 80 L 73 {t:1769056920.688209,offset:0x4a67d7c} ERR mtlTexPrevColor gfxFormat: 70, size: 1024 x 512 ~ 1024 x 512

OSLOG-8E9458FC-7294-45FC-B2B2-10E6E69096A7 7 80 L 74 {t:1769056920.688219,offset:0x4a67dec} ERR mtlTexMotion gfxFormat: 65, size: 1024 x 512 ~ 1024 x 512

OSLOG-8E9458FC-7294-45FC-B2B2-10E6E69096A7 7 80 L 75 {t:1769056920.688224,offset:0x4a67e5c} ERR mtlTexDepth gfxFormat: 55, size: 1024 x 512 ~ 1024 x 512

OSLOG-8E9458FC-7294-45FC-B2B2-10E6E69096A7 7 80 L 76 {t:1769056920.688230,offset:0x4a67ecc} ERR mtlTexUI gfxFormat: 70, size: 2048 x 1024 ~ 2048 x 1024

OSLOG-8E9458FC-7294-45FC-B2B2-10E6E69096A7 7 80 L 77 {t:1769056920.688235,offset:0x4a67f3c} ERR mtlTexOutput gfxFormat: 80, size: 2048 x 1024 ~ 2048 x 1024

OSLOG-8E9458FC-7294-45FC-B2B2-10E6E69096A7 7 80 L 78 {t:1769056920.688245,offset:0x4a67f98} ERR m_MTLFXTemporalScaler inputSize: 1024 x 512 outputSize: 2048 x 1024

OSLOG-8E9458FC-7294-45FC-B2B2-10E6E69096A7 7 80 L 79 {t:1769056920.688252,offset:0x4a67ff4} ERR m_MTLFXTemporalScaler colorFormat- 70 depthFormat- 55 motionFormat- 65 outputFormat- 80

OSLOG-8E9458FC-7294-45FC-B2B2-10E6E69096A7 7 80 L 7a {t:1769056920.688259,offset:0x4a68064} ERR FrameInterpolator colorFormat- 70 depthFormat- 55 motionFormat- 65 outputFormat- 80 uiFormat- 70 /AppleInternal/Library/BuildRoots/4~CB3qugCnkOnvt3utP5EjRtMy8k8SCYMtZU_Tyn4/Library/Caches/com.apple.xbs/Sources/MetalFX/Framework/MetalFXDebugError.h:29: failed assertion `Color texture width mismatch from descriptor'

0
ovenxp OP
1w

when I set input and output resolution all the same, 2048x1024, validation error get fixed. But still I cant figure out where is the problem. Which attachment RT size get wrong?

0
MetalFX FrameInterpolator assertion: &#96;Color texture width mismatch from descriptor&#96; even when all texture sizes match
First post date Last post date
 
 
Q