i met the -12911 error which is the error code for kVTVideoDecoderMulfunctionErr, the subsequent logs shows it was caused by an err in allocating the memory for the ioSurface.
default 13:16:36.985714+0800 videocodecd AppleAVD: AppleAVDWrapperFghrnDecoderStartSession: storage->miscPreferences 8
default 13:16:36.985774+0800 videocodecd AppleAVD: AppleAVDWrapperFghrnDecoderStartSession() codecType: Fghrn, encryptionScheme 6, 7680 x 4320, tryAllFrames = 0, session : 0xc54f1c020, built 01:13:29 Aug 8 2024
default 13:16:37.011217+0800 videocodecd AppleAVD: INFO: AppleAVDCreateDecodeDeviceInternal(): Allocating new AppleAVDCommandBuilder
default 13:16:37.011267+0800 videocodecd AppleAVD: CAVDAvxDecoder(): set av1 dual vp mode as 0
default 13:16:37.011369+0800 videocodecd AppleAVD: INFO: Av1 decoder does not support kVASetSliceHeaderThreshold, command has been ignored
default 13:16:37.011811+0800 videocodecd AppleAVD: ERROR: mapAVDMemory(): failed error: -536870212
default 13:16:37.011850+0800 videocodecd AppleAVD: VAMapPixelBuffer(): error allocating surface
default 13:16:37.011862+0800 videocodecd AppleAVD: createDecoder(): VAMapPixelBuffer failed for input buf IOSurface, err=-536870212
default 13:16:37.011890+0800 videocodecd AppleAVD: createDecoder(): error -536870212 and decoder is 0xc55400000
But that error is caused by decoding a 8K AV1 error. when decoding a 4K video the error is gone.