I am trying to get an H264 streaming app working on various platforms using a combination of Apple Video Toolbox and OpenH264. There is one use-case that doesn't work and I can't find any solution. When the source uses video Toolbox on a 2011 iMac running MacOS High Sierra and the receiver is a MacBook pro running Big Sur.
On the receiver the decoded image is about 3/4 green. If I scale the image down to about 1/8 of original before encoding then it works fine. If I capture the frames on the MacBook and then run exactly the same decoding software in a test program on the iMac then it decodes fine. Doing the same on the Macbook (same image of test program) give 3/4 green again.
I have a similar problem when receiving from an OpenH264 encoder on a slower Windows machine. I suspect that this has something to do with temporal processing, but really don't understand H264 well enough to work it out. One thing that I did notice is that the decode call returns with no error code but a NULL pixel buffer about 70% of the time.
The "guts" of the decoding part looks like this (modified from a demo on GitHub)
void didDecompress(void *decompressionOutputRefCon, void *sourceFrameRefCon, OSStatus status, VTDecodeInfoFlags infoFlags, CVImageBufferRef pixelBuffer, CMTime presentationTimeStamp, CMTime presentationDuration )
{
		CVPixelBufferRef *outputPixelBuffer = (CVPixelBufferRef *)sourceFrameRefCon;
		*outputPixelBuffer = CVPixelBufferRetain(pixelBuffer);
}
void initVideoDecodeToolBox ()
		{
				if (!decodeSession)
				{
						const uint8_t* parameterSetPointers[2] = { mSPS, mPPS };
						const size_t parameterSetSizes[2] = { mSPSSize, mPPSSize };
						OSStatus status = CMVideoFormatDescriptionCreateFromH264ParameterSets(kCFAllocatorDefault,2, //param count
																																									parameterSetPointers,
																																									parameterSetSizes,
																																									4, //nal start code size
																																									&formatDescription);
						if(status == noErr)
						{
								CFDictionaryRef attrs = NULL;
								const void *keys[] = { kCVPixelBufferPixelFormatTypeKey, kVTDecompressionPropertyKey_RealTime };
								uint32_t v = kCVPixelFormatType_32BGRA;
								const void *values[] = { CFNumberCreate(NULL, kCFNumberSInt32Type, &v), kCFBooleanTrue };
								attrs = CFDictionaryCreate(NULL, keys, values, 2, NULL, NULL);
								VTDecompressionOutputCallbackRecord callBackRecord;
								callBackRecord.decompressionOutputCallback = didDecompress;
								callBackRecord.decompressionOutputRefCon = NULL;
								status = VTDecompressionSessionCreate(kCFAllocatorDefault, formatDescription, NULL, attrs, &callBackRecord, &decodeSession);
								CFRelease(attrs);
						}
						else
						{
								NSLog(@"IOS8VT: reset decoder session failed status=%d", status);
						}
				}
		}
CVPixelBufferRef decode ( const char *NALBuffer, size_t NALSize )
		{
				CVPixelBufferRef outputPixelBuffer = NULL;
				if (decodeSession && formatDescription )
				{
						// The NAL buffer has been stripped of the NAL length data, so this has to be put back in
						MemoryBlock buf ( NALSize + 4);
						memcpy ( (char*)buf.getData()+4, NALBuffer, NALSize );
						*((uint32*)buf.getData()) = CFSwapInt32HostToBig ((uint32)NALSize);
						
						CMBlockBufferRef blockBuffer = NULL;
						OSStatus status	= CMBlockBufferCreateWithMemoryBlock(kCFAllocatorDefault, buf.getData(), NALSize+4,kCFAllocatorNull,NULL, 0, NALSize+4, 0, &blockBuffer);
						
						if(status == kCMBlockBufferNoErr)
						{
								CMSampleBufferRef sampleBuffer = NULL;
								const size_t sampleSizeArray[] = {NALSize + 4};
								status = CMSampleBufferCreateReady(kCFAllocatorDefault,blockBuffer,formatDescription,1, 0, NULL, 1, sampleSizeArray,&sampleBuffer);
								
								if (status == kCMBlockBufferNoErr && sampleBuffer)
								{
										VTDecodeFrameFlags flags = 0;VTDecodeInfoFlags flagOut = 0;
										
										// The default is synchronous operation.
										// Call didDecompress and call back after returning.
										OSStatus decodeStatus = VTDecompressionSessionDecodeFrame ( decodeSession, sampleBuffer, flags, &outputPixelBuffer, &flagOut );
										if(decodeStatus != noErr)
										{
												DBG ( "decode failed status=" + String ( decodeStatus) );
										}
										CFRelease(sampleBuffer);
								}
								CFRelease(blockBuffer);
						}
				}
				return outputPixelBuffer;
		}
Selecting any option will automatically load the page
Post
Replies
Boosts
Views
Activity
I've been implementing in app purchases into an existing C++ app. I'm using the latest Swift StoreKit since the old ObjC interface is deprecated . There is a really weird problem where the swift/C++ bridging seems to get into a loop. After the Product structure is retrieved I have the following structure which I use to bridge to C++
public struct storeData
{
public var id : String
public var displayName : String
public var description : String
public var price : String
public var purchased : Bool = false
public var level : Int = 0
}
and this is passed back to the caller as follows
public func getProducts (bridge : StoreBridge) -> [storeData]
{
bridge.products.sort { $0.price > $1.price }
var productList : [storeData] = []
for product in bridge.products
{
let data : storeData = storeData(id: product.id,
displayName: product.displayName,
description: product.description,
price: product.displayPrice,
purchased: bridge.purchasedProductIds.contains(product.id)
)
productList.append(data)
}
return productList
}
the "bridge" variable is a bridging class where the guts of the bridge resides, and contains the "products" array as a publishable variable.
In the C++ code the data is retrieved by
outProd->id = String(inProd.getId());
outProd->displayName = String(inProd.getDisplayName());
outProd->description = String(inProd.getDescription());
outProd->price = String(String(inProd.getPrice()));
outProd->purchased = inProd.getPurchased();
The "String" is actually a JUCE string but that's not part of the problem. Testing this with a local StoreKit config file works fine but when I test with a sandbox AppStore the app hangs. Very specifically it hangs somewhere in the Swift thunk when retrieving the price. When I remove the line to retrieve the price everything works. And - and this is the weird bit - when I pad the price out with some random text, it now starts working (so I have a workaround). This is, however, slightly worrying behaviour. Ideas?
Topic:
App Store Distribution & Marketing
SubTopic:
App Store Connect API
Tags:
StoreKit Test
Swift
StoreKit