You might be sending or consuming sample buffers repeatedly.
In your your device source (CMIOExtensionDeviceSource) implementation, you should have methods that start/stop the source stream and another pair that start/stop the sink stream.
@interface WhateverExtensionDeviceSource : NSObject<CMIOExtensionDeviceSource>
{
...
CMSampleBufferRef _sampleBuffer;
}
...
- (void)startStreaming;
- (void)stopStreaming;
- (void)startStreamingSink:(CMIOExtensionClient *)client;
- (void)stopStreamingSink:(CMIOExtensionClient *)client;
@end
Notice the reference to a sample buffer (_sampleBuffer).
The sink stream's "start" method should keep a reference to the client (the application feeding samples to the extension via the queue):
@implementation WhateverExtensionStreamSink
...
- (BOOL)authorizedToStartStreamForClient:(CMIOExtensionClient *)client
{
_client = client;
return YES;
}
- (BOOL)startStreamAndReturnError:(NSError * _Nullable *)outError
{
WhateverExtensionDeviceSource *deviceSource = (WhateverExtensionDeviceSource *)_device.source;
[deviceSource startStreamingSink:_client];
return YES;
}
In that method you should have a block running repeatedly on a timer. Here, you consume a sample buffer from the sink stream (consumeSampleBufferFromClient), keep a reference to it (_sampleBuffer = CFRetain(sample_buffer)), and notify the stream that you got the sample buffer (notifyScheduledOutputChanged):
- (void)startStreamingSink:(CMIOExtensionClient *)client
{
_streamingCounterSink++;
_timerSink = dispatch_source_create(DISPATCH_SOURCE_TYPE_TIMER, 0, DISPATCH_TIMER_STRICT, _timerQueueSink);
dispatch_source_set_timer(_timerSink, DISPATCH_TIME_NOW, (uint64_t) (1e9 / (2 * kFrameRate)), 0);
dispatch_source_set_event_handler(_timerSink, ^{
[self->_streamSink.stream
consumeSampleBufferFromClient:client
completionHandler:^(
CMSampleBufferRef sample_buffer,
uint64_t sample_buffer_sequence_number,
CMIOExtensionStreamDiscontinuityFlags discontinuity,
BOOL has_more_sample_buffers,
NSError *error
) {
if (sample_buffer == nil)
return;
if (self->_sampleBuffer == nil) {
self->_sampleBuffer = CFRetain(sample_buffer);
}
CMIOExtensionScheduledOutput *output = [
[CMIOExtensionScheduledOutput alloc]
initWithSequenceNumber:sample_buffer_sequence_number
hostTimeInNanoseconds:...
];
[self->_streamSink.stream notifyScheduledOutputChanged:output];
}
];
});
dispatch_source_set_cancel_handler(_timerSink, ^{
});
dispatch_resume(_timerSink);
}
You have a similar block on a timer in the source stream's "start" method. There, you wait for the sample buffer reference to change from nil to non-nil, and pass it on to the source stream (sendSampleBuffer), and finally release _sampleBuffer and reset it to nil:
dispatch_source_set_event_handler(_timerSource, ^{
if (self->_sampleBuffer != nil) {
if (self->_streamingCounterSource > 0) {
CMTime time = CMClockGetTime(CMClockGetHostTimeClock());
Float64 ns = CMTimeGetSeconds(time) * 1e9;
CMSampleBufferRef sbuf = nil;
CMSampleTimingInfo timing_info = {
.presentationTimeStamp = time,
};
OSStatus err = CMSampleBufferCreateCopyWithNewTiming(
kCFAllocatorDefault,
self->_sampleBuffer,
1,
(const CMSampleTimingInfo []) {timing_info},
&sbuf
);
[self->_streamSource.stream
sendSampleBuffer:sbuf
discontinuity:CMIOExtensionStreamDiscontinuityFlagNone
hostTimeInNanoseconds:ns
];
CFRelease(sbuf);
}
CFRelease(self->_sampleBuffer);
self->_sampleBuffer = nil;
}
});
Notice:
_sampleBuffer is retained in the sink timer block and released in the source timer block;
the source and sink timers are set to twice the frame rate, so you don't miss a frame;
