Post

Replies

Boosts

Views

Activity

ScreenCapture + CMSampleBuffer logic issue
i'm trying to work on a simple screen recording app on macOS that always records the last 'x' seconds of your screen and saves it whenever you want, as a way to get comfortable with swift programming and apple APIs. i was able to get it running for the past '30 seconds' and record and store it. however i realised that there was a core issue with my solution: i was defining the SCStreamConfiguration.queueDepth = 900 (to account for 30fps for 30 seconds) which goes completely against apple's instructions: https://developer.apple.com/documentation/screencapturekit/scstreamconfiguration/queuedepth?language=objc now when i changed queueDepth back to 8, i am only able to record 8 frames and it saves only those first 8 frames. i am unsure what the flow of the apis should be while dealing with screenCaptureKit. for context, here's my recording manager code that handles this logic (queueDepth = 900) import Foundation import ScreenCaptureKit import AVFoundation class RecordingManager: NSObject, ObservableObject, SCStreamDelegate { static let shared = RecordingManager() @Published var isRecording = false private var isStreamActive = false // Custom state flag private var stream: SCStream? private var streamOutputQueue = DispatchQueue(label: "com.clipback.StreamOutput", qos: .userInteractive) private var screenStreamOutput: ScreenStreamOutput? // Strong reference to output private var lastDisplayID: CGDirectDisplayID? private let displayCheckQueue = DispatchQueue(label: "com.clipback.DisplayCheck", qos: .background) // In-memory rolling buffer for last 30 seconds private var rollingFrameBuffer: [(CMSampleBuffer, CMTime)] = [] private let rollingFrameBufferQueue = DispatchQueue(label: "com.clipback.RollingBuffer", qos: .userInteractive) private let rollingBufferDuration: TimeInterval = 30.0 // seconds // Track frame statistics private var frameCount: Int = 0 private var lastReportTime: Date = Date() // Monitor for display availability private var displayCheckTimer: Timer? private var isWaitingForDisplay = false func startRecording() { print("[DEBUG] startRecording called.") guard !isRecording && !isWaitingForDisplay else { print("[DEBUG] Already recording or waiting, ignoring startRecording call") return } isWaitingForDisplay = true isStreamActive = true // Set active state checkForDisplay() } private func setupAndStartRecording(for display: SCDisplay, excluding appToExclude: SCRunningApplication?) { print("[DEBUG] setupAndStartRecording called for display: \(display.displayID)") let excludedApps = [appToExclude].compactMap { $0 } let filter = SCContentFilter(display: display, excludingApplications: excludedApps, exceptingWindows: []) let config = SCStreamConfiguration() config.width = display.width config.height = display.height config.minimumFrameInterval = CMTime(value: 1, timescale: 30) // 30 FPS config.queueDepth = 900 config.showsCursor = true print("[DEBUG] SCStreamConfiguration created: width=\(config.width), height=\(config.height), FPS=\(config.minimumFrameInterval.timescale)") stream = SCStream(filter: filter, configuration: config, delegate: self) print("[DEBUG] SCStream initialized.") self.screenStreamOutput = ScreenStreamOutput { [weak self] sampleBuffer, outputType in guard let self = self else { return } guard outputType == .screen else { return } guard sampleBuffer.isValid else { return } guard let attachments = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer, createIfNecessary: false) as? [[SCStreamFrameInfo: Any]], let statusRawValue = attachments.first?[.status] as? Int, let status = SCFrameStatus(rawValue: statusRawValue), status == .complete else { return } self.trackFrameRate() self.handleFrame(sampleBuffer) } do { try stream?.addStreamOutput(screenStreamOutput!, type: .screen, sampleHandlerQueue: streamOutputQueue) stream?.startCapture { [weak self] error in print("[DEBUG] SCStream.startCapture completion handler.") guard error == nil else { print("[DEBUG] Failed to start capture: \(error!.localizedDescription)") self?.handleStreamError(error!) return } DispatchQueue.main.async { self?.isRecording = true self?.isStreamActive = true // Update state on successful start print("[DEBUG] Recording started. isRecording = true.") } } } catch { print("[DEBUG] Error adding stream output: \(error.localizedDescription)") handleStreamError(error) } } private func handleFrame(_ sampleBuffer: CMSampleBuffer) { rollingFrameBufferQueue.async { [weak self] in guard let self = self else { return } let pts = CMSampleBufferGetPresentationTimeStamp(sampleBuffer) var retainedBuffer: CMSampleBuffer? CMSampleBufferCreateCopy(allocator: kCFAllocatorDefault, sampleBuffer: sampleBuffer, sampleBufferOut: &retainedBuffer) guard let buffer = retainedBuffer else { print("[DEBUG] Failed to copy sample buffer") return } self.rollingFrameBuffer.append((buffer, pts)) if let lastPTS = self.rollingFrameBuffer.last?.1 { while let firstPTS = self.rollingFrameBuffer.first?.1, CMTimeGetSeconds(CMTimeSubtract(lastPTS, firstPTS)) > self.rollingBufferDuration { self.rollingFrameBuffer.removeFirst() } } } } func stream(_ stream: SCStream, didStopWithError error: Error) { print("[DEBUG] Stream stopped with error: \(error.localizedDescription)") displayCheckQueue.async { [weak self] in // Move to displayCheckQueue for synchronization self?.handleStreamError(error) } } what could be the reason for this and what would be the possible fix logically? i dont understand why it's dependant on queueDepth, and if it is, how can I empty and append new recorded frames to it so that it continues working? any help or resource is greatly appreciated!
3
0
148
Jul ’25
Getting error 'Can't Decode' when exporting a video file via AVAssetExportSession
I'm working on a video player app that has the basic functionality of viewing a video and then be able to trim and crop that video and then save it. My flow of trimming a video and then saving it works well with any and every video. Cropping, however, doesn't work in the sense that I am unable to Save the video and export it. Whenever I crop a video, in the video player, I can see the cropped version of the video (it plays too!) but on saving said video, I get the error: Export failed with status: 4, error: Cannot Decode I've been debugging for 2 days now but I'm still unsure as to why this happens. I'm almost certain the bug is somewhere cause of cropping and then saving/exporting. If anyone has dealt with this before, please let me know what the best step to do is! If you could help me refine the flow for cropping and exporting, that'd be really helpful too. Thanks!
0
0
57
3w