Post

Replies

Boosts

Views

Activity

When observing a notification that may be posted "on a thread other than the one used to registered the observer," how should I ensure thread-safe UI work?
I observe when an AVPlayer finishes play in order to present a UIAlert at the end time. NotificationCenter.default.addObserver( self, selector: #selector(presentAlert), name: .AVPlayerItemDidPlayToEndTime, object: nil ) I've had multiple user reports of the alert happening where they're not intended, such as the middle of the video after replaying, and on other views. I'm unable to reproduce this myself, but my guess is that it's a threading issue since AVPlayerItemDidPlayToEndTime says "the system may post this notification on a thread other than the one used to registered the observer." How then do I make sure the alert is present on the main thread? Should I dispatch to the main queue from within my presentAlert function, or add the above observer with addObserver(forName:object:queue:using:) instead, passing in the main operation queue?
1
0
1.3k
Oct ’21
Should a delegate property passed into a struct also be declared as weak in the struct?
The Swift book says that "to prevent strong reference cycles, delegates are declared as weak references." protocol SomeDelegate: AnyObject { } class viewController: UIViewController, SomeDelegate { weak var delegate: SomeDelegate? override func viewDidLoad() { delegate = self } } Say the class parameterizes a struct with that delegate class viewController: UIViewController, SomeDelegate { weak var delegate: SomeDelegate? override func viewDidLoad() { delegate = self let exampleView = ExampleView(delegate: delegate) let hostingController = UIHostingController(rootView: exampleView) self.present(hostingController, animated: true) } } struct ExampleView: View { var delegate: SomeDelegate! var body: some View { Text("") } } Should the delegate property in the struct also be marked with weak?
1
0
2.2k
Oct ’21
How do you extend an iOS app's background execution time when continuing an upload operation?
I'd like a user's upload operation that's started in the foreground to continue when they leave the app. Apple's article Extending Your App's Background Execution Time has the following code listing func sendDataToServer( data : NSData ) { // Perform the task on a background queue. DispatchQueue.global().async { // Request the task assertion and save the ID. self.backgroundTaskID = UIApplication.shared. beginBackgroundTask (withName: "Finish Network Tasks") { // End the task if time expires. UIApplication.shared.endBackgroundTask(self.backgroundTaskID!) self.backgroundTaskID = UIBackgroundTaskInvalid } // Send the data synchronously. self.sendAppDataToServer( data: data) // End the task assertion. UIApplication.shared.endBackgroundTask(self.backgroundTaskID!) self.backgroundTaskID = UIBackgroundTaskInvalid } } The call to self.sendAppDataToServer( data: data) is unclear. Is this where the upload operation would go, wrapped in Dispatch.global().sync { }?
1
0
654
Nov ’21
How do you create a new AVAsset video that consists of only frames from given `CMTimeRange`s of another video?
Apple's sample code Identifying Trajectories in Video contains the following delegate callback: func cameraViewController(_ controller: CameraViewController, didReceiveBuffer buffer: CMSampleBuffer, orientation: CGImagePropertyOrientation) { let visionHandler = VNImageRequestHandler(cmSampleBuffer: buffer, orientation: orientation, options: [:]) if gameManager.stateMachine.currentState is GameManager.TrackThrowsState { DispatchQueue.main.async { // Get the frame of rendered view let normalizedFrame = CGRect(x: 0, y: 0, width: 1, height: 1) self.jointSegmentView.frame = controller.viewRectForVisionRect(normalizedFrame) self.trajectoryView.frame = controller.viewRectForVisionRect(normalizedFrame) } // Perform the trajectory request in a separate dispatch queue. trajectoryQueue.async { do { try visionHandler.perform([self.detectTrajectoryRequest]) if let results = self.detectTrajectoryRequest.results { DispatchQueue.main.async { self.processTrajectoryObservations(controller, results) } } } catch { AppError.display(error, inViewController: self) } } } } However, instead of drawing UI whenever detectTrajectoryRequest.results exist (https://developer.apple.com/documentation/vision/vndetecttrajectoriesrequest/3675672-results), I'm interested in using the CMTimeRange provided by each result to construct a new video. In effect, this would filter down the original video to only frames with trajectories. How might I accomplish this, perhaps through writing only specific time ranges' frames from one AVFoundation video to a new AVFoundation video?
1
0
818
Nov ’21
Trying to modernize and get working Apple's sample code AVReaderWriter. Why is this `DispatchGroup` work block not being called?
Apple's sample code "AVReaderWriter: Offline Audio / Video Processing" has the following listing let writingGroup = dispatch_group_create() // Transfer data from input file to output file. self.transferVideoTracks(videoReaderOutputsAndWriterInputs, group: writingGroup) self.transferPassthroughTracks(passthroughReaderOutputsAndWriterInputs, group: writingGroup) // Handle completion. let queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0) dispatch_group_notify(writingGroup, queue) { // `readingAndWritingDidFinish()` is guaranteed to call `finish()` exactly once. self.readingAndWritingDidFinish(assetReader, assetWriter: assetWriter) } in CynanifyOperation.swift (an NSOperation subclass that stylizes imported video and exports it). How would I get about writing this part in modern Swift so that it compiles and works? I've tried writing this as let writingGroup = DispatchGroup() // Transfer data from input file to output file. self.transferVideoTracks(videoReaderOutputsAndWriterInputs: videoReaderOutputsAndWriterInputs, group: writingGroup) self.transferPassthroughTracks(passthroughReaderOutputsAndWriterInputs: passthroughReaderOutputsAndWriterInputs, group: writingGroup) // Handle completion. writingGroup.notify(queue: .global()) { // `readingAndWritingDidFinish()` is guaranteed to call `finish()` exactly once. self.readingAndWritingDidFinish(assetReader: assetReader, assetWriter: assetWriter) } However, it's taking an extremely long time for self.readingAndWritingDidFinish(assetReader: assetReader, assetWriter: assetWriter) to be called, and my UI is stuck in the ProgressViewController with a loading spinner. Is there something I wrote incorrectly or missed conceptually in the Swift 5 version?
1
0
485
Nov ’21
How do restrict pan gesture recognizers to when a pinch gesture is occurring?
How do you only accept pan gestures when the user is in the process of a pinch gesture? In other words, I'd like to avoid delivering one finger pan gestures. @IBAction func pinchPiece(_ pinchGestureRecognizer: UIPinchGestureRecognizer) { guard pinchGestureRecognizer.state == .began || pinchGestureRecognizer.state == .changed, let piece = pinchGestureRecognizer.view else { // After pinch releases, zoom back out. if pinchGestureRecognizer.state == .ended { UIView.animate(withDuration: 0.3, animations: { pinchGestureRecognizer.view?.transform = CGAffineTransform.identity }) } return } adjustAnchor(for: pinchGestureRecognizer) let scale = pinchGestureRecognizer.scale piece.transform = piece.transform.scaledBy(x: scale, y: scale) pinchGestureRecognizer.scale = 1 // Clear scale so that it is the right delta next time. } @IBAction func panPiece(_ panGestureRecognizer: UIPanGestureRecognizer) { guard panGestureRecognizer.state == .began || panGestureRecognizer.state == .changed, let piece = panGestureRecognizer.view else { return } let translation = panGestureRecognizer.translation(in: piece.superview) piece.center = CGPoint(x: piece.center.x + translation.x, y: piece.center.y + translation.y) panGestureRecognizer.setTranslation(.zero, in: piece.superview) } public func gestureRecognizer(_ gestureRecognizer: UIGestureRecognizer, shouldRecognizeSimultaneouslyWith otherGestureRecognizer: UIGestureRecognizer) -> Bool { true }
1
0
860
Nov ’21
Selectively reading sample buffers from specific time ranges and then writing them to an asset writer - why is the AVPlayer stuck loading?
Given an AVAsset, I'm performing a Vision trajectory request on it and would like to write out a video asset that only contains frames with trajectories (filter out downtime in sports footage where there's no ball moving). I'm unsure what would be a good approach, but as a starting point I tried the following pipeline: Copy sample buffer from the source AVAssetReaderOutput. Perform trajectory request on a vision handler parameterized by the sample buffer. For each resulting VNTrajectoryObservation (trajectory detected), use its associated CMTimeRange to configure a new AVAssetReader set to that time range. Append the time range constrained sample buffer to one AVAssetWriterInput until the forEach is complete. In code: private func transferSamplesAsynchronously(from readerOutput: AVAssetReaderOutput, to writerInput: AVAssetWriterInput, onQueue queue: DispatchQueue, sampleBufferProcessor: SampleBufferProcessor, completionHandler: @escaping () -> Void) { /* The writerInput continously invokes this closure until finished or cancelled. It throws an NSInternalInconsistencyException if called more than once for the same writer. */ writerInput.requestMediaDataWhenReady(on: queue) { var isDone = false /* While the writerInput accepts more data, process the sampleBuffer and then transfer the processed sample to the writerInput. */ while writerInput.isReadyForMoreMediaData { if self.isCancelled { isDone = true break } // Get the next sample from the asset reader output. guard let sampleBuffer = readerOutput.copyNextSampleBuffer() else { // The asset reader output has no more samples to vend. isDone = true break } let visionHandler = VNImageRequestHandler(cmSampleBuffer: sampleBuffer, orientation: self.orientation, options: [:]) do { try visionHandler.perform([self.detectTrajectoryRequest]) if let results = self.detectTrajectoryRequest.results { try results.forEach { result in let assetReader = try AVAssetReader(asset: self.asset) assetReader.timeRange = result.timeRange let trackOutput = AVTrackOutputs.firstTrackOutput(ofType: .video, fromTracks: self.asset.tracks, withOutputSettings: nil) assetReader.add(trackOutput) assetReader.startReading() guard let sampleBuffer = trackOutput.copyNextSampleBuffer() else { // The asset reader output has no more samples to vend. isDone = true return } // Append the sample to the asset writer input. guard writerInput.append(sampleBuffer) else { /* The writer could not append the sample buffer. The `readingAndWritingDidFinish()` function handles any error information from the asset writer. */ isDone = true return } } } } catch { print(error) } } if isDone { /* Calling `markAsFinished()` on the asset writer input does the following: 1. Unblocks any other inputs needing more samples. 2. Cancels further invocations of this "request media data" callback block. */ writerInput.markAsFinished() /* Tell the caller the reader output and writer input finished transferring samples. */ completionHandler() } } } private func readingAndWritingDidFinish(assetReaderWriter: AVAssetReaderWriter, completionHandler: @escaping FinishHandler) { if isCancelled { completionHandler(.success(.cancelled)) return } // Handle any error during processing of the video. guard sampleTransferError == nil else { assetReaderWriter.cancel() completionHandler(.failure(sampleTransferError!)) return } // Evaluate the result reading the samples. let result = assetReaderWriter.readingCompleted() if case .failure = result { completionHandler(result) return } /* Finish writing, and asynchronously evaluate the results from writing the samples. */ assetReaderWriter.writingCompleted { result in completionHandler(result) return } } When run I get the following: No error is caught in the first catch clause, and none are caught in private func readingAndWritingDidFinish(assetReaderWriter: AVAssetReaderWriter, completionHandler: @escaping FinishHandler), the completion handler is called. Help with any of the following questions would be appreciated: What is causing what appears to be indefinite loading? How might I isolate the problem further? Am I misusing or misunderstanding how to selectively read from time ranges of AVAssetReader objects? Should I forego the AVAssetReader / AVAsssetWriter route entirely, and use the time ranges with AVAssetExportSession instead? I don't know how the two approaches compare, or what to consider when choosing between the two.
1
0
964
Dec ’21
How can I improve the speed of running a `VNDetectHumanBodyPoseRequest` on a `VNImageRequestHandler` for every `CMSampleBuffer` of an imported video?
Below, the sampleBufferProcessor closure is where the Vision body pose detection occurs. /// Transfers the sample data from the AVAssetReaderOutput to the AVAssetWriterInput, /// processing via a CMSampleBufferProcessor. /// /// - Parameters: /// - readerOutput: The source sample data. /// - writerInput: The destination for the sample data. /// - queue: The DispatchQueue. /// - completionHandler: The completion handler to run when the transfer finishes. /// - Tag: transferSamplesAsynchronously private func transferSamplesAsynchronously(from readerOutput: AVAssetReaderOutput, to writerInput: AVAssetWriterInput, onQueue queue: DispatchQueue, sampleBufferProcessor: SampleBufferProcessor, completionHandler: @escaping () -> Void) { /* The writerInput continously invokes this closure until finished or cancelled. It throws an NSInternalInconsistencyException if called more than once for the same writer. */ writerInput.requestMediaDataWhenReady(on: queue) { var isDone = false /* While the writerInput accepts more data, process the sampleBuffer and then transfer the processed sample to the writerInput. */ while writerInput.isReadyForMoreMediaData { if self.isCancelled { isDone = true break } // Get the next sample from the asset reader output. guard let sampleBuffer = readerOutput.copyNextSampleBuffer() else { // The asset reader output has no more samples to vend. isDone = true break } // Process the sample, if requested. do { try sampleBufferProcessor?(sampleBuffer) } catch { /* The `readingAndWritingDidFinish()` function picks up this error. */ self.sampleTransferError = error isDone = true } // Append the sample to the asset writer input. guard writerInput.append(sampleBuffer) else { /* The writer could not append the sample buffer. The `readingAndWritingDidFinish()` function handles any error information from the asset writer. */ isDone = true break } } if isDone { /* Calling `markAsFinished()` on the asset writer input does the following: 1. Unblocks any other inputs needing more samples. 2. Cancels further invocations of this "request media data" callback block. */ writerInput.markAsFinished() /* Tell the caller the reader output and writer input finished transferring samples. */ completionHandler() } } } The processor closure runs body pose detection on every sample buffer so that later in the VNDetectHumanBodyPoseRequest completion handler, VNHumanBodyPoseObservation results are fed into a custom Core ML action classifier. private func videoProcessorForActivityClassification() -> SampleBufferProcessor { let videoProcessor: SampleBufferProcessor = { sampleBuffer in do { let requestHandler = VNImageRequestHandler(cmSampleBuffer: sampleBuffer) try requestHandler.perform([self.detectHumanBodyPoseRequest]) } catch { print("Unable to perform the request: \(error.localizedDescription).") } } return videoProcessor } How could I improve the performance of this pipeline? After testing with an hour long 4K video at 60 FPS, it took several hours to process running as a Mac Catalyst app on M1 Max.
1
0
1.1k
Jan ’22
How do you track when a VNVideoProcessor analysis is finished?
I'm adopting and transitioning to VNVideoProcessor away from performing Vision requests on individual frames, since it more concisely does the same. However, I'm not sure how to detect when analysis of a video is finished. Previously when reading frames with AVFoundation I could check with // Get the next sample from the asset reader output. guard let sampleBuffer = readerOutput.copyNextSampleBuffer() else { // The asset reader output has no more samples to vend. isDone = true break } What would be an equivalent when using VNVideoProcessor?
1
0
738
Jan ’22
Apple sample code "Detecting Human Actions in a Live Video Feed" - accessing the observations associated with an action prediction
Say I have an alert @State var showingAlert = false var body: some View { Text("Hello, world!") .alert("Here's an alert with multiple possible buttons.", isPresented: $showingAlert) { Button("OK") { } Button("Another button that may or may not show") { } } } How could I display the second button based only on some condition? I tried factoring out one button into fileprivate func extractedFunc() -> Button<Text> { return Button("OK") { } } and this would work for conditionally displaying the button content given a fixed number of buttons, but how could optionality of buttons be taken into account?
1
0
519
Jan ’22
What would be a good way to partition instances by a particular matching property, such that each partition is a collection view section?
Say that in this example here, the struct struct Reminder: Identifiable { var id: String = UUID().uuidString var title: String var dueDate: Date var notes: String? = nil var isComplete: Bool = false var city: String } is modified slightly to include a city string. In the collection view that displays the reminders, I'd like each section to be each unique city, so if two reminder cells have the same city string then they would be in the same section of the collection view. The progress I've made to this end is sorting the reminders array so that reminders cells are grouped together by city func updateSnapshot(reloading ids: [Reminder.ID] = []) { var snapshot = Snapshot() snapshot.appendSections([0]) let reminders = reminders.sorted { $0.city } snapshot.appendItems(reminders.map { $0.id }) if !ids.isEmpty { snapshot.reloadItems(ids) } dataSource.apply(snapshot) } Where I'm stuck is in coming up with a way to make the snapshot represent sections by unique cities, and not just one flat section of all reminders.
1
0
628
May ’22
`Undefined symbol: _OBJC_CLASS_$_DDLog` when trying to run unit tests with Simulator target
On Xcode 13.4.1, when specifically trying to run unit tests, I get an error Undefined symbol: _OBJC_CLASS_$_DDLog. After clicking on it in the Issue navigator, a longer error message says Undefined symbols for architecture arm64:   "_OBJC_CLASS_$_DDLog", referenced from:       objc-class-ref in How might I fix this? I tried running both Xcode and the Simulator in Rosetta, but I get the same error message but for x86.
1
0
7.5k
Sep ’22