Post

Replies

Boosts

Views

Activity

AVPlayerViewController content on both external screen and iOS device
I see AVPlayerViewController automatically hides video preview on iOS device the moment it detects external display. I want to present video preview on external display as well as on iOS device at the same time. The audio should be routed to external screen via HDMI and not playback on iOS device. Is it possible to do this using standard APIs? I could think of a couple of ways mentioned below and would like to hear from AVFoundation team and others what is recommended or the way out: Create two AVPlayerViewController and two AVPlayers. One of it is attached to externalScreen window, and both display same local video file. Audio would be muted for the playerController displaying video on iOS device. Some play/pause/seek syncing code is required to sync both the players. Use only one AVPlayerViewController that airplays to external screen. Attach AVPlayerItemVideoOutput to the playerItem and intercept video frames and display them using Metal/CoreImage on the iOS device. Use two AVPlayerLayer and add them to external and device screen. Video frames will be replicated to both external screen and iOS device , but not sure if it is possible to route audio only to external screen via HDMI but muted locally.
0
0
808
May ’22
AVAudioEngine playAndRecord noise over HDMI speakers
I am using AVAudioSession with playAndRecord category as follows: private func setupAudioSessionForRecording() { let audioSession = AVAudioSession.sharedInstance() do { try audioSession.setActive(false) try audioSession.setPreferredSampleRate(Double(48000)) } catch { NSLog("Unable to deactivate Audio session") } let options:AVAudioSession.CategoryOptions = [.allowAirPlay, .mixWithOthers] do { try AVAudioSession.sharedInstance().setCategory(AVAudioSession.Category.playAndRecord, mode: AVAudioSession.Mode.default, options: options) } catch { NSLog("Could not set audio session category \(error)") } do { try audioSession.setActive(true) } catch { NSLog("Unable to activate AudioSession") } } Next I use AVAudioEngine to repeat what I say in the microphone to external speakers (on the TV connected with iPhone using HDMI Cable). //MARK:- AudioEngine var engine: AVAudioEngine! var playerNode: AVAudioPlayerNode! var mixer: AVAudioMixerNode! var audioEngineRunning = false public func setupAudioEngine() { self.engine = AVAudioEngine() engine.connect(self.engine.inputNode, to: self.engine.outputNode, format: nil) do { engine.prepare() try self.engine.start() } catch { print("error couldn't start engine") } audioEngineRunning = true } public func stopAudioEngine() { engine.stop() audioEngineRunning = false } The issue is I hear some kind of reverb/humming noise after I speak for a few seconds that keeps getting amplified and repeated. If I use a RemoteIO unit instead, no such noise comes out of speakers. I am not sure if my setup of AVAudioEngine is correct. I have tried all kinds of AVAudioSession configuration but nothing changes. The link to sample audio with background speaker noise is posted [here] in the Stackoverflow forum (https://stackoverflow.com/questions/72170548/echo-when-using-avaudioengine-over-hdmi#comment127514327_72170548)
0
0
1k
May ’22
AVAudioEngine player node excessive delay
I am trying to use AVAudioEngine for listening to mic samples and playing them simultaneously via external speaker or headphones (assuming they are attached to iOS device). I tried the following using AVAudioPlayerNode and it works, but there is too much delay in the audio playback. Is there a way to hear sound realtime without delay? Why the scheduleBuffer API has so much delay I wonder. var engine: AVAudioEngine! var playerNode: AVAudioPlayerNode! var mixer: AVAudioMixerNode! var audioEngineRunning = false public func setupAudioEngine() { self.engine = AVAudioEngine() let input = engine.inputNode let format = input.inputFormat(forBus: 0) playerNode = AVAudioPlayerNode() engine.attach(playerNode) self.mixer = engine.mainMixerNode engine.connect(self.playerNode, to: self.mixer, format: playerNode.outputFormat(forBus: 0)) engine.inputNode.installTap(onBus: 0, bufferSize: 4096, format: format, block: { buffer, time in self.playerNode.scheduleBuffer(buffer, completionHandler: nil) }) do { engine.prepare() try self.engine.start() audioEngineRunning = true self.playerNode.play() } catch { print("error couldn't start engine") audioEngineRunning = false } }
0
0
1.6k
May ’22
Adding UILabel to UIContentViewCell disturbs it's size
I have a UICollectionViewCell defined in storyboard which has a UILabel added to it's contentView. Collection view uses a flow layout and I return a fixed size of cell in flowlayout delegate as follows: let sizeOfItem = CGFloat(210.0) func collectionView(_ collectionView: UICollectionView, layout collectionViewLayout: UICollectionViewLayout, sizeForItemAt indexPath: IndexPath) -> CGSize { return CGSize(width: sizeOfItem, height: sizeOfItem) } I added the following constraint for UILabel in storyboard and the cell automatically starts resizing itself to match the text size in UILabel. This is not what I want. I want the cell to be fixed size and label to autoshrink instead. I even tried setting contentHuggingPriority of label to lowest value (i.e. 1). This stops the cell from auto-shrinking if the text in label is small. But the cell grows when the text is bigger. I don't want this to happen either. I want the cell to remain fixed size and label to adapt it's font and be constrained within the cell.
0
0
565
Jun ’22
UICollectionViewFlowLayout disable dynamic sizing
I have a UICollectionViewFlowLayout assigned to UICollectionView. It's delegate method returns fixed CGSize in sizeForItemAtIndexPath. The problem is if UICollectionViewCell contains UIImageView or any other dynamically sized view with autolayout constraints that match leading, trailing, top, bottom to contentView, the cell resizes depending on the size of image in the imageView. Is it possible to get fixed size cells with UICollectionViewFlowLayout? I tried setting intrinsicContentSize for UIImageView in storyboard but it doesn't work.
Topic: UI Frameworks SubTopic: UIKit Tags:
0
0
404
Jun ’22
UITextView contentInset default animation
I have this code to adjust UITextView when keyboard is about to be displayed. @objc func keyboardWillShow(notification: NSNotification) { if let rectValue = notification.userInfo?[UIResponder.keyboardFrameEndUserInfoKey] as? NSValue { let keyboardSize = rectValue.cgRectValue.size let contentInsets: UIEdgeInsets = UIEdgeInsets(top: 0, left: 0, bottom: keyboardSize.height - view.safeAreaInsets.bottom, right: 0) textView.contentInset = contentInsets textView.scrollIndicatorInsets = contentInsets } } I see setting contentInset automatically creates a scrolling animation (after the keyboard appears fully) that makes the text at the cursor visible. The animation happens automatically and there is no way to disable it. I want to scroll to a given location in the textView sometimes when the keyboard is presented (preferably, animating it along with keyboard appearance). The implicit contentInset animation makes it impossible however. First the keyboard animates, then contentInset animates, and now only can I animate to my chosen location in textView. This is awkward. I am wondering what is the right way to achieve it?
Topic: UI Frameworks SubTopic: UIKit Tags:
0
0
585
Jul ’22
Workaround for iOS 16 Autorotation bugs
iOS 16 has serious bugs in UIKit without any known workarounds, so much that I had to remove my app from sale. Obviously I am affected and am desperate to know if anyone has found any workaround or if UIKit Engineers can tell me any workarounds. To summarise, the issue is calling setNeedsUpdateOfSupportedInterfaceOrientations shortly (within 0.5 seconds) after app launch doesn't trigger autorotation. It does trigger autorotation when the device is launched from the debugger but not when the app is launched directly. Maybe the debugger incurs some delay that is perhaps sufficient to trigger autorotation? I have filed FB11516363 but need a workaround desperately so as to get my app back on AppStore.
0
0
1.8k
Sep ’22
iOS 16 mismatch between windowScene & UIViewController interfaceOrientation
This is another strange issue on iOS 16, and one among the many woes that cause autoRotation problems on the platform. At the app startup time, there is a mismatch between -[UIViewController interfaceOrientation] (now deprecated), -[UIViewController supportedInterfaceOrientations] (which returns .landscapeRight), and -[windowScene interfaceOrientation].  public var windowOrientation: UIInterfaceOrientation {         return view.window?.windowScene?.interfaceOrientation ?? .unknown     }  override func viewDidLayoutSubviews() {         super.viewDidLayoutSubviews()      layoutInterfaceForOrientation(windowOrientation)     }  override var supportedInterfaceOrientations: UIInterfaceOrientationMask {         return .landscapeRight Putting a breakpoint in viewDidLayoutSubviews reveals that windowOrientation is portrait while interfaceOrientation is .landscapeRight. Is there anyway this can be fixed or any workaround possible?
Topic: UI Frameworks SubTopic: UIKit Tags:
0
0
553
Sep ’22
iOS 16.1 CDPurgeableResultCache logs
I keep getting repeated logs on iOS 16.1: [client] 114 CDPurgeableResultCache _recentPurgeableTotals no result for /private/var/wireless/baseband_data, setting to zero The logs fill the console and it's difficult to find useful logs that are there in my code. What can be done to disable these logs?
0
3
1.5k
Nov ’22
CVPixelBuffer HDR10 to BT.709 conversion using AVAssetWriter
I am getting CMSampleBuffers in kCVPixelFormatType_422YpCbCr10BiPlanarVideoRange format from the camera. The pixel buffers are 10 bit HDR. I need to record using ProRes422 codec but in non-HDR format. I am not sure what is a reliable way of doing this so reaching out here. What I did is simply set AVAssetWriter compression dictionary as follows: compressionSettings[AVVideoTransferFunctionKey] = AVVideoTransferFunction_ITU_R_709_2 compressionSettings[AVVideoColorPrimariesKey] = AVVideoColorPrimaries_ITU_R_709_2 compressionSettings[AVVideoYCbCrMatrixKey] = AVVideoYCbCrMatrix_ITU_R_709_2 It works and the end video recording shows the color space to be HD 1-1-1 with Apple ProRes codec. But I am not sure if AVAssetWriter has actually performed colorspace conversion from HDR 10 to BT.709, or simply clipped the colors out of range. I need to know a definitive way to achieve this. I see Apple's native camera app is doing this, but not sure how.
0
0
712
Mar ’23
AVPlayerViewController causing CPU spikes after deallocation
Dear AVKit Engineers, I see a strange bug in AVKit. Even after the viewcontroller hosting AVPlayerViewController is dismissed, I see CPU spiking to over 100% which is caused by some animation code still running after AVPlayerViewController no more exists ([AVMobileChromelessControlsViewController__animateSliderToTintState:duration:completionHandler:]). How does this code continue to run even after AVPlayerViewController is no more? And what can I do to fix it?
0
0
699
Jul ’23
Color conversion matrix YCbCr422 and YCbCr420 10 bit to RGB
I need to know correct color conversion matrices for converting YCbCr422 and YCbCr420 10 bit video range sample buffers (BT.2020 color space) to RGB. AVFoundation framework mentions AVVideoYCbCrMatrix_ITU_R_2020 which is a string constant. But I need to know the full matrix that can be used to perform color conversion. I have this matrix for full range BT2020, not sure if this is correct and what is the correct way to adapt it to video range. let colorMatrixBT2020_fullRange = ColorConversion(matrix: matrix_float3x3(columns: (simd_float3(1.0, 1.0, 1.0), simd_float3(0.000, -0.11156702/0.6780, 1.8814), simd_float3(1.4746, -0.38737742/0.6780, 0.000))), offset: vector_float3(0.0, -0.5, -0.5))
0
0
521
Oct ’23
AVCaptureDevice exception in setting WhiteBalance
I have the following code adapted from AVCamManual sample code to set white balance. I still see crash reports in analytics where exception is raised in setting WB: *** -[AVCaptureDevice temperatureAndTintValuesForDeviceWhiteBalanceGains:] whiteBalanceGains contain an out-of-range value - red, green, and blue gain Here is my code, it is not clear how things are turning out of range. public func normalizedGains(gains:AVCaptureDevice.WhiteBalanceGains) -> AVCaptureDevice.WhiteBalanceGains { var g = gains if let device = videoDevice { g.redGain = max(1.0, g.redGain) g.blueGain = max(1.0, g.blueGain) g.greenGain = max(1.0, g.greenGain) g.redGain = min(device.maxWhiteBalanceGain, g.redGain) g.blueGain = min(device.maxWhiteBalanceGain, g.blueGain) g.greenGain = min(device.maxWhiteBalanceGain, g.greenGain) } return g } And my code to set WB: public func setTemperatureAndTint( colorTemperature:Float?, tint:Float?) { if let device = videoDevice { var tint = tint var colorTemperature = colorTemperature if colorTemperature == nil { colorTemperature = device.temperatureAndTintValues(for: device.deviceWhiteBalanceGains).temperature } if tint == nil { tint = device.temperatureAndTintValues(for: device.deviceWhiteBalanceGains).tint } let temperatureTint = AVCaptureDevice.WhiteBalanceTemperatureAndTintValues(temperature: colorTemperature!, tint: tint!) NSLog("Setting tint \(temperatureTint.tint)") do { try device.lockForConfiguration() device.setWhiteBalanceModeLocked(with: normalizedGains(gains: device.deviceWhiteBalanceGains(for: temperatureTint)) , completionHandler: nil) device.unlockForConfiguration() wbLockedtoGray = false } catch { NSLog("Unable to change White balance gain \(error)") } } } Is there anything I am doing wrong?
0
0
501
Oct ’23