I have been allocating pixel buffers from CVPixelBufferPool and the code has been adapted from older various Apple sample codes such as RosyWriter. I see direct API such as CVPixelBufferCreate are highly performant and rarely cause frame drops as opposed to allocating from pixel buffer pool where I regularly get frame drops. Is this a known issue or a bad use of API? Here is the code for creating pixel buffer pool: private func createPixelBufferPool(_ width: Int32, _ height: Int32, _ pixelFormat: FourCharCode, _ maxBufferCount: Int32) -> CVPixelBufferPool? { var outputPool: CVPixelBufferPool? = nil let sourcePixelBufferOptions: NSDictionary = [kCVPixelBufferPixelFormatTypeKey: pixelFormat, kCVPixelBufferWidthKey: width, kCVPixelBufferHeightKey: height, kCVPixelFormatOpenGLESCompatibility: true, kCVPixelBufferIOSurfacePropertiesKey: [:] as CFDictionary] let pixelBufferPoolOptions: NSDictionary = [kCVPixelBufferPoolMinimumBufferCountKey: maxBufferCount] CVPixelBufferPoolCreate(kCFAllocatorDefault, pixelBufferPoolOptions, sourcePixelBufferOptions, &outputPool) return outputPool } private func createPixelBufferPoolAuxAttributes(_ maxBufferCount: size_t) -> NSDictionary { // CVPixelBufferPoolCreatePixelBufferWithAuxAttributes() will return kCVReturnWouldExceedAllocationThreshold if we have already vended the max number of buffers return [kCVPixelBufferPoolAllocationThresholdKey: maxBufferCount] } private func preallocatePixelBuffersInPool(_ pool: CVPixelBufferPool, _ auxAttributes: NSDictionary) { // Preallocate buffers in the pool, since this is for real-time display/capture var pixelBuffers: [CVPixelBuffer] = [] while true { var pixelBuffer: CVPixelBuffer? = nil let err = CVPixelBufferPoolCreatePixelBufferWithAuxAttributes(kCFAllocatorDefault, pool, auxAttributes, &pixelBuffer) if err == kCVReturnWouldExceedAllocationThreshold { break } assert(err == noErr) pixelBuffers.append(pixelBuffer!) } pixelBuffers.removeAll() } And here is the usage: bufferPool = createPixelBufferPool(outputDimensions.width, outputDimensions.height, outputPixelFormat, Int32(maxRetainedBufferCount)) if bufferPool == nil { NSLog("Problem initializing a buffer pool.") success = false break bail } bufferPoolAuxAttributes = createPixelBufferPoolAuxAttributes(maxRetainedBufferCount) preallocatePixelBuffersInPool(bufferPool!, bufferPoolAuxAttributes!) And then creating pixel buffers from pool err = CVPixelBufferPoolCreatePixelBufferWithAuxAttributes( kCFAllocatorDefault, bufferPool!, bufferPoolAuxAttributes, &dstPixelBuffer ) if err == kCVReturnWouldExceedAllocationThreshold { // Flush the texture cache to potentially release the retained buffers and try again to create a pixel buffer err = CVPixelBufferPoolCreatePixelBufferWithAuxAttributes( kCFAllocatorDefault, bufferPool!, bufferPoolAuxAttributes, &dstPixelBuffer ) } if err != 0 { if err == kCVReturnWouldExceedAllocationThreshold { NSLog("Pool is out of buffers, dropping frame") } else { NSLog("Error at CVPixelBufferPoolCreatePixelBuffer %d", err) } break bail } When used with AVAssetWriter, I see lot of frame drops caused due to kCVReturnWouldExceedAllocationThreshold error. No frame drops are seen when I directly allocate the pixel buffer without using a pool: CVPixelBufferCreate(kCFAllocatorDefault, Int(dimensions.width), Int(dimensions.height), outputPixelFormat, sourcePixelBufferOptions, &dstPixelBuffer) What could be the cause?

I have two CIContexts configured with the following options: let options1:[CIContextOption:Any] = [CIContextOption.cacheIntermediates: false, CIContextOption.outputColorSpace: NSNull(), CIContextOption.workingColorSpace: NSNull()]; let options2:[CIContextOption:Any] = [CIContextOption.cacheIntermediates: false]; And an MTKView with CAMetalLayer configured with HDR output. metalLayer = self.layer as? CAMetalLayer metalLayer?.wantsExtendedDynamicRangeContent = true metalLayer.colorspace = CGColorSpace(name: CGColorSpace.itur_2100_HLG) colorPixelFormat = .bgr10a2Unorm The two context options produce different outputs when input is in BT.2020 pixel buffers. But I believe the outputs shouldn't be different. Because the first option simply disables color management. The second one performs intermediate buffer calculations in sRGB extended linear color space and then converts those buffers to BT.2020 color space in the output.

I have set AVCaptureVideoDataOutput with 10-bit 420 YCbCr sample buffers. I use Core Image to process these pixel buffers for simple scaling/translation. var dstBounds = CGRect.zero dstBounds.size = dstImage.extent.size /* *srcImage is created from sample buffer received from Video Data Output */ _ciContext.render(dstImage, to: dstPixelBuffer!, bounds: dstImage.extent, colorSpace: srcImage.colorSpace ) I then set the color attachments to this dstPixelBuffer using set colorProfile in the app settings (BT.709 or BT.2020). switch colorProfile { case .BT709: CVBufferSetAttachment(dstPixelBuffer!, kCVImageBufferColorPrimariesKey, kCVImageBufferColorPrimaries_ITU_R_709_2, .shouldPropagate) CVBufferSetAttachment(dstPixelBuffer!, kCVImageBufferTransferFunctionKey, kCVImageBufferTransferFunction_ITU_R_709_2, .shouldPropagate) CVBufferSetAttachment(dstPixelBuffer!, kCVImageBufferYCbCrMatrixKey, kCVImageBufferYCbCrMatrix_ITU_R_709_2, .shouldPropagate) case .HLG2100: CVBufferSetAttachment(dstPixelBuffer!, kCVImageBufferColorPrimariesKey, kCVImageBufferColorPrimaries_ITU_R_2020, .shouldPropagate) CVBufferSetAttachment(dstPixelBuffer!, kCVImageBufferTransferFunctionKey, kCVImageBufferTransferFunction_ITU_R_2100_HLG, .shouldPropagate) CVBufferSetAttachment(dstPixelBuffer!, kCVImageBufferYCbCrMatrixKey, kCVImageBufferYCbCrMatrix_ITU_R_2020, .shouldPropagate) } These pixel buffers are then vended to AVAssetWriter whose videoSettings is set to recommendedSettings by VDO. But the output seems to be washed out completely, esp. for SDR (BT.709). What am I doing wrong?

I am trying to use the new API CAEDRMetadata.hlg(ambientViewingEnvironment:) introduced in iOS 17.0. Since ambientViewingEnvironmentData is dynamic, I understand the edrMetaData of CAMetalLayer needs to be set on every draw call. But doing so causes CAMetalLayer to freeze and even crash. if let pixelBuffer = image.pixelBuffer, let aveData = pixelBuffer.attachments.propagated[kCVImageBufferAmbientViewingEnvironmentKey as String] as? Data { if #available(iOS 17.0, *) { metalLayer.edrMetadata = CAEDRMetadata.hlg(ambientViewingEnvironment: aveData) } else { // Fallback on earlier versions } }

I understand that by default, Core image uses extended linear sRGB as default working color space for executing kernels. This means that the color values received (or sampled from sampler) in the Metal Core Image kernel are linear values without gamma correction applied. But if we disable color management by setting let options:[CIContextOption:Any] = [CIContextOption.workingColorSpace:NSNull()]; do we receive color values as it exists in the input texture (which may have gamma correction already applied)? In other words, the color values received in the kernel are gamma corrected and we need to manually convert them to linear values in the Metal kernel if required?

I am trying to carefully process HDR pixel buffers (10-bit YCbCr buffers) from the camera. I have watched all WWDC videos on this topic but have some doubts expressed below. Q. What assumptions are safe to make about sample values in Metal Core Image Kernels? Are the sample values received in Metal Core Image kernel linear or gamma corrected? Or does that depend on workingColorSpace property, or the input image that is supplied (though imageByMatchingToColorSpace() API, etc.)? And what could be the max and min values of these samples in either case? I see that setting workingColorSpace to NSNull() in context creation options will guarantee receiving the samples as is and normalised to [0-1]. But then it's possible the values are non-linear gamma corrected, and extracting linear values would involve writing conversion functions in the shader. In short, how do you safely process HDR pixel buffers received from the camera (which are in YCrCr420_10bit, which I believe have gamma correction applied, so Y in YCbCr is actually Y'. Can AVFoundation team clarify this?) ?

The title says it all, XCode 15 freezes in so many cases that it has to be force quit and reopened, be it opening Swift packages or recognising connected devices. This happens on Intel based 2018 Macbook Pro. Is it the problem on Intel based Macs or XCode 15 is super buggy on all devices? For instance, I can not open and build this project downloaded from Github, specifically opening the file named MetalViewUI.swift in the project, it hangs forever.

I have been using MTKView to display CVPixelBuffer from the camera. I use so many options to configure color space of the MTKView/CAMetalLayer that may be needed to tonemap content to the display (CAEDRMetadata for instance). If however I use AVSampleBufferDisplayLayer, there are not many configuration options for color matching. I believe AVSampleBufferDisplayLayer uses pixel buffer attachments to determine the native color space of the input image and does the tone mapping automatically. Does AVSampleBufferDisplayLayer have any limitations compared to MTKView, or both can be used without any compromise on functionality?

I am embedding SwiftUI VideoPlayer in a VStack and see that the screen goes black (i.e. the content disappears even though video player gets autorotated) when the device is rotated. The issue happens even when I use AVPlayerViewController (as UIViewControllerRepresentable). Is this a bug or I am doing something wrong? var videoURL:URL let player = AVPlayer() var body: some View { VStack { VideoPlayer(player: player) .frame(maxWidth:.infinity) .frame(height:300) .padding() .ignoresSafeArea() .background { Color.black } .onTapGesture { player.rate = player.rate == 0.0 ? 1.0 : 0.0 } Spacer() } .ignoresSafeArea() .background(content: { Color.black }) .onAppear { let audioSession = AVAudioSession.sharedInstance() do { try audioSession.setCategory(AVAudioSession.Category.playback, mode: AVAudioSession.Mode.default, options: AVAudioSession.CategoryOptions.duckOthers) } catch { NSLog("Unable to set session category to playback") } let playerItem = AVPlayerItem(url: videoURL) player.replaceCurrentItem(with: playerItem) } }

I am trying to recreate Discrete scrubber in SwiftUI with haptic feedback and snap to nearest integer step. I use ScrollView and LazyHStack as follows: struct DiscreteScrubber: View { @State var numLines:Int = 100 var body: some View { ScrollView(.horizontal, showsIndicators: false) { LazyHStack { ForEach(0..<numLines, id: \.self) { _ in Rectangle().frame(width: 2, height: 10, alignment: .center) .foregroundStyle(Color.red) Spacer().frame(width: 10) } } } } } Problem: I need to add content inset of half the frame width of ScrollView so that the first line in the scrubber starts at the center of the view and so does the last line, and also generate haptic feedback as it scrolls. This was easy in UIKit but not obvious in SwiftUI.

I am trying to build a video editing timeline using SwiftUI which consists of a series of trimmers (sample code for trimmer below). I plan to embed multiple of these trimmers in an HStack to make an editing timeline (HStack in turn will be embedded in a ScrollView). What I want to achieve is that if I trim end of any of these trimmers by dragging it's left/right edge, the other trimmers on timeline should move left/right to fill the gap. I understand as the view shrinks/expands during trimming, there might be a need for spacers on the edges of HStack whose widths need to be adjusted while trimming is ongoing. Right now the code I have for the trimmer uses a fixed frameWidth of the view, so the view occupies full space even if the trimming ends move. It's not clear how to modify my code below to achieve what I want. This was pretty much possible to do in UIKit by the way. import SwiftUI struct SimpleTrimmer: View { @State var frameWidth:CGFloat = 300 let minWidth: CGFloat = 30 @State private var leftOffset: CGFloat = 0 @State private var rightOffset: CGFloat = 0 @GestureState private var leftDragOffset: CGFloat = 0 @GestureState private var rightDragOffset: CGFloat = 0 private var leftAdjustment: CGFloat { // NSLog("Left offset \(leftOffset + leftDragOffset)") var adjustment = max(0, leftOffset + leftDragOffset) if frameWidth - rightOffset - adjustment - 60 < minWidth { adjustment = frameWidth - rightOffset - minWidth - 60.0 } return adjustment } private var rightAdjustment: CGFloat { var adjustment = max(0, rightOffset - rightDragOffset) if frameWidth - adjustment - leftOffset - 60 < minWidth { adjustment = frameWidth - leftOffset - 60 - minWidth } return adjustment } var body: some View { HStack(spacing: 10) { Image(systemName: "chevron.compact.left") .frame(width: 30, height: 70) .background(Color.blue) .offset(x: leftAdjustment) .gesture( DragGesture(minimumDistance: 0) .updating($leftDragOffset) { value, state, trans in state = value.translation.width } .onEnded { value in var maxLeftOffset = max(0, leftOffset + value.translation.width) if frameWidth - rightAdjustment - maxLeftOffset - 60 < minWidth { maxLeftOffset = frameWidth - rightAdjustment - minWidth - 60 } leftOffset = maxLeftOffset } ) Spacer() Image(systemName: "chevron.compact.right") .frame(width: 30, height: 70) .background(Color.blue) .offset(x: -rightAdjustment) .gesture( DragGesture(minimumDistance: 0) .updating($rightDragOffset) { value, state, trans in state = value.translation.width } .onEnded { value in var minRightOffset = max(0, rightOffset - value.translation.width) if minRightOffset < leftAdjustment - 60 - minWidth { minRightOffset = leftAdjustment - 60 - minWidth } rightOffset = minRightOffset } ) } .foregroundColor(.black) .font(.title3.weight(.semibold)) .padding(.horizontal, 7) .padding(.vertical, 3) .background { RoundedRectangle(cornerRadius: 7) .fill(.yellow) .padding(.leading, leftAdjustment) .padding(.trailing, rightAdjustment) } .frame(width: frameWidth) } } #Preview { SimpleTrimmer() }

I want to understand the utility of using AsyncStream when iOS 17 introduced @Observable macro where we can directly observe changes in the value of any variable in the model(& observation tracking can happen even outside SwiftUI view). So if I am observing a continuous stream of values, such as download progress of a file using AsyncStream in a SwiftUI view, the same can be observed in the same SwiftUI view using onChange(of:initial) of download progress (stored as a property in model object). I am looking for benefits, drawbacks, & limitations of both approaches. Specifically, my question is with regards to AVCam sample code by Apple where they observe few states as follows. This is done in CameraModel class which is attached to SwiftUI view. // MARK: - Internal state observations // Set up camera's state observations. private func observeState() { Task { // Await new thumbnails that the media library generates when saving a file. for await thumbnail in mediaLibrary.thumbnails.compactMap({ $0 }) { self.thumbnail = thumbnail } } Task { // Await new capture activity values from the capture service. for await activity in await captureService.$captureActivity.values { if activity.willCapture { // Flash the screen to indicate capture is starting. flashScreen() } else { // Forward the activity to the UI. captureActivity = activity } } } Task { // Await updates to the capabilities that the capture service advertises. for await capabilities in await captureService.$captureCapabilities.values { isHDRVideoSupported = capabilities.isHDRSupported cameraState.isVideoHDRSupported = capabilities.isHDRSupported } } Task { // Await updates to a person's interaction with the Camera Control HUD. for await isShowingFullscreenControls in await captureService.$isShowingFullscreenControls.values { withAnimation { // Prefer showing a minimized UI when capture controls enter a fullscreen appearance. prefersMinimizedUI = isShowingFullscreenControls } } } } If we see the structure CaptureCapabilities, it is a small structure with two Bool members. These changes could have been directly observed by a SwiftUI view. I wonder if there is a specific advantage or reason to use AsyncStream here & continuously iterate over changes in a for loop. /// A structure that represents the capture capabilities of `CaptureService` in /// its current configuration. struct CaptureCapabilities { let isLivePhotoCaptureSupported: Bool let isHDRSupported: Bool init(isLivePhotoCaptureSupported: Bool = false, isHDRSupported: Bool = false) { self.isLivePhotoCaptureSupported = isLivePhotoCaptureSupported self.isHDRSupported = isHDRSupported } static let unknown = CaptureCapabilities() }

I imported few files in my Xcode project from other projects using drag and drop. Even though the files are copied in the new project and there are no softlinks pointing to the location of other project, the issue is whenever I do a git commit and push, Xcode keeps showing all the projects to commit to rather than just the current project. There seem to be no setting to permanently remove git dependency of other projects. Is there anyway to remove references to other projects?

I have this build error with Xcode 26 beta 2: var asset:AVURLAsset? func loadAsset() { let assetURL = URL.documentsDirectory .appendingPathComponent("sample.mov") asset = AVURLAsset(url: assetURL, options: [AVURLAssetPreferPreciseDurationAndTimingKey: true]) /*Error: Type of expression is ambiguous without a type annotation */ if let result = try? await asset?.load(.tracks, .isPlayable, .isComposable) { } } Is there an issue with try? in the new Swift compiler? Error: Type of expression is ambiguous without a type annotation

XCode 26 beta 2 is taking more than 20 seconds to start AVCaptureSession when AVCaptureVideoDataOutput and AVCaptureAudioDataOutput are added. The problem occurs only during debugging and is clearly seen with Cinematic Capture sample code by Apple. I am using iPhone 14 Pro running iOS 26 beta 2 for reference.