Audio Q&A

Certain apps (eg. DJay) seem to have direct or indirect access to the audio data from Apple Music, either on the device or remotely preprocessed. Is that feature available for other third party apps? Am I correct in my understanding that the new framework MusicUnderstanding (which looks quite interesting, judging from the docs so far) is not in any way usable for streams or local files from Apple Music / MusicKit since it relies entirely on AVReadOnlyAudioPCMBuffer? Is there eg. a way to get the same kind of music understanding data (or some other meta data suitable for real time visualization) provided directly form Apple Music without going via a readable audio buffer?

What's the best way of dealing with presentation mode (when the phone screen is captured by Google Meet or any other presenter software) when AudioSession is activated with .playAndRecord mode? The goal is to share the audio output between the phone and Google Meet so participants can hear it as well, while also enabling a microphone for future interactions within the app, for, let's say, voice chat.

Up to iOS 18 there were many voices available on device by default including "Nicky" that I use in my app. Somewhere around iOS 18.6 or 26.0 these voices were removed and no longer included in the OS by default. This broke my app that counted on the Nicky voice being available. My only workaround is to direct my users to download the voice again by going to Accessibility > VoiceOver > Speech > Voices and downloading "Nicky". I cannot simply switch to using Samantha (the only voice that is still available by default) because I have heavily customized how IPA pronunciations are tweaked to make Nicky speak Old English correctly. I tried briefly with Samantha but the results were not as good. I have 2 questions: Is there any way for me to include the "Nicky" voice in my app or for me to trigger the download from the app so I don't have to show an error and direct users through the settings app to get the voice? Are there any new features with AI voices in iOS 27 that I can use with IPA pronunciations to get them to speak possibly better than the Nicky voice?

With the release of liquid glass last year, I was trying to migrate my AUv3 to use native toolbars, and I noticed on macOS, AUv3's toolbar wouldn't display in Logic Pro (pretty sure it was the same with Ableton Live). But on iOS, it was fine. In the end, I decided to continue to use my custom toolbar because I had more control, but I thought I'd mention it here in case it can be fixed and warn the other users hitting the same issue. Another thing I noticed on the desktop is that presentation animations were removed. For example, AUv3 trying to show a popover on macOS inside, say Logic Pro, would make the popover appear instantly, with no animation. But animations worked great on iPadOS, however.

What's the best way of dealing with AVAudioInputNode from resources perspective? Ideally, to install it as soon as it is necessary for microphone access and use it as long as it is necessary (for example, when you have a long voice chat where users exchange voice messages intermittently AVAudioInputNode should be installed at the beginning and kept until the chat ends) Or is it better to setup it inputNode.installTap( onBus: bus, bufferSize: bufferSize, format: format, block: { buffer, _ in } ) everytime user accessing microphone with push to talk The question is what's better from a performance perspective

My app provides real-time microphone monitoring and level metering on iPhone, iPad, and Mac using AVAudioEngine. When users switch between built-in microphones, USB microphones, Bluetooth microphones, and Continuity Camera audio, the audio route often changes before audio buffers begin flowing. This can result in a brief period where the app sees a valid route but receives no audio data. What is Apple's recommended approach in iOS 26 and macOS 26 for determining when an input route is truly ready for real-time monitoring after a route change?

Hello team! Wanted to ask you if there is any resource to learn and practice PHASE, the theory in the official site is great but not much is out there to put it into practice with a real project, for instance. Thank you!

...when will they be supported natively?

What's the best way of detecting if player node / audio engine is in broken state (for example as a result of AVAudioSession.mediaServicesWereResetNotification). Sometimes it requires being reset, but from time to time it is not enought to just reset it for example playerNode.reset() engine.reset() But also required to reinitialize it playerNode.reset() playerNode = .init() engine.reset() engine = .init() Can we subscribe on the broken states and reinitialize them proactively?

This is kind of related to my other question, where I think AUv3's KVO to AUv2 C API notification mapping doesn't seem to work. For example, Reaper supports AUv3 plug-ins but I think it uses AUv2 APIs, and the developer told me that they observe these: kAudioUnitEvent_PropertyChange / kAudioUnitScope_Global / kAudioUnitProperty_ParameterList If one of them changes, they will rescan the parameter list/tree. My AUv3 Mela has a dynamic parameter tree, and changes based on the preset loaded, and it works well enough in Logic Pro for iPad/Mac and AUM on iOS. I know it sends out KVOs for the parameterTree property and I've also tried sending out allParameterValues KVOs but it seems Reaper is not getting these notifications. Anything I can do? (Reaper forum discussion, check last few messages https://forum.cockos.com/showthread.php?t=300840)

AUv3 was introduced back when iOS 9 came out, and it only had very minor updates. In a way, it didn't seem important enough for the desktop world, as they continue to use AUv2, and only iOS musicians/developers embraced AUv3. Even Logic Pro doesn't feel like it fully embraces it. It is almost like there was no good enough reason for the desktop world to switch over. Also, because of this, I think many bugs remain. One of the bugs I keep hitting, I think, is an AUv2 to AUv3 API mapping issue. Where AUv3 uses KVO for certain properties, and somehow doesn't work well if the host uses the C API. Here's one example: https://developer.apple.com/forums/thread/828549

Handling multilingual text. When a user selects a block of text that is mostly Chinese but contains embedded English words (e.g., technical terms in parentheses), the system reader often stutters, stops, or skips the English entirely. What is the best way to handle mixed-language text processing so that the speech engine can seamlessly and fluidly read Chinese and English together without dropping words?

I am designing an educational app. I notice that current system text-to-speech (like AVSpeechSynthesizer) often sounds too mechanical because the time intervals between characters are strictly equal, making it lack natural human prosody, phrasing, and warmth-which is a huge dealbreaker for sensitive users like children. How can we customize text-to-speech to break this uniform word-spacing, manage prosody dynamically, and make the Al voice sound more emotionally engaging and natural rather than a cold robot? I really want to create an elegant listening experience that feels like a real human storytelling, not just machine reading.