After upgrading to watchOS 26, users report that when playing music on Apple Watch, if a fitness reminder is received, the music automatically pauses and users need to manually tap the play button to resume music playback. This phenomenon occurs with multiple music and podcast apps.
This issue did not exist before the upgrade. We would like to know if this is an Apple bug or if there are any special development configurations needed?"
Audio
RSS for tagDive into the technical aspects of audio on your device, including codecs, format support, and customization options.
Selecting any option will automatically load the page
Post
Replies
Boosts
Views
Activity
Overview
We are producing audio in real time from an editing application and are trying to put that on an HLS stream. We attempt to submit PCM samples through an audio writer but are getting a crash after a select number of samples have been appended.
Depending on the number of audio frames in the PCM buffer, we might get more iterations before the crash but it always has the same traceback (see below).
Code
The setup is rather simple. We took inspiration from a few sources around the web.
NSMutableDictionary *audio = [[NSMutableDictionary alloc] init];
[audio setObject:@(kAudioFormatMPEG4AAC) forKey:AVFormatIDKey];
[audio setObject:[NSNumber numberWithInt:config.audioSampleRate] // 48000
forKey:AVSampleRateKey];
[audio setObject:[NSNumber numberWithInt:config.audioChannels] // 2
forKey:AVNumberOfChannelsKey];
[audio setObject:@160000 forKey:AVEncoderBitRateKey];
m_audioConfig = [[NSDictionary alloc] initWithDictionary:audio];
m_audio = [[AVAssetWriterInput alloc] initWithMediaType:AVMediaTypeAudio
outputSettings:m_audioConfig];
AVAudioFrameCount audioFrames = BUFFER_SAMPLES * bCount;
AVAudioPCMBuffer *pcmBuffer = [[AVAudioPCMBuffer alloc] initWithPCMFormat:m_full.pcmFormat
frameCapacity:audioFrames];
pcmBuffer.frameLength = pcmBuffer.frameCapacity;
AudioChannelLayout layout;
memset(&layout, 0, sizeof(layout));
layout.mChannelLayoutTag = kAudioChannelLayoutTag_Stereo;
CMFormatDescriptionRef format;
OSStatus stats = CMAudioFormatDescriptionCreate(
kCFAllocatorDefault,
pcmBuffer.format.streamDescription,
sizeof(layout),
&layout,
0,
nil,
nil,
&format
);
for (int i = 0; i < bCount; i++)
{
AudioPCM pcm;
audioCallback->callback(pcm);
memcpy(*(pcmBuffer.int16ChannelData) + (bufferSize * i), pcm.data, bufferSize);
}
size_t samplesConsumed = BUFFER_SAMPLES * bCount;
CMSampleBufferRef sampleBuffer;
CMSampleTimingInfo timing;
timing.duration = CMTimeMake(1, config.audioSampleRate);
timing.presentationTimeStamp = presentationTime;
timing.decodeTimeStamp = kCMTimeInvalid;
OSStatus ostatus = CMSampleBufferCreate(
kCFAllocatorDefault,
nil,
false,
nil,
nil,
format,
(CMItemCount)pcmBuffer.frameLength,
1,
&timing,
0,
nil,
&sampleBuffer
);
////
ostatus = CMSampleBufferSetDataBufferFromAudioBufferList(
sampleBuffer,
kCFAllocatorDefault,
kCFAllocatorDefault,
kCMSampleBufferFlag_AudioBufferList_Assure16ByteAlignment,
pcmBuffer.audioBufferList
);
if (ostatus != noErr)
{
NSLog(@"fill audio sample from buffer list failed: %s", logAudioError(ostatus));
return;
}
ostatus = CMSampleBufferSetDataReady(sampleBuffer);
if (ostatus != noErr)
{
NSLog(@"set sample buffer ready failed: %s", logAudioError(ostatus));
return;
}
// Finally we can attach it, then shove the presentation time forward
[m_audio appendSampleBuffer:sampleBuffer];
The Crash
The crash points towards some level of deallocation when the conversion tooling is done or has enough samples to process an output packet? It's had to say.
0 caulk 0x1a1e9532c caulk::alloc::tiered_allocator<caulk::alloc::size_range_tier<0ul, 1008ul, caulk::alloc::tree_allocator<caulk::alloc::chunk_allocator<caulk::alloc::page_allocator, caulk::alloc::bitmap_allocator, caulk::alloc::embed_block_memory, 16384ul, 16ul, 6ul>>>, caulk::alloc::size_range_tier<1009ul, 256000ul, caulk::alloc::guarded_edges_allocator<caulk::alloc::consolidating_free_map<caulk::alloc::page_allocator, 10485760ul>, 4ul>>, caulk::alloc::tracking_allocator<caulk::alloc::page_allocator>>::deallocate(caulk::alloc::block, unsigned long) + 636
1 AudioToolboxCore 0x1993fbfe4 ExtendedAudioBufferList_Destroy + 112
2 AudioToolboxCore 0x1993d5fe0 std::__1::__optional_destruct_base<ACCodecOutputBuffer, false>::~__optional_destruct_base[abi:ne180100]() + 68
3 AudioToolboxCore 0x1993d5f48 acv2::CodecConverter::~CodecConverter() + 196
4 AudioToolboxCore 0x1993d5e5c acv2::CodecConverter::~CodecConverter() + 16
5 AudioToolboxCore 0x1992574d8 std::__1::vector<std::__1::unique_ptr<acv2::AudioConverterBase, std::__1::default_delete<acv2::AudioConverterBase>>, std::__1::allocator<std::__1::unique_ptr<acv2::AudioConverterBase, std::__1::default_delete<acv2::AudioConverterBase>>>>::__clear[abi:ne180100]() + 84
6 AudioToolboxCore 0x199259acc acv2::AudioConverterChain::RebuildConverterChain(acv2::ChainBuildSettings const&) + 116
7 AudioToolboxCore 0x1992596ec acv2::AudioConverterChain::SetProperty(unsigned int, unsigned int, void const*) + 1808
8 AudioToolboxCore 0x199324acc acv2::AudioConverterV2::setProperty(unsigned int, unsigned int, void const*) + 84
9 AudioToolboxCore 0x199327f08 with_resolved(OpaqueAudioConverter*, caulk::function_ref<int (AudioConverterAPI*)>) + 60
10 AudioToolboxCore 0x1993281e4 AudioConverterSetProperty + 72
11 MediaToolbox 0x1a7566c2c FigSampleBufferProcessorCreateWithAudioCompression + 2296
12 MediaToolbox 0x1a754db08 0x1a70b5000 + 4819720
13 MediaToolbox 0x1a754dab4 FigMediaProcessorCreateForAudioCompressionWithFormatWriter + 100
14 MediaToolbox 0x1a77ebb98 0x1a70b5000 + 7564184
15 MediaToolbox 0x1a7804158 0x1a70b5000 + 7663960
16 MediaToolbox 0x1a7801da0 0x1a70b5000 + 7654816
17 AVFCore 0x1ada530c4 -[AVFigAssetWriterTrack addSampleBuffer:error:] + 192
18 AVFCore 0x1ada55164 -[AVFigAssetWriterAudioTrack _flushPendingSampleBuffersReturningError:] + 500
19 AVFCore 0x1ada55354 -[AVFigAssetWriterAudioTrack addSampleBuffer:error:] + 472
20 AVFCore 0x1ada4ebf0 -[AVAssetWriterInputWritingHelper appendSampleBuffer:error:] + 128
21 AVFCore 0x1ada4c354 -[AVAssetWriterInput appendSampleBuffer:] + 168
22 lib_devapple_hls.dylib 0x115d2c7cc detail::AppleHLSImplementation::audioRuntime() + 1052
23 lib_devapple_hls.dylib 0x115d2d094 void* std::__1::__thread_proxy[abi:ne180100]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void (detail::AppleHLSImplementation::*)(), detail::AppleHLSImplementation*>>(void*) + 72
24 libsystem_pthread.dylib 0x196e5b2e4 _pthread_start + 136
Any insight would be welcome!
Hi all!
I have been experiencing some issues when using the AVAudioEngine to play audio and record input while doing a voice chat (through the PTT Interface).
I noticed if I connect any players to the AudioGraph OR call start that the audio session becomes active (this is on iOS).
I don't see anything in the docs or the header files in the AVFoundation, but is it possible that calling the stop method on an engine deactivates the audio session too?
In a normal app this behavior seems logical, but when using PTT all activation and deactivation of the audio session must go through the framework and its delegate methods.
The issue I am debugging is that when the engine with the input node tapped gets stopped, and there is a gap between the input and when the server replies with inbound audio to be played and something seems to be getting the hardware/audio session into a jammed state.
Thanks for any feedback and/or confirmation on this behavior!
Hello,
i can successfully match music using shazamkit on Apple using SwiftUI, a simple app that let user to load an audio file and exctracts the relative match, while i am unable to match music using shamzamkit on Android. I am trying to make the same simple app but i cannot match music as i get MATCH_ATTEMPT_FAILED every time i try to. I don't know what i am doing wrong but the shazam part in the kotlin Android code is in this method :
suspend fun processAudioFileInBackground(
filePath: String,
developerTokenProvider: DeveloperTokenProvider
) = withContext(Dispatchers.IO) {
val bufferSize = 1024 * 1024
val audioFile = FileInputStream(filePath)
val byteBuffer = ByteBuffer.allocate(bufferSize)
byteBuffer.order(ByteOrder.LITTLE_ENDIAN)
var bytesRead: Int
while (audioFile.read(byteBuffer.array()).also { bytesRead = it } != -1) {
val signatureGenerator = (ShazamKit.createSignatureGenerator(AudioSampleRateInHz.SAMPLE_RATE_44100) as ShazamKitResult.Success).data
signatureGenerator.append(byteBuffer.array(), bytesRead, System.currentTimeMillis())
val signature = signatureGenerator.generateSignature()
println("Signature: ${signature.durationInMs}")
val catalog = ShazamKit.createShazamCatalog(developerTokenProvider, Locale.ENGLISH)
val session = (ShazamKit.createSession(catalog) as ShazamKitResult.Success).data
val matchResult = session.match(signature)
println("MatchResult : $matchResult")
setMatchResult(matchResult)
byteBuffer.clear()
}
audioFile.close()
}
I noticed that changing Locale in catalog creation results in different result as i get NoMatch without exception. Can you please help me with this? Do i need to create a custom catalog?
My app utilizes background audio to play music files. I have the audio background mode enabled and I initialize the AVAudioSession in playback mode with the mixWithOthers option. And it usually works great while the app is backgrounded. I listen for audio interruptions as well as route changes and I am able to handle them appropriately and I can usually resume my background audio no problem.
I discovered an issue while connected to CarPlay though. Roughly 50% of the time when I disconnect from a phone call while connected to CarPlay I get the following error after calling the play() method of my AVAudioPlayer instance:
"ATAudioSessionClientImpl.mm:281 activation failed. status = 561015905"
If I instead try to start a new audio session I get a similar error:
Error Domain=NSOSStatusErrorDomain Code=561015905 "Session activation failed" UserInfo={NSLocalizedDescription=Session activation failed}
Like I said, this isn't reproducible 100% of the time and is so far only seen while connected to CarPlay. I don't think Im forgetting so additional capability or plist setting, but if anyone has any clues it would be greatly appreciated. Otherwise this is likely just a bug that I need to report to Apple.
One very important note, and reason I believe it's just a bug, is that while I was testing I found that other music apps like Spotify will also fail to resume their audio at the same time my app fails.
Another important detail is that when it works successfully I receive the audio session interruption ended notification, and when it doesn't work I only receive a route configuration change or route override notification. From there I am able to still successfully granted background time to execute code, but my call to resume audio fails with the above mentioned error codes.
I'm using a 4 channel USB Audio interface, with 4 microphones, and want to process them through 4 independent effect chains. However the output from AVAudioInputNode is a single 4 channel bus. How can I split this into 4 mono busses?
The following code splits the input into 4 copies, and routes them through the effects, but each bus contains all four channels. How can I remap the channels to remove the unwanted channels from the bus? I tried using channelMap on the mixer node but that had no effect.
I'm currently using this code primarily on iOS but it should be portable between iOS and MacOS. It would be possible to do this through a Matrix Mixer Node, but that seems completely overkill, for such a basic operation. I'm already using a Matrix Mixer to combine the inputs, and it's not well supported in AVAudioEngine.
AVAudioInputNode *inputNode=[engine inputNode];
[inputNode setVoiceProcessingEnabled:NO error:nil];
NSMutableArray *micDestinations=[NSMutableArray arrayWithCapacity:trackCount];
for(i=0;i<trackCount;i++)
{
fixMicFormat[i]=[AVAudioMixerNode new];
[engine attachNode:fixMicFormat[i]];
// And create reverb/compressor and eq the same way...
[engine connect:reverb[i] to:matrixMixerNode fromBus:0 toBus:i format:nil];
[engine connect:eq[i] to:reverb[i] fromBus:0 toBus:0 format:nil];
[engine connect:compressor[i] to:eq[i] fromBus:0 toBus:0 format:nil];
[engine connect:fixMicFormat[i] to:compressor[i] fromBus:0 toBus:0 format:nil];
[micDestinations addObject:[[AVAudioConnectionPoint alloc] initWithNode:fixMicFormat[i] bus:0] ];
}
AVAudioFormat *inputFormat = [inputNode outputFormatForBus: 1];
[engine connect:inputNode toConnectionPoints:micDestinations fromBus:1 format:inputFormat];
Everytime I put my AirPods in and connect them to my phone or my Mac or my iPad since the iOS 18.3 update on my devices they’ve been disconnecting without reason, pausing songs I’m in the middle of playing, and only partially reconnecting in one pod and it’s getting really frustrating
Topic:
Media Technologies
SubTopic:
Audio
Hi,
I'm working on an audio mixing app, that comes with bundled audio units that provide some of the app's core functionality.
For the next release of that app, we are planning to make two changes:
make the app sandboxed
package the bundled audio units as .appex bundles instead as .component bundles, so we don't need to take care of the installation at the correct spot in the file system
When trying this new approach, we run into problems where [[AVAudioUnitEffect alloc] initWithAudioComponentDescription:] crashes when trying to load our audio unit with the exception:
AVAEInternal.h:109 [AUInterface.mm:468:AUInterfaceBaseV3: (AudioComponentInstanceNew(comp, &_auv2)): error -10863
Our audio unit has the `sandboxSafe flag enabled, and loads fine when the host app is not sandboxed, so I'm guessing I got the bundle id/code signing requirements for the .appex correct.
It seems, that my .appex isn't even loaded, and the system rejects it because of its metadata. Maybe there something wrong the Info.plist generated by Juice?
"BuildMachineOSBuild" => "23H222"
"CFBundleDisplayName" => "elgato_sample_recorder"
"CFBundleExecutable" => "ElgatoSampleRecorder"
"CFBundleIdentifier" => "com.iwascoding.EffectLoader.samplerecorderAUv3"
"CFBundleName" => "elgato_sample_recorder"
"CFBundlePackageType" => "XPC!"
"CFBundleShortVersionString" => "1.0.0.0"
"CFBundleSignature" => "????"
"CFBundleSupportedPlatforms" => [
0 => "MacOSX"
]
"CFBundleVersion" => "1.0.0.0"
"DTCompiler" => "com.apple.compilers.llvm.clang.1_0"
"DTPlatformBuild" => "24C94"
"DTPlatformName" => "macosx"
"DTPlatformVersion" => "15.2"
"DTSDKBuild" => "24C94"
"DTSDKName" => "macosx15.2"
"DTXcode" => "1620"
"DTXcodeBuild" => "16C5032a"
"LSMinimumSystemVersion" => "10.13"
"NSExtension" => {
"NSExtensionAttributes" => {
"AudioComponents" => [
0 => {
"description" => "Elgato Sample Recorder"
"factoryFunction" => "elgato_sample_recorderAUFactoryAUv3"
"manufacturer" => "Manu"
"name" => "Elgato: Elgato Sample Recorder"
"sandboxSafe" => 1
"subtype" => "Znyk"
"tags" => [
0 => "Effects"
]
"type" => "aufx"
"version" => 65536
}
]
}
"NSExtensionPointIdentifier" => "com.apple.AudioUnit-UI"
"NSExtensionPrincipalClass" => "elgato_sample_recorderAUFactoryAUv3"
}
"NSHighResolutionCapable" => 1
}
Any ideas what I am missing?
I have a Catalyst app ('container') which hosts an embedded AUv3 Audio Unit extension ('plugin'). This used to work for years and has worked with this project until a few days ago.
it still works on iOS as expected
on MacOS the extension is never registered/installed and won't load
extension won't show up with AUVal
seems to have stopped working with the 26.1 XCode update
I'm fairly certain the problem is not code related (i.e. likely build settings, project settings, entitlements, signing, etc.)
I have compared all settings with another still-working project and can't find any meaningful difference
(I can't request code-level support because even the minimal thing vastly exceeds the 250 lines of code limit.)
How can I debug the issue? I literally don't know where to start to fix this problem, short of rebuilding the entire thing and hope that it magically starts working again.
I am work an app development on an app which request an audio function in background as an alert sound.
during debug testing , the function work fine,
but once I testing standalone without debugging , The function not work , it will play out the sound when I back to app.
does any way to trace the issues ?
I’m running the iOS 26.2 Public Beta update and my album artwork is missing from the music app (I’m not using Apple Music). I use google to get my album artwork. Do I need to wait for a new update?
Topic:
Media Technologies
SubTopic:
Audio
Environment→ ・Device: iPad 10th generation ・OS:**iOS18.3.2
We're using AVAudioPlayer to play a sound when a button is tapped. In our use case, this button can be tapped very frequently — roughly every 0.1 to 0.2 seconds. Each tap triggers the following function:
var audioPlayer: AVAudioPlayer?
func soundPlay(resource: String, type: String){
guard let path = Bundle.main.path(forResource: resource, ofType: type) else {
return
}
do {
audioPlayer = try AVAudioPlayer(contentsOf: URL(fileURLWithPath: path))
audioPlayer!.delegate = self
try audioSession.setCategory(.playback)
} catch {
return
}
self.audioPlayer!.play()
}
The issue is that under high-frequency tapping (especially around 0.1–0.15s intervals), the app occasionally crashes. The crash does not occur every time, but it happens randomly — sometimes within 30 seconds, within 1 minute, or even 3 minutes of continuous tapping.
Interestingly, adding a delay of 0.2 seconds between button taps seems to prevent the crash entirely. Delays shorter than 0.2 seconds (e.g.,0.15s,0.18s) still result in occasional crashes.
My questions are:
**Is this expected behavior from AVAudioPlayer or AVAudioSession?
Could this be a known issue or a limitation in AVFoundation?
Is there any documentation or guidance on handling frequent sound playback safely?**
Any insights or recommendations on how to handle rapid, repeated audio playback more reliably would be appreciated.
Hi folks - I'm having trouble finding specific documentation about Audio Unit MIDI plugins - as in MIDI -only. Any suggestions welcome as searches aren't returning much. (too niche? user error?)
Topic:
Media Technologies
SubTopic:
Audio
Hi,
On macOS I used to open MP3 and MP4 files with ExtAudioFile. For a few years it doesn't work anymore.
So I decided to try different macOS API using the AudioFileID of AudioToolbox framework.
I decided to write a test:
https://gist.github.com/joelkraehemann/7f5b241b52ca38c3a765c138fb647588
It fails right here:
AudioFileOpenWithCallbacks()
By telling OSStatus error 1954115647, which means kAudioFileUnsupportedFileTypeError.
The filename was set to an MP4 file:
~/Music/test.mp4
Howto fix this?
regards, Joël
I've got a web app built with MusicKit that displays a list of songs.
I have player controls for play, pause, skip next, skip, previous, toggle shuffle and set repeat mode.
All of these work by using music.
The play button, when nothing is playing and nothing is in the queue, will enqueue all the tracks and start playing with the below, for example:
await music.setQueue({ songs, startPlaying: true });
I've implemented a progress slider based on feedback from the "playbackProgressDidChange" listener.
Now, how in the world can I set the volume? This seems like it should be simple, but I am at a complete loss here.
The docs say:
"The volume of audio playback, which is set directly on the HTMLMediaElement as the HTMLMediaElement.volume property. This value ranges between 0, which would be muting the audio, and 1, which would be the loudest possible."
Given that all my controls work off the music instance, I don't understand how I can do that.
In this video from WWDC 2022, music web components are touched on briefly. These are also documented very sparsely. The volume docs are here.
For the life of me, I can't even get the volume web component to display in the UI.
It appears that MusicKit Web is hobbled compared to the native implementation, but surely adjusting volume shouldn't be that hard right?
I'd appreciate any insight on how to do this, including how to get web components to work (in a Next JS app).
Thanks.
After updating to 18.3 my iPad 7th generation has no sound and will not play videos
3
I am working on an application to get when input audio device is being used. Basically I want to know the application using the microphone (built-in or external)
This app runs on macOS. For Mac versions starting from Sonoma I can use this code:
int getAudioProcessPID(AudioObjectID process)
{
pid_t pid;
if (@available(macOS 14.0, *)) {
constexpr AudioObjectPropertyAddress prop {
kAudioProcessPropertyPID,
kAudioObjectPropertyScopeGlobal,
kAudioObjectPropertyElementMain
};
UInt32 dataSize = sizeof(pid);
OSStatus error = AudioObjectGetPropertyData(process, &prop, 0, nullptr, &dataSize, &pid);
if (error != noErr) {
return -1;
}
} else {
// Pre sonoma code goes here
}
return pid;
}
which works.
However, kAudioProcessPropertyPID was added in macOS SDK 14.0.
Does anyone know how to achieve the same functionality on previous versions?
Mobile app - Ellie's Gift
https://apps.apple.com/gb/app/ellies-gift/id1617597875
Using AVFoundation to play audio tracks within the app.
Has always been working fine across apple and android, but iphone 14 and newer devices are unable to play audio.
Any idea's or suggestions?
I have tried everything. The songs load unto the playlists and on searches, but when prompted to play, they just won't play.
I have a wrapper since my main player (which carries the buttons for play/rewind/forward/etc.), is in Objc.
//
// ApplePlayerWrapper.swift
// UniversallyMac
//
// Created by Dorian Mattar on 11/10/24.
//
import Foundation
import MusicKit
import MediaPlayer
@objc public class MusicKitWrapper: NSObject {
@objc public static let shared = MusicKitWrapper()
private let player = ApplicationMusicPlayer.shared
// Play the current track
@objc public func play() {
guard !player.queue.entries.isEmpty else {
print("Queue is empty. Cannot start playback.")
return
}
logPlayerState(message: "Before play")
Task {
do {
try await player.prepareToPlay()
try await player.play()
print("Playback started successfully.")
} catch {
if let nsError = error as NSError? {
print("NSError Code: \(nsError.code), Domain: \(nsError.domain)")
}
}
logPlayerState(message: "After play")
}
}
// Log the current player state
@objc public func logPlayerState(message: String = "") {
print("Player State - \(message):")
print("Playback Status: \(player.state.playbackStatus)")
print("Queue Count: \(player.queue.entries.count)")
// Only log current track details if the player is playing
if player.state.playbackStatus == .playing {
if let currentEntry = player.queue.currentEntry {
print("Current Track: \(currentEntry.title)")
print("Current Position: \(player.playbackTime) seconds")
print("Track Length: \(currentEntry.endTime ?? 0.0) seconds")
} else {
print("No current track.")
}
} else {
print("No track is playing.")
}
print("----------")
}
// Debug the queue
@objc public func debugQueue() {
print("Debugging Queue:")
for (index, entry) in player.queue.entries.enumerated() {
print("\(index): \(entry.title)")
}
}
// Ensure track availability in the queue
public func queueTracks(_ tracks: [Track]) {
Task {
do {
for track in tracks {
// Validate Play Parameters
guard let playParameters = track.playParameters else {
print("Track \(track.title) has no Play Parameters.")
continue
}
// Log the Play Parameters
print("Track Title: \(track.title)")
print("Play Parameters: \(playParameters)")
print("Raw Values: \(track.id.rawValue)")
// Ensure the ID is valid
if track.id.rawValue.isEmpty {
print("Track \(track.title) has an invalid or empty ID in Play Parameters.")
continue
}
// Queue the track
try await player.queue.insert(track, position: .afterCurrentEntry)
print("Queued track: \(track.title)")
}
print("Tracks successfully added to the queue.")
} catch {
print("Error queuing tracks: \(error)")
}
debugQueue()
}
}
// Clear the current queue
@objc public func resetMusicPlayer() {
Task {
player.stop()
player.queue.entries.removeAll()
print("Queue cleared.")
print("Apple Music player reset successfully.")
}
}
}
I opened an Apple Dev. ticket, but I'm trying here as well. Thanks!
I've been trying to use AVMIDIControlChangeEvent with a bankSelect message type to change the instrument the sequencer uses on a AVMusicTrack with no luck.
I started with the Apple AVAEMixerSample, converting the initial setup/loading and portions dealing with the sequencer to Swift. I got that working and playing the "bluesyRiff" and then modified it to play individual notes. So my createAndSetupSequencer looked like
func createAndSetupSequencer() {
sequencer = AVAudioSequencer(audioEngine: engine)
// guard let midiFileURL = Bundle.main.url(forResource: "bluesyRiff", withExtension: "mid") else {
// print (" failed guard trying to get URL for bluesyRiff")
// return
// }
let track = sequencer.createAndAppendTrack()
var currTime = 1.0
for i: UInt32 in 0...8 {
let newNoteEvent = AVMIDINoteEvent(channel: 0, key: 60+i, velocity: 64, duration: 2.0)
track.addEvent(newNoteEvent, at: AVMusicTimeStamp(currTime))
currTime += 2.0
}
The notes played, so then I also replaced the gs_instruments sound bank with GeneralUser GS MuseScore v1.442 first by trying
guard let soundBankURL = Bundle.main.url(forResource: "GeneralUser GS MuseScore v1.442", withExtension: "sf2") else {
return}
do {
try sampler.loadSoundBankInstrument(at: soundBankURL, program: 0x001C, bankMSB: 0x79, bankLSB: 0x08)
} catch{....
}
This appears to work, the instrument (8 which is "Funk Guitar") plays. If I change to bankLSB: 0x00 I get the "Palm Muted guitar". So I know that the soundfont has these instruments
Stuff goes off the rails when I try to change the instruments in createAndSetupSequencer. Putting
let programChange = AVMIDIProgramChangeEvent(channel: 0, programNumber: 0x001C)
let bankChange = AVMIDIControlChangeEvent(channel: 0, messageType: AVMIDIControlChangeEvent.MessageType.bankSelect, value: 0x00)
track.addEvent(programChange, at: AVMusicTimeStamp(1.0))
track.addEvent(bankChange, at: AVMusicTimeStamp(1.0))
just before my add note loop doesn't produce any change. Loading bankLSB 8 (Funk) in sampler.loadSoundBankInstrument and trying to change with bankSelect 0 (Palm muted) in createAndSetupSequencer results in instrument 8 (Funk) playing not Palm Muted.
Loading bankLSB 0 (Palm muted) and trying to change with bankSelect 8 (Funk) doesn't work, 0 (Palm muted) plays
I also tried sampler.loadInstrument(at: soundBankURL) and then I always get the first instrument in the sound font file (piano)no matter what values I put in my programChange/bankChange
I've also changed the time in the track.addEvent to be 0, 1.0, 3.0 etc to no success
The sampler.loadSoundBankInstrument specifies two UInt8 parameters, bankMSB and BankLSB while the AVMIDIControlChangeEvent bankSelect value is UInt32 suggesting it might be some combination of bankMSB and BankLSB. But the documentation makes no mention of what this should look like. I tried various combinations of 0x7908, 0X0879 etc to no avail
I will also point out that I am able to successfully execute other control change events
For example adding
if i == 1 {
let portamentoOnEvent = AVMIDIControlChangeEvent(channel: 0, messageType: AVMIDIControlChangeEvent.MessageType.portamento, value: 0xFF)
track.addEvent(portamentoOnEvent, at: AVMusicTimeStamp(currTime))
let portamentoRateEvent = AVMIDIControlChangeEvent(channel: 0, messageType: AVMIDIControlChangeEvent.MessageType.portamentoTime, value: 64)
track.addEvent(portamentoRateEvent, at: AVMusicTimeStamp(currTime))
}
does produce a change in the sound. (As an aside, a definition of what portamento time is, other than "the rate of portamento" would be welcome. is it notes/seconds? freq/minute? beats/hour?)
I was able to get the instrument to change in a different program using MusicPlayer and a series of MusicTrackNewMIDIChannelEvent on a track but these operate on a MusicTrack not the AVMusicTrack which the sequencer uses.
Has anyone been successful in switching instruments through an AVMIDIControlChangeEvent or have any feedback on how to do this?