Post

Replies

Boosts

Views

Activity

Chapter Marks - Adding AVTimedMetadataGroup to Audio (M4A)
Could you provide guidance on how to add chapter marks to an M4A. I've been attempting bookmark. From what I've read, it requires the use of AVMetadataKey.quickTimeUserDataKeyChapter track.addTrackAssociation(to: ... type: .chapterList) or both. I've looked into AVTimedMetadataGroup but I havent found a way to get it added based on the documentation. I also havent found anyone who has used native Swift to add chapter marks. They've always given in and used ffmpeg or some other external solution. inputURL is for the file that is being read in outputURL is for the the final file chapters is an array of dictionaries, where time is the start of each chapter and its name in the list The target is macOS import AVFoundation class AudioChapterCreator { // Function to create an audio file with chapters and a chapter list func createAudioFileWithChapters(inputURL: URL, outputURL: URL, chapters: [(time: CMTime, title: String)]) { let options = [AVURLAssetPreferPreciseDurationAndTimingKey: true] let asset = AVURLAsset(url: inputURL, options: options) let durationInSeconds = CMTimeGetSeconds(asset.duration) print("asset durationInSeconds: \(durationInSeconds)") guard let audioTrack = asset.tracks(withMediaType: .audio).first else { print("Error: Unable to find audio track in the asset.") return } // Create metadata items for chapters let chapterMetadataItems = chapters.map { chapter -> AVMetadataItem in let item = AVMutableMetadataItem() // this duration is just for testing let tempDur = CMTime(seconds: 100, preferredTimescale: 1) item.keySpace = AVMetadataKeySpace.quickTimeUserData item.key = AVMetadataKey.quickTimeUserDataKeyChapter as NSString item.value = chapter.title as NSString item.time = chapter.time item.duration = tempDur return item } // Create an AVAssetExportSession for writing the output file guard let exportSession = AVAssetExportSession(asset: asset, presetName: AVAssetExportPresetAppleM4A) else { print("Error: Unable to create AVAssetExportSession.") return } // Configure the AVAssetExportSession exportSession.outputFileType = .m4a exportSession.outputURL = outputURL exportSession.metadata = asset.metadata + chapterMetadataItems exportSession.timeRange = CMTimeRangeMake(start: CMTime.zero, duration: asset.duration); // Export the audio file exportSession.exportAsynchronously { switch exportSession.status { case .completed: print("Audio file with chapters and chapter list created successfully.") case .failed: print("Error: Failed to create the audio file.") case .cancelled: print("Export cancelled.") default: print("Export failed with unknown status.") } } } }
0
0
775
Aug ’23
IPA notation AVSpeechSynthesizer is not working?
I am attempting to utilize alternative pronunciation utilizing the IPA notation for AVSpeechSynthesizer on macOS (Big Sur 11.4). The attributed string is being ignored and so the functionality is not working. I tried this on iOS simulator and it works properly. The India English voice pronounces the word "shame" as shy-em, so I applied the correct pronunciation but no change was heard. I then substituted the pronunciation for a completely different word but there was no change. Is there something else that must be done to make this work? AVSpeechSynthesisIPANotationAttribute Attributed String: It's a '{ }shame{ AVSpeechSynthesisIPANotationAttribute = "\U0283\U02c8e\U0361\U026am"; }' it didn't work out.{ } Target Range: {8, 5} Target String: shame, Substitution: ʃˈe͡ɪm Attributed String: It's a '{ }shame{ AVSpeechSynthesisIPANotationAttribute = "\U0283\U02c8e\U0361\U026am"; }' it didn't work out.{ } Target Range: {8, 5} Target String: shame, Substitution: ʃˈe͡ɪm Attributed String: It's a '{ }shame{ AVSpeechSynthesisIPANotationAttribute = "t\U0259.\U02c8me\U0361\U026a.do\U0361\U028a"; }' it didn't work out.{ } Target Range: {8, 5} Target String: shame, Substitution: tə.ˈme͡ɪ.do͡ʊ Attributed String: It's a '{ }shame{ AVSpeechSynthesisIPANotationAttribute = "t\U0259.\U02c8me\U0361\U026a.do\U0361\U028a"; }' it didn't work out.{ } Target Range: {8, 5} Target String: shame, Substitution: tə.ˈme͡ɪ.do͡ʊ class SpeakerTest: NSObject, AVSpeechSynthesizerDelegate { let synth = AVSpeechSynthesizer() func speakIPA_Substitution(subst: String, voice: AVSpeechSynthesisVoice) { let text = "It's a 'shame' it didn't work out." let mutAttrStr = NSMutableAttributedString(string: text) let range = NSString(string: text).range(of: "shame") let pronounceKey = NSAttributedString.Key(rawValue: AVSpeechSynthesisIPANotationAttribute) mutAttrStr.setAttributes([pronounceKey: subst], range: range) let utterance = AVSpeechUtterance(attributedString: mutAttrStr) utterance.voice = voice utterance.postUtteranceDelay = 1.0 let swiftRange = Range(range, in: text)! print("Attributed String: \(mutAttrStr)") print("Target Range: \(range)") print("Target String: \(text[swiftRange]), Substitution: \(subst)\n") synth.speak(utterance) } func customPronunciation() { let shame = "ʃˈe͡ɪm" // substitute correct pronunciation let tomato = "tə.ˈme͡ɪ.do͡ʊ" // completely different word pronunciation let britishVoice = AVSpeechSynthesisVoice(language: "en-GB")! let indiaVoice = AVSpeechSynthesisVoice(language: "en-IN")! speakIPA_Substitution(subst: shame, voice: britishVoice) // already correct, no substitute needed // pronounced incorrectly and ignoring the corrected pronunciation from IPA Notation speakIPA_Substitution(subst: shame, voice: indiaVoice) // ignores substitution speakIPA_Substitution(subst: tomato, voice: britishVoice) // ignores substitution speakIPA_Substitution(subst: tomato, voice: indiaVoice) // ignores substitution } }
3
0
2.1k
Apr ’22
How to make AVSpeechSynthesizer work for write and delegate (Big Sur)
I am unable to get AVSpeechSynthesizer to write or to acknowledge the delegate actions . I was informed this was resolved in macOS 11. I thought it was a lot to ask but am now running on macOS 11.4 (Big Sur). My target is to output speech faster than real-time and and drive the output through AVAudioengine. First, I need to know why the write doesnt occur and neither do delegates get called whether I am using write or simply uttering to the default speakers in "func speak(_ string: String)". What am I missing? Is there a workaround? Reference: https://developer.apple.com/forums/thread/678287 let sentenceToSpeak = "This should write to buffer and also call 'didFinish' and 'willSpeakRangeOfSpeechString' delegates." SpeakerTest().writeToBuffer(sentenceToSpeak) SpeakerTest().speak(sentenceToSpeak) class SpeakerTest: NSObject, AVSpeechSynthesizerDelegate { let synth = AVSpeechSynthesizer() override init() { super.init() synth.delegate = self } func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, didFinish utterance: AVSpeechUtterance) { print("Utterance didFinish") } func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, willSpeakRangeOfSpeechString characterRange: NSRange, utterance: AVSpeechUtterance) { print("speaking range: \(characterRange)") } func speak(_ string: String) { let utterance = AVSpeechUtterance(string: string) var usedVoice = AVSpeechSynthesisVoice(language: "en") // should be the default voice let voices = AVSpeechSynthesisVoice.speechVoices() let targetVoice = "Allison" for voice in voices { // print("\(voice.identifier) \(voice.name) \(voice.quality) \(voice.language)") if (voice.name.lowercased() == targetVoice.lowercased()) { usedVoice = AVSpeechSynthesisVoice(identifier: voice.identifier) break } } utterance.voice = usedVoice print("utterance.voice: \(utterance.voice)") synth.speak(utterance) } func writeToBuffer(_ string: String) { print("entering writeToBuffer") let utterance = AVSpeechUtterance(string: string) synth.write(utterance) { (buffer: AVAudioBuffer) in print("executing synth.write") guard let pcmBuffer = buffer as? AVAudioPCMBuffer else { fatalError("unknown buffer type: \(buffer)") } if pcmBuffer.frameLength == 0 { print("buffer is empty") } else { print("buffer has content \(buffer)") } } } }
3
0
3.1k
Jan ’22
Can you perform two or more OFFLINE speech recognition tasks simultaneously?
Can you perform two or more OFFLINE speech recognition tasks simultaneously? SFSpeechRecognizer, SFSpeechURLRecognitionRequest offline limitation? Running on macOS Big Sur 11.5.2 I would like to be perform two or more offline speech recognition tasks simultaneously. I've executed two tasks in the same application AND executed two different applications, both using offline recognition. Once I initiate the other thread or other application, the first recognition stops. Since the computer supports multiple threads, I planned to take make use of the concurrency. Use cases #1 multiple Audio or video files that I wish to transcribe -- cuts down on the wait time. #2 split a single large file up into multiple sections and stitch the results together -- again cuts down on the wait time. I set on device recognition to TRUE because my target files can be up to two hours in length. My test files are 15-30 minutes in length and I have a number of them, so recognition must be done on the device. func recognizeFile_Compact(url:NSURL) { let language = "en-US" //"en-GB" let recognizer = SFSpeechRecognizer(locale: Locale.init(identifier: language))! let recogRequest = SFSpeechURLRecognitionRequest(url: url as URL) recognizer.supportsOnDeviceRecognition = true // ensure the DEVICE does the work -- don't send to cloud recognizer.defaultTaskHint = .dictation // give a hint as dictation recogRequest.requiresOnDeviceRecognition = true // don recogRequest.shouldReportPartialResults = false // we dont want partial results var strCount = 0 let recogTask = recognizer.recognitionTask(with: recogRequest, resultHandler: { (result, error) in guard let result = result else { print("Recognition failed, \(error!)") return } let text = result.bestTranscription.formattedString strCount += 1 print(" #\(strCount), "Best: \(text) \n" ) if (result.isFinal) { print("WE ARE FINALIZED") } }) }
1
0
2.1k
Nov ’21
Chapter Marks - Adding AVTimedMetadataGroup to Audio (M4A)
Could you provide guidance on how to add chapter marks to an M4A. I've been attempting bookmark. From what I've read, it requires the use of AVMetadataKey.quickTimeUserDataKeyChapter track.addTrackAssociation(to: ... type: .chapterList) or both. I've looked into AVTimedMetadataGroup but I havent found a way to get it added based on the documentation. I also havent found anyone who has used native Swift to add chapter marks. They've always given in and used ffmpeg or some other external solution. inputURL is for the file that is being read in outputURL is for the the final file chapters is an array of dictionaries, where time is the start of each chapter and its name in the list The target is macOS import AVFoundation class AudioChapterCreator { // Function to create an audio file with chapters and a chapter list func createAudioFileWithChapters(inputURL: URL, outputURL: URL, chapters: [(time: CMTime, title: String)]) { let options = [AVURLAssetPreferPreciseDurationAndTimingKey: true] let asset = AVURLAsset(url: inputURL, options: options) let durationInSeconds = CMTimeGetSeconds(asset.duration) print("asset durationInSeconds: \(durationInSeconds)") guard let audioTrack = asset.tracks(withMediaType: .audio).first else { print("Error: Unable to find audio track in the asset.") return } // Create metadata items for chapters let chapterMetadataItems = chapters.map { chapter -> AVMetadataItem in let item = AVMutableMetadataItem() // this duration is just for testing let tempDur = CMTime(seconds: 100, preferredTimescale: 1) item.keySpace = AVMetadataKeySpace.quickTimeUserData item.key = AVMetadataKey.quickTimeUserDataKeyChapter as NSString item.value = chapter.title as NSString item.time = chapter.time item.duration = tempDur return item } // Create an AVAssetExportSession for writing the output file guard let exportSession = AVAssetExportSession(asset: asset, presetName: AVAssetExportPresetAppleM4A) else { print("Error: Unable to create AVAssetExportSession.") return } // Configure the AVAssetExportSession exportSession.outputFileType = .m4a exportSession.outputURL = outputURL exportSession.metadata = asset.metadata + chapterMetadataItems exportSession.timeRange = CMTimeRangeMake(start: CMTime.zero, duration: asset.duration); // Export the audio file exportSession.exportAsynchronously { switch exportSession.status { case .completed: print("Audio file with chapters and chapter list created successfully.") case .failed: print("Error: Failed to create the audio file.") case .cancelled: print("Export cancelled.") default: print("Export failed with unknown status.") } } } }
Replies
0
Boosts
0
Views
775
Activity
Aug ’23
IPA notation AVSpeechSynthesizer is not working?
I am attempting to utilize alternative pronunciation utilizing the IPA notation for AVSpeechSynthesizer on macOS (Big Sur 11.4). The attributed string is being ignored and so the functionality is not working. I tried this on iOS simulator and it works properly. The India English voice pronounces the word "shame" as shy-em, so I applied the correct pronunciation but no change was heard. I then substituted the pronunciation for a completely different word but there was no change. Is there something else that must be done to make this work? AVSpeechSynthesisIPANotationAttribute Attributed String: It's a '{ }shame{ AVSpeechSynthesisIPANotationAttribute = "\U0283\U02c8e\U0361\U026am"; }' it didn't work out.{ } Target Range: {8, 5} Target String: shame, Substitution: ʃˈe͡ɪm Attributed String: It's a '{ }shame{ AVSpeechSynthesisIPANotationAttribute = "\U0283\U02c8e\U0361\U026am"; }' it didn't work out.{ } Target Range: {8, 5} Target String: shame, Substitution: ʃˈe͡ɪm Attributed String: It's a '{ }shame{ AVSpeechSynthesisIPANotationAttribute = "t\U0259.\U02c8me\U0361\U026a.do\U0361\U028a"; }' it didn't work out.{ } Target Range: {8, 5} Target String: shame, Substitution: tə.ˈme͡ɪ.do͡ʊ Attributed String: It's a '{ }shame{ AVSpeechSynthesisIPANotationAttribute = "t\U0259.\U02c8me\U0361\U026a.do\U0361\U028a"; }' it didn't work out.{ } Target Range: {8, 5} Target String: shame, Substitution: tə.ˈme͡ɪ.do͡ʊ class SpeakerTest: NSObject, AVSpeechSynthesizerDelegate { let synth = AVSpeechSynthesizer() func speakIPA_Substitution(subst: String, voice: AVSpeechSynthesisVoice) { let text = "It's a 'shame' it didn't work out." let mutAttrStr = NSMutableAttributedString(string: text) let range = NSString(string: text).range(of: "shame") let pronounceKey = NSAttributedString.Key(rawValue: AVSpeechSynthesisIPANotationAttribute) mutAttrStr.setAttributes([pronounceKey: subst], range: range) let utterance = AVSpeechUtterance(attributedString: mutAttrStr) utterance.voice = voice utterance.postUtteranceDelay = 1.0 let swiftRange = Range(range, in: text)! print("Attributed String: \(mutAttrStr)") print("Target Range: \(range)") print("Target String: \(text[swiftRange]), Substitution: \(subst)\n") synth.speak(utterance) } func customPronunciation() { let shame = "ʃˈe͡ɪm" // substitute correct pronunciation let tomato = "tə.ˈme͡ɪ.do͡ʊ" // completely different word pronunciation let britishVoice = AVSpeechSynthesisVoice(language: "en-GB")! let indiaVoice = AVSpeechSynthesisVoice(language: "en-IN")! speakIPA_Substitution(subst: shame, voice: britishVoice) // already correct, no substitute needed // pronounced incorrectly and ignoring the corrected pronunciation from IPA Notation speakIPA_Substitution(subst: shame, voice: indiaVoice) // ignores substitution speakIPA_Substitution(subst: tomato, voice: britishVoice) // ignores substitution speakIPA_Substitution(subst: tomato, voice: indiaVoice) // ignores substitution } }
Replies
3
Boosts
0
Views
2.1k
Activity
Apr ’22
How to make AVSpeechSynthesizer work for write and delegate (Big Sur)
I am unable to get AVSpeechSynthesizer to write or to acknowledge the delegate actions . I was informed this was resolved in macOS 11. I thought it was a lot to ask but am now running on macOS 11.4 (Big Sur). My target is to output speech faster than real-time and and drive the output through AVAudioengine. First, I need to know why the write doesnt occur and neither do delegates get called whether I am using write or simply uttering to the default speakers in "func speak(_ string: String)". What am I missing? Is there a workaround? Reference: https://developer.apple.com/forums/thread/678287 let sentenceToSpeak = "This should write to buffer and also call 'didFinish' and 'willSpeakRangeOfSpeechString' delegates." SpeakerTest().writeToBuffer(sentenceToSpeak) SpeakerTest().speak(sentenceToSpeak) class SpeakerTest: NSObject, AVSpeechSynthesizerDelegate { let synth = AVSpeechSynthesizer() override init() { super.init() synth.delegate = self } func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, didFinish utterance: AVSpeechUtterance) { print("Utterance didFinish") } func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, willSpeakRangeOfSpeechString characterRange: NSRange, utterance: AVSpeechUtterance) { print("speaking range: \(characterRange)") } func speak(_ string: String) { let utterance = AVSpeechUtterance(string: string) var usedVoice = AVSpeechSynthesisVoice(language: "en") // should be the default voice let voices = AVSpeechSynthesisVoice.speechVoices() let targetVoice = "Allison" for voice in voices { // print("\(voice.identifier) \(voice.name) \(voice.quality) \(voice.language)") if (voice.name.lowercased() == targetVoice.lowercased()) { usedVoice = AVSpeechSynthesisVoice(identifier: voice.identifier) break } } utterance.voice = usedVoice print("utterance.voice: \(utterance.voice)") synth.speak(utterance) } func writeToBuffer(_ string: String) { print("entering writeToBuffer") let utterance = AVSpeechUtterance(string: string) synth.write(utterance) { (buffer: AVAudioBuffer) in print("executing synth.write") guard let pcmBuffer = buffer as? AVAudioPCMBuffer else { fatalError("unknown buffer type: \(buffer)") } if pcmBuffer.frameLength == 0 { print("buffer is empty") } else { print("buffer has content \(buffer)") } } } }
Replies
3
Boosts
0
Views
3.1k
Activity
Jan ’22
Can you perform two or more OFFLINE speech recognition tasks simultaneously?
Can you perform two or more OFFLINE speech recognition tasks simultaneously? SFSpeechRecognizer, SFSpeechURLRecognitionRequest offline limitation? Running on macOS Big Sur 11.5.2 I would like to be perform two or more offline speech recognition tasks simultaneously. I've executed two tasks in the same application AND executed two different applications, both using offline recognition. Once I initiate the other thread or other application, the first recognition stops. Since the computer supports multiple threads, I planned to take make use of the concurrency. Use cases #1 multiple Audio or video files that I wish to transcribe -- cuts down on the wait time. #2 split a single large file up into multiple sections and stitch the results together -- again cuts down on the wait time. I set on device recognition to TRUE because my target files can be up to two hours in length. My test files are 15-30 minutes in length and I have a number of them, so recognition must be done on the device. func recognizeFile_Compact(url:NSURL) { let language = "en-US" //"en-GB" let recognizer = SFSpeechRecognizer(locale: Locale.init(identifier: language))! let recogRequest = SFSpeechURLRecognitionRequest(url: url as URL) recognizer.supportsOnDeviceRecognition = true // ensure the DEVICE does the work -- don't send to cloud recognizer.defaultTaskHint = .dictation // give a hint as dictation recogRequest.requiresOnDeviceRecognition = true // don recogRequest.shouldReportPartialResults = false // we dont want partial results var strCount = 0 let recogTask = recognizer.recognitionTask(with: recogRequest, resultHandler: { (result, error) in guard let result = result else { print("Recognition failed, \(error!)") return } let text = result.bestTranscription.formattedString strCount += 1 print(" #\(strCount), "Best: \(text) \n" ) if (result.isFinal) { print("WE ARE FINALIZED") } }) }
Replies
1
Boosts
0
Views
2.1k
Activity
Nov ’21