Post

Replies

Boosts

Views

Activity

Chapter Marks - Adding AVTimedMetadataGroup to Audio (M4A)
Could you provide guidance on how to add chapter marks to an M4A. I've been attempting bookmark. From what I've read, it requires the use of AVMetadataKey.quickTimeUserDataKeyChapter track.addTrackAssociation(to: ... type: .chapterList) or both. I've looked into AVTimedMetadataGroup but I havent found a way to get it added based on the documentation. I also havent found anyone who has used native Swift to add chapter marks. They've always given in and used ffmpeg or some other external solution. inputURL is for the file that is being read in outputURL is for the the final file chapters is an array of dictionaries, where time is the start of each chapter and its name in the list The target is macOS import AVFoundation class AudioChapterCreator { // Function to create an audio file with chapters and a chapter list func createAudioFileWithChapters(inputURL: URL, outputURL: URL, chapters: [(time: CMTime, title: String)]) { let options = [AVURLAssetPreferPreciseDurationAndTimingKey: true] let asset = AVURLAsset(url: inputURL, options: options) let durationInSeconds = CMTimeGetSeconds(asset.duration) print("asset durationInSeconds: \(durationInSeconds)") guard let audioTrack = asset.tracks(withMediaType: .audio).first else { print("Error: Unable to find audio track in the asset.") return } // Create metadata items for chapters let chapterMetadataItems = chapters.map { chapter -> AVMetadataItem in let item = AVMutableMetadataItem() // this duration is just for testing let tempDur = CMTime(seconds: 100, preferredTimescale: 1) item.keySpace = AVMetadataKeySpace.quickTimeUserData item.key = AVMetadataKey.quickTimeUserDataKeyChapter as NSString item.value = chapter.title as NSString item.time = chapter.time item.duration = tempDur return item } // Create an AVAssetExportSession for writing the output file guard let exportSession = AVAssetExportSession(asset: asset, presetName: AVAssetExportPresetAppleM4A) else { print("Error: Unable to create AVAssetExportSession.") return } // Configure the AVAssetExportSession exportSession.outputFileType = .m4a exportSession.outputURL = outputURL exportSession.metadata = asset.metadata + chapterMetadataItems exportSession.timeRange = CMTimeRangeMake(start: CMTime.zero, duration: asset.duration); // Export the audio file exportSession.exportAsynchronously { switch exportSession.status { case .completed: print("Audio file with chapters and chapter list created successfully.") case .failed: print("Error: Failed to create the audio file.") case .cancelled: print("Export cancelled.") default: print("Export failed with unknown status.") } } } }
0
0
763
Aug ’23
IPA notation AVSpeechSynthesizer is not working?
I am attempting to utilize alternative pronunciation utilizing the IPA notation for AVSpeechSynthesizer on macOS (Big Sur 11.4). The attributed string is being ignored and so the functionality is not working. I tried this on iOS simulator and it works properly. The India English voice pronounces the word "shame" as shy-em, so I applied the correct pronunciation but no change was heard. I then substituted the pronunciation for a completely different word but there was no change. Is there something else that must be done to make this work? AVSpeechSynthesisIPANotationAttribute Attributed String: It's a '{ }shame{ AVSpeechSynthesisIPANotationAttribute = "\U0283\U02c8e\U0361\U026am"; }' it didn't work out.{ } Target Range: {8, 5} Target String: shame, Substitution: ʃˈe͡ɪm Attributed String: It's a '{ }shame{ AVSpeechSynthesisIPANotationAttribute = "\U0283\U02c8e\U0361\U026am"; }' it didn't work out.{ } Target Range: {8, 5} Target String: shame, Substitution: ʃˈe͡ɪm Attributed String: It's a '{ }shame{ AVSpeechSynthesisIPANotationAttribute = "t\U0259.\U02c8me\U0361\U026a.do\U0361\U028a"; }' it didn't work out.{ } Target Range: {8, 5} Target String: shame, Substitution: tə.ˈme͡ɪ.do͡ʊ Attributed String: It's a '{ }shame{ AVSpeechSynthesisIPANotationAttribute = "t\U0259.\U02c8me\U0361\U026a.do\U0361\U028a"; }' it didn't work out.{ } Target Range: {8, 5} Target String: shame, Substitution: tə.ˈme͡ɪ.do͡ʊ class SpeakerTest: NSObject, AVSpeechSynthesizerDelegate { let synth = AVSpeechSynthesizer() func speakIPA_Substitution(subst: String, voice: AVSpeechSynthesisVoice) { let text = "It's a 'shame' it didn't work out." let mutAttrStr = NSMutableAttributedString(string: text) let range = NSString(string: text).range(of: "shame") let pronounceKey = NSAttributedString.Key(rawValue: AVSpeechSynthesisIPANotationAttribute) mutAttrStr.setAttributes([pronounceKey: subst], range: range) let utterance = AVSpeechUtterance(attributedString: mutAttrStr) utterance.voice = voice utterance.postUtteranceDelay = 1.0 let swiftRange = Range(range, in: text)! print("Attributed String: \(mutAttrStr)") print("Target Range: \(range)") print("Target String: \(text[swiftRange]), Substitution: \(subst)\n") synth.speak(utterance) } func customPronunciation() { let shame = "ʃˈe͡ɪm" // substitute correct pronunciation let tomato = "tə.ˈme͡ɪ.do͡ʊ" // completely different word pronunciation let britishVoice = AVSpeechSynthesisVoice(language: "en-GB")! let indiaVoice = AVSpeechSynthesisVoice(language: "en-IN")! speakIPA_Substitution(subst: shame, voice: britishVoice) // already correct, no substitute needed // pronounced incorrectly and ignoring the corrected pronunciation from IPA Notation speakIPA_Substitution(subst: shame, voice: indiaVoice) // ignores substitution speakIPA_Substitution(subst: tomato, voice: britishVoice) // ignores substitution speakIPA_Substitution(subst: tomato, voice: indiaVoice) // ignores substitution } }
3
0
2.1k
Apr ’22
How to make AVSpeechSynthesizer work for write and delegate (Big Sur)
I am unable to get AVSpeechSynthesizer to write or to acknowledge the delegate actions . I was informed this was resolved in macOS 11. I thought it was a lot to ask but am now running on macOS 11.4 (Big Sur). My target is to output speech faster than real-time and and drive the output through AVAudioengine. First, I need to know why the write doesnt occur and neither do delegates get called whether I am using write or simply uttering to the default speakers in "func speak(_ string: String)". What am I missing? Is there a workaround? Reference: https://developer.apple.com/forums/thread/678287 let sentenceToSpeak = "This should write to buffer and also call 'didFinish' and 'willSpeakRangeOfSpeechString' delegates." SpeakerTest().writeToBuffer(sentenceToSpeak) SpeakerTest().speak(sentenceToSpeak) class SpeakerTest: NSObject, AVSpeechSynthesizerDelegate { let synth = AVSpeechSynthesizer() override init() { super.init() synth.delegate = self } func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, didFinish utterance: AVSpeechUtterance) { print("Utterance didFinish") } func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, willSpeakRangeOfSpeechString characterRange: NSRange, utterance: AVSpeechUtterance) { print("speaking range: \(characterRange)") } func speak(_ string: String) { let utterance = AVSpeechUtterance(string: string) var usedVoice = AVSpeechSynthesisVoice(language: "en") // should be the default voice let voices = AVSpeechSynthesisVoice.speechVoices() let targetVoice = "Allison" for voice in voices { // print("\(voice.identifier) \(voice.name) \(voice.quality) \(voice.language)") if (voice.name.lowercased() == targetVoice.lowercased()) { usedVoice = AVSpeechSynthesisVoice(identifier: voice.identifier) break } } utterance.voice = usedVoice print("utterance.voice: \(utterance.voice)") synth.speak(utterance) } func writeToBuffer(_ string: String) { print("entering writeToBuffer") let utterance = AVSpeechUtterance(string: string) synth.write(utterance) { (buffer: AVAudioBuffer) in print("executing synth.write") guard let pcmBuffer = buffer as? AVAudioPCMBuffer else { fatalError("unknown buffer type: \(buffer)") } if pcmBuffer.frameLength == 0 { print("buffer is empty") } else { print("buffer has content \(buffer)") } } } }
3
0
3k
Jan ’22
Can you perform two or more OFFLINE speech recognition tasks simultaneously?
Can you perform two or more OFFLINE speech recognition tasks simultaneously? SFSpeechRecognizer, SFSpeechURLRecognitionRequest offline limitation? Running on macOS Big Sur 11.5.2 I would like to be perform two or more offline speech recognition tasks simultaneously. I've executed two tasks in the same application AND executed two different applications, both using offline recognition. Once I initiate the other thread or other application, the first recognition stops. Since the computer supports multiple threads, I planned to take make use of the concurrency. Use cases #1 multiple Audio or video files that I wish to transcribe -- cuts down on the wait time. #2 split a single large file up into multiple sections and stitch the results together -- again cuts down on the wait time. I set on device recognition to TRUE because my target files can be up to two hours in length. My test files are 15-30 minutes in length and I have a number of them, so recognition must be done on the device. func recognizeFile_Compact(url:NSURL) { let language = "en-US" //"en-GB" let recognizer = SFSpeechRecognizer(locale: Locale.init(identifier: language))! let recogRequest = SFSpeechURLRecognitionRequest(url: url as URL) recognizer.supportsOnDeviceRecognition = true // ensure the DEVICE does the work -- don't send to cloud recognizer.defaultTaskHint = .dictation // give a hint as dictation recogRequest.requiresOnDeviceRecognition = true // don recogRequest.shouldReportPartialResults = false // we dont want partial results var strCount = 0 let recogTask = recognizer.recognitionTask(with: recogRequest, resultHandler: { (result, error) in guard let result = result else { print("Recognition failed, \(error!)") return } let text = result.bestTranscription.formattedString strCount += 1 print(" #\(strCount), "Best: \(text) \n" ) if (result.isFinal) { print("WE ARE FINALIZED") } }) }
1
0
1.9k
Nov ’21
AVSpeechSynthesizer buffer conversion, write format bug?
Is the format description AVSpeechSynthesizer for the speech buffer is correct? When I attempt to convert it, I get back noise from two different conversion methods. I am seeking to convert the speech buffer provided by the AVSpeechSynthesizer "func write(_ utterance: AVSpeechUtterance..." method. The goal is to convert the sample type, change the sample rate and change from mono to stereo buffer. I later manipulate the buffer data and pass it through AVAudioengine. For testing purposes, I have kept the sample rate to the original 22050.0 What have I tried? I have a method that I've been using for years named "resampleBuffer" that does this. When I apply it to the speech buffer, I get back noise. When I attempt to manually convert format and to stereo with "convertSpeechBufferToFloatStereo", I am getting back clipped output. I tested flipping the samples, addressing the Big Endian, Signed Integer but that didn't work. The speech buffer description is inBuffer description: <AVAudioFormat 0x6000012862b0: 1 ch, 22050 Hz, 'lpcm' (0x0000000E) 32-bit big-endian signed integer> import Cocoa import AVFoundation class SpeakerTest: NSObject, AVSpeechSynthesizerDelegate { let synth = AVSpeechSynthesizer() override init() { super.init() } func resampleBuffer( inSource: AVAudioPCMBuffer, newSampleRate: Double) -> AVAudioPCMBuffer? { // resample and convert mono to stereo var error : NSError? let kChannelStereo = AVAudioChannelCount(2) let convertRate = newSampleRate / inSource.format.sampleRate let outFrameCount = AVAudioFrameCount(Double(inSource.frameLength) * convertRate) let outFormat = AVAudioFormat(standardFormatWithSampleRate: newSampleRate, channels: kChannelStereo)! let avConverter = AVAudioConverter(from: inSource.format, to: outFormat ) let outBuffer = AVAudioPCMBuffer(pcmFormat: outFormat, frameCapacity: outFrameCount)! let inputBlock : AVAudioConverterInputBlock = { (inNumPackets, outStatus) -> AVAudioBuffer? in outStatus.pointee = AVAudioConverterInputStatus.haveData // very important, must have let audioBuffer : AVAudioBuffer = inSource return audioBuffer } avConverter?.sampleRateConverterAlgorithm = AVSampleRateConverterAlgorithm_Mastering avConverter?.sampleRateConverterQuality = .max if let converter = avConverter { let status = converter.convert(to: outBuffer, error: &error, withInputFrom: inputBlock) // print("\(status): \(status.rawValue)") if ((status != .haveData) || (error != nil)) { print("\(status): \(status.rawValue), error: \(String(describing: error))") return nil // conversion error } } else { return nil // converter not created } // print("success!") return outBuffer } func writeToFile(_ stringToSpeak: String, speaker: String) { var output : AVAudioFile? let utterance = AVSpeechUtterance(string: stringToSpeak) let desktop = "~/Desktop" let fileName = "Utterance_Test.caf" // not in sandbox var tempPath = desktop + "/" + fileName tempPath = (tempPath as NSString).expandingTildeInPath let usingSampleRate = 22050.0 // 44100.0 let outSettings = [ AVFormatIDKey : kAudioFormatLinearPCM, // kAudioFormatAppleLossless AVSampleRateKey : usingSampleRate, AVNumberOfChannelsKey : 2, AVEncoderAudioQualityKey : AVAudioQuality.max.rawValue ] as [String : Any] // temporarily ignore the speaker and use the default voice let curLangCode = AVSpeechSynthesisVoice.currentLanguageCode() utterance.voice = AVSpeechSynthesisVoice(language: curLangCode) // utterance.volume = 1.0 print("Int32.max: \(Int32.max), Int32.min: \(Int32.min)") synth.write(utterance) { (buffer: AVAudioBuffer) in guard let pcmBuffer = buffer as? AVAudioPCMBuffer else { fatalError("unknown buffer type: \(buffer)") } if ( pcmBuffer.frameLength == 0 ) { // done } else { // append buffer to file var outBuffer : AVAudioPCMBuffer outBuffer = self.resampleBuffer( inSource: pcmBuffer, newSampleRate: usingSampleRate)! // doesnt work // outBuffer = self.convertSpeechBufferToFloatStereo( pcmBuffer ) // doesnt work // outBuffer = pcmBuffer // original format does work if ( output == nil ) { //var bufferSettings = utterance.voice?.audioFileSettings // Audio files cannot be non-interleaved. var outSettings = outBuffer.format.settings outSettings["AVLinearPCMIsNonInterleaved"] = false let inFormat = pcmBuffer.format print("inBuffer description: \(inFormat.description)") print("inBuffer settings: \(inFormat.settings)") print("inBuffer format: \(inFormat.formatDescription)") print("outBuffer settings: \(outSettings)\n") print("outBuffer format: \(outBuffer.format.formatDescription)") output = try! AVAudioFile( forWriting: URL(fileURLWithPath: tempPath),settings: outSettings) } try! output?.write(from: outBuffer) print("done") } } } } class ViewController: NSViewController { let speechDelivery = SpeakerTest() override func viewDidLoad() { super.viewDidLoad() let targetSpeaker = "Allison" var sentenceToSpeak = "" for indx in 1...10 { sentenceToSpeak += "This is sentence number \(indx). [[slnc 3000]] \n" } speechDelivery.writeToFile(sentenceToSpeak, speaker: targetSpeaker) } } Three test can be performed. The only one that works is to directly write the buffer to disk Is this really "32-bit big-endian signed integer"? Am I addressing this correctly or is this a bug? I'm on macOS 11.4
5
0
2.9k
Oct ’21
How to catch "unregister notification for read_timeout failed" from going to XCode console
How do you register to catch these notifications from going to XCode console? The messages occur whenever I execute a URLSession.shared.dataTask, which is often. The messages are not an indication the code has faulted but are notifications for unknown reasons that fill the console. You should only get a notification if something is wrong. How do you register to catch this message so it does not go to the XCode console? nw_endpoint_handler_set_adaptive_read_handler [C14.1 104.21.42.21:443 ready socket-flow (satisfied (Path is satisfied), viable, interface: en0, ipv4, dns)] unregister notification for read_timeout failed nw_endpoint_handler_set_adaptive_write_handler [C14.1 104.21.42.21:443 ready socket-flow (satisfied (Path is satisfied), viable, interface: en0, ipv4, dns)] unregister notification for write_timeout failed I believe this will duplicate the issue func queryEndpoint(address: String) { let url = URL(string: address) let task = URLSession.shared.dataTask(with: url!) {(data, response, error) in let result = String(data: data!, encoding: String.Encoding.utf8)! print(result) } task.resume() }
1
0
1.8k
Sep ’21
XCode 13 Quick Help partially working
I upgraded to XCode 13. Previously versions of XCode would show the variable name and type in Quickhelp, when you clicked on the variable "let classRef: ViewController" (example) or a user defined methods, it would show its declaration information. Now, quick help only shows information when you click on a built-in function or a function parameter I thought quitting XCode or cleaning the project would resolve this but it did not. This features was extremely beneficial, where you could easily click on an item and copy its declaration information or just see the types being reference. How do I get the previous behavior back?
1
0
1.5k
Sep ’21
Chapter Marks - Adding AVTimedMetadataGroup to Audio (M4A)
Could you provide guidance on how to add chapter marks to an M4A. I've been attempting bookmark. From what I've read, it requires the use of AVMetadataKey.quickTimeUserDataKeyChapter track.addTrackAssociation(to: ... type: .chapterList) or both. I've looked into AVTimedMetadataGroup but I havent found a way to get it added based on the documentation. I also havent found anyone who has used native Swift to add chapter marks. They've always given in and used ffmpeg or some other external solution. inputURL is for the file that is being read in outputURL is for the the final file chapters is an array of dictionaries, where time is the start of each chapter and its name in the list The target is macOS import AVFoundation class AudioChapterCreator { // Function to create an audio file with chapters and a chapter list func createAudioFileWithChapters(inputURL: URL, outputURL: URL, chapters: [(time: CMTime, title: String)]) { let options = [AVURLAssetPreferPreciseDurationAndTimingKey: true] let asset = AVURLAsset(url: inputURL, options: options) let durationInSeconds = CMTimeGetSeconds(asset.duration) print("asset durationInSeconds: \(durationInSeconds)") guard let audioTrack = asset.tracks(withMediaType: .audio).first else { print("Error: Unable to find audio track in the asset.") return } // Create metadata items for chapters let chapterMetadataItems = chapters.map { chapter -> AVMetadataItem in let item = AVMutableMetadataItem() // this duration is just for testing let tempDur = CMTime(seconds: 100, preferredTimescale: 1) item.keySpace = AVMetadataKeySpace.quickTimeUserData item.key = AVMetadataKey.quickTimeUserDataKeyChapter as NSString item.value = chapter.title as NSString item.time = chapter.time item.duration = tempDur return item } // Create an AVAssetExportSession for writing the output file guard let exportSession = AVAssetExportSession(asset: asset, presetName: AVAssetExportPresetAppleM4A) else { print("Error: Unable to create AVAssetExportSession.") return } // Configure the AVAssetExportSession exportSession.outputFileType = .m4a exportSession.outputURL = outputURL exportSession.metadata = asset.metadata + chapterMetadataItems exportSession.timeRange = CMTimeRangeMake(start: CMTime.zero, duration: asset.duration); // Export the audio file exportSession.exportAsynchronously { switch exportSession.status { case .completed: print("Audio file with chapters and chapter list created successfully.") case .failed: print("Error: Failed to create the audio file.") case .cancelled: print("Export cancelled.") default: print("Export failed with unknown status.") } } } }
Replies
0
Boosts
0
Views
763
Activity
Aug ’23
IPA notation AVSpeechSynthesizer is not working?
I am attempting to utilize alternative pronunciation utilizing the IPA notation for AVSpeechSynthesizer on macOS (Big Sur 11.4). The attributed string is being ignored and so the functionality is not working. I tried this on iOS simulator and it works properly. The India English voice pronounces the word "shame" as shy-em, so I applied the correct pronunciation but no change was heard. I then substituted the pronunciation for a completely different word but there was no change. Is there something else that must be done to make this work? AVSpeechSynthesisIPANotationAttribute Attributed String: It's a '{ }shame{ AVSpeechSynthesisIPANotationAttribute = "\U0283\U02c8e\U0361\U026am"; }' it didn't work out.{ } Target Range: {8, 5} Target String: shame, Substitution: ʃˈe͡ɪm Attributed String: It's a '{ }shame{ AVSpeechSynthesisIPANotationAttribute = "\U0283\U02c8e\U0361\U026am"; }' it didn't work out.{ } Target Range: {8, 5} Target String: shame, Substitution: ʃˈe͡ɪm Attributed String: It's a '{ }shame{ AVSpeechSynthesisIPANotationAttribute = "t\U0259.\U02c8me\U0361\U026a.do\U0361\U028a"; }' it didn't work out.{ } Target Range: {8, 5} Target String: shame, Substitution: tə.ˈme͡ɪ.do͡ʊ Attributed String: It's a '{ }shame{ AVSpeechSynthesisIPANotationAttribute = "t\U0259.\U02c8me\U0361\U026a.do\U0361\U028a"; }' it didn't work out.{ } Target Range: {8, 5} Target String: shame, Substitution: tə.ˈme͡ɪ.do͡ʊ class SpeakerTest: NSObject, AVSpeechSynthesizerDelegate { let synth = AVSpeechSynthesizer() func speakIPA_Substitution(subst: String, voice: AVSpeechSynthesisVoice) { let text = "It's a 'shame' it didn't work out." let mutAttrStr = NSMutableAttributedString(string: text) let range = NSString(string: text).range(of: "shame") let pronounceKey = NSAttributedString.Key(rawValue: AVSpeechSynthesisIPANotationAttribute) mutAttrStr.setAttributes([pronounceKey: subst], range: range) let utterance = AVSpeechUtterance(attributedString: mutAttrStr) utterance.voice = voice utterance.postUtteranceDelay = 1.0 let swiftRange = Range(range, in: text)! print("Attributed String: \(mutAttrStr)") print("Target Range: \(range)") print("Target String: \(text[swiftRange]), Substitution: \(subst)\n") synth.speak(utterance) } func customPronunciation() { let shame = "ʃˈe͡ɪm" // substitute correct pronunciation let tomato = "tə.ˈme͡ɪ.do͡ʊ" // completely different word pronunciation let britishVoice = AVSpeechSynthesisVoice(language: "en-GB")! let indiaVoice = AVSpeechSynthesisVoice(language: "en-IN")! speakIPA_Substitution(subst: shame, voice: britishVoice) // already correct, no substitute needed // pronounced incorrectly and ignoring the corrected pronunciation from IPA Notation speakIPA_Substitution(subst: shame, voice: indiaVoice) // ignores substitution speakIPA_Substitution(subst: tomato, voice: britishVoice) // ignores substitution speakIPA_Substitution(subst: tomato, voice: indiaVoice) // ignores substitution } }
Replies
3
Boosts
0
Views
2.1k
Activity
Apr ’22
How to make AVSpeechSynthesizer work for write and delegate (Big Sur)
I am unable to get AVSpeechSynthesizer to write or to acknowledge the delegate actions . I was informed this was resolved in macOS 11. I thought it was a lot to ask but am now running on macOS 11.4 (Big Sur). My target is to output speech faster than real-time and and drive the output through AVAudioengine. First, I need to know why the write doesnt occur and neither do delegates get called whether I am using write or simply uttering to the default speakers in "func speak(_ string: String)". What am I missing? Is there a workaround? Reference: https://developer.apple.com/forums/thread/678287 let sentenceToSpeak = "This should write to buffer and also call 'didFinish' and 'willSpeakRangeOfSpeechString' delegates." SpeakerTest().writeToBuffer(sentenceToSpeak) SpeakerTest().speak(sentenceToSpeak) class SpeakerTest: NSObject, AVSpeechSynthesizerDelegate { let synth = AVSpeechSynthesizer() override init() { super.init() synth.delegate = self } func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, didFinish utterance: AVSpeechUtterance) { print("Utterance didFinish") } func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, willSpeakRangeOfSpeechString characterRange: NSRange, utterance: AVSpeechUtterance) { print("speaking range: \(characterRange)") } func speak(_ string: String) { let utterance = AVSpeechUtterance(string: string) var usedVoice = AVSpeechSynthesisVoice(language: "en") // should be the default voice let voices = AVSpeechSynthesisVoice.speechVoices() let targetVoice = "Allison" for voice in voices { // print("\(voice.identifier) \(voice.name) \(voice.quality) \(voice.language)") if (voice.name.lowercased() == targetVoice.lowercased()) { usedVoice = AVSpeechSynthesisVoice(identifier: voice.identifier) break } } utterance.voice = usedVoice print("utterance.voice: \(utterance.voice)") synth.speak(utterance) } func writeToBuffer(_ string: String) { print("entering writeToBuffer") let utterance = AVSpeechUtterance(string: string) synth.write(utterance) { (buffer: AVAudioBuffer) in print("executing synth.write") guard let pcmBuffer = buffer as? AVAudioPCMBuffer else { fatalError("unknown buffer type: \(buffer)") } if pcmBuffer.frameLength == 0 { print("buffer is empty") } else { print("buffer has content \(buffer)") } } } }
Replies
3
Boosts
0
Views
3k
Activity
Jan ’22
Can you perform two or more OFFLINE speech recognition tasks simultaneously?
Can you perform two or more OFFLINE speech recognition tasks simultaneously? SFSpeechRecognizer, SFSpeechURLRecognitionRequest offline limitation? Running on macOS Big Sur 11.5.2 I would like to be perform two or more offline speech recognition tasks simultaneously. I've executed two tasks in the same application AND executed two different applications, both using offline recognition. Once I initiate the other thread or other application, the first recognition stops. Since the computer supports multiple threads, I planned to take make use of the concurrency. Use cases #1 multiple Audio or video files that I wish to transcribe -- cuts down on the wait time. #2 split a single large file up into multiple sections and stitch the results together -- again cuts down on the wait time. I set on device recognition to TRUE because my target files can be up to two hours in length. My test files are 15-30 minutes in length and I have a number of them, so recognition must be done on the device. func recognizeFile_Compact(url:NSURL) { let language = "en-US" //"en-GB" let recognizer = SFSpeechRecognizer(locale: Locale.init(identifier: language))! let recogRequest = SFSpeechURLRecognitionRequest(url: url as URL) recognizer.supportsOnDeviceRecognition = true // ensure the DEVICE does the work -- don't send to cloud recognizer.defaultTaskHint = .dictation // give a hint as dictation recogRequest.requiresOnDeviceRecognition = true // don recogRequest.shouldReportPartialResults = false // we dont want partial results var strCount = 0 let recogTask = recognizer.recognitionTask(with: recogRequest, resultHandler: { (result, error) in guard let result = result else { print("Recognition failed, \(error!)") return } let text = result.bestTranscription.formattedString strCount += 1 print(" #\(strCount), "Best: \(text) \n" ) if (result.isFinal) { print("WE ARE FINALIZED") } }) }
Replies
1
Boosts
0
Views
1.9k
Activity
Nov ’21
AVSpeechSynthesizer buffer conversion, write format bug?
Is the format description AVSpeechSynthesizer for the speech buffer is correct? When I attempt to convert it, I get back noise from two different conversion methods. I am seeking to convert the speech buffer provided by the AVSpeechSynthesizer "func write(_ utterance: AVSpeechUtterance..." method. The goal is to convert the sample type, change the sample rate and change from mono to stereo buffer. I later manipulate the buffer data and pass it through AVAudioengine. For testing purposes, I have kept the sample rate to the original 22050.0 What have I tried? I have a method that I've been using for years named "resampleBuffer" that does this. When I apply it to the speech buffer, I get back noise. When I attempt to manually convert format and to stereo with "convertSpeechBufferToFloatStereo", I am getting back clipped output. I tested flipping the samples, addressing the Big Endian, Signed Integer but that didn't work. The speech buffer description is inBuffer description: <AVAudioFormat 0x6000012862b0: 1 ch, 22050 Hz, 'lpcm' (0x0000000E) 32-bit big-endian signed integer> import Cocoa import AVFoundation class SpeakerTest: NSObject, AVSpeechSynthesizerDelegate { let synth = AVSpeechSynthesizer() override init() { super.init() } func resampleBuffer( inSource: AVAudioPCMBuffer, newSampleRate: Double) -> AVAudioPCMBuffer? { // resample and convert mono to stereo var error : NSError? let kChannelStereo = AVAudioChannelCount(2) let convertRate = newSampleRate / inSource.format.sampleRate let outFrameCount = AVAudioFrameCount(Double(inSource.frameLength) * convertRate) let outFormat = AVAudioFormat(standardFormatWithSampleRate: newSampleRate, channels: kChannelStereo)! let avConverter = AVAudioConverter(from: inSource.format, to: outFormat ) let outBuffer = AVAudioPCMBuffer(pcmFormat: outFormat, frameCapacity: outFrameCount)! let inputBlock : AVAudioConverterInputBlock = { (inNumPackets, outStatus) -> AVAudioBuffer? in outStatus.pointee = AVAudioConverterInputStatus.haveData // very important, must have let audioBuffer : AVAudioBuffer = inSource return audioBuffer } avConverter?.sampleRateConverterAlgorithm = AVSampleRateConverterAlgorithm_Mastering avConverter?.sampleRateConverterQuality = .max if let converter = avConverter { let status = converter.convert(to: outBuffer, error: &error, withInputFrom: inputBlock) // print("\(status): \(status.rawValue)") if ((status != .haveData) || (error != nil)) { print("\(status): \(status.rawValue), error: \(String(describing: error))") return nil // conversion error } } else { return nil // converter not created } // print("success!") return outBuffer } func writeToFile(_ stringToSpeak: String, speaker: String) { var output : AVAudioFile? let utterance = AVSpeechUtterance(string: stringToSpeak) let desktop = "~/Desktop" let fileName = "Utterance_Test.caf" // not in sandbox var tempPath = desktop + "/" + fileName tempPath = (tempPath as NSString).expandingTildeInPath let usingSampleRate = 22050.0 // 44100.0 let outSettings = [ AVFormatIDKey : kAudioFormatLinearPCM, // kAudioFormatAppleLossless AVSampleRateKey : usingSampleRate, AVNumberOfChannelsKey : 2, AVEncoderAudioQualityKey : AVAudioQuality.max.rawValue ] as [String : Any] // temporarily ignore the speaker and use the default voice let curLangCode = AVSpeechSynthesisVoice.currentLanguageCode() utterance.voice = AVSpeechSynthesisVoice(language: curLangCode) // utterance.volume = 1.0 print("Int32.max: \(Int32.max), Int32.min: \(Int32.min)") synth.write(utterance) { (buffer: AVAudioBuffer) in guard let pcmBuffer = buffer as? AVAudioPCMBuffer else { fatalError("unknown buffer type: \(buffer)") } if ( pcmBuffer.frameLength == 0 ) { // done } else { // append buffer to file var outBuffer : AVAudioPCMBuffer outBuffer = self.resampleBuffer( inSource: pcmBuffer, newSampleRate: usingSampleRate)! // doesnt work // outBuffer = self.convertSpeechBufferToFloatStereo( pcmBuffer ) // doesnt work // outBuffer = pcmBuffer // original format does work if ( output == nil ) { //var bufferSettings = utterance.voice?.audioFileSettings // Audio files cannot be non-interleaved. var outSettings = outBuffer.format.settings outSettings["AVLinearPCMIsNonInterleaved"] = false let inFormat = pcmBuffer.format print("inBuffer description: \(inFormat.description)") print("inBuffer settings: \(inFormat.settings)") print("inBuffer format: \(inFormat.formatDescription)") print("outBuffer settings: \(outSettings)\n") print("outBuffer format: \(outBuffer.format.formatDescription)") output = try! AVAudioFile( forWriting: URL(fileURLWithPath: tempPath),settings: outSettings) } try! output?.write(from: outBuffer) print("done") } } } } class ViewController: NSViewController { let speechDelivery = SpeakerTest() override func viewDidLoad() { super.viewDidLoad() let targetSpeaker = "Allison" var sentenceToSpeak = "" for indx in 1...10 { sentenceToSpeak += "This is sentence number \(indx). [[slnc 3000]] \n" } speechDelivery.writeToFile(sentenceToSpeak, speaker: targetSpeaker) } } Three test can be performed. The only one that works is to directly write the buffer to disk Is this really "32-bit big-endian signed integer"? Am I addressing this correctly or is this a bug? I'm on macOS 11.4
Replies
5
Boosts
0
Views
2.9k
Activity
Oct ’21
How to catch "unregister notification for read_timeout failed" from going to XCode console
How do you register to catch these notifications from going to XCode console? The messages occur whenever I execute a URLSession.shared.dataTask, which is often. The messages are not an indication the code has faulted but are notifications for unknown reasons that fill the console. You should only get a notification if something is wrong. How do you register to catch this message so it does not go to the XCode console? nw_endpoint_handler_set_adaptive_read_handler [C14.1 104.21.42.21:443 ready socket-flow (satisfied (Path is satisfied), viable, interface: en0, ipv4, dns)] unregister notification for read_timeout failed nw_endpoint_handler_set_adaptive_write_handler [C14.1 104.21.42.21:443 ready socket-flow (satisfied (Path is satisfied), viable, interface: en0, ipv4, dns)] unregister notification for write_timeout failed I believe this will duplicate the issue func queryEndpoint(address: String) { let url = URL(string: address) let task = URLSession.shared.dataTask(with: url!) {(data, response, error) in let result = String(data: data!, encoding: String.Encoding.utf8)! print(result) } task.resume() }
Replies
1
Boosts
0
Views
1.8k
Activity
Sep ’21
XCode 13 Quick Help partially working
I upgraded to XCode 13. Previously versions of XCode would show the variable name and type in Quickhelp, when you clicked on the variable "let classRef: ViewController" (example) or a user defined methods, it would show its declaration information. Now, quick help only shows information when you click on a built-in function or a function parameter I thought quitting XCode or cleaning the project would resolve this but it did not. This features was extremely beneficial, where you could easily click on an item and copy its declaration information or just see the types being reference. How do I get the previous behavior back?
Replies
1
Boosts
0
Views
1.5k
Activity
Sep ’21