Post

Replies

Boosts

Views

Activity

Reply to SIGSTOP logd causes panic?
We hit userspace watchdog panics (logd unresponsive on firehose I/O) during sustained writes into a large APFS file that was created with clonefile and then overwritten at many offsets. The workload was high-rate, relatively small pwrites with F_NOCACHE (and F_NODIRECT) on the clone. That behaved like sparse/COW allocation pressure even though the logical file size looked fully populated. Only reproducible on 2TB AP2048Z SSD. Couldn't get it to panic on any other type of hardware we have :( We understand clone + overwrite implies COW extent work similar to sparse writes. We’re mainly sharing this as a data point: under prolonged high-IOPS uncached pwrite into a clone, APFS metadata / system services (esp. logd/firehose) can fall far enough behind to trip the userspace watchdog, and the failure mode is easy to misread as a 'preallocated file should be fine' thing. Unfortunately we had to use customer hardware don't have any 2TB drives, but two customers did) and they won't allow us to share anything else. If it changes, I'll submit a feedback report. Otherwise we can just close this. I found a way to work around it. Thanks!
Topic: App & System Services SubTopic: Core OS Tags:
2w
Reply to SIGSTOP logd causes panic?
Some other logs to understand what we're seeing: com.apple.logd.store in TH_WAIT | TH_UNINT for ~120–200s com.apple.firehose.drain-io / io-wl blocked behind it then watchdog: no logd checkins for 120s userspace watchdog timeout: no successful checkins from logd ... 120 seconds unresponsive dispatch queue(s): com.apple.firehose.io-wl
Topic: App & System Services SubTopic: Core OS Tags:
3w
Reply to Does virtualizing macOS 27 require a macOS 27 host?
Both my dev and I confirmed 26.6b3 host OS with not only a pre-existing broken Xcode 27 beta 2 install upgraded to Xcode 27 beta 3, but also no Xcode beta 27 installed at all being installed after the 26.6b3 install. Both work fine now with Anka.
Topic: Core OS SubTopic:
Virtualization Q&A
Tags:
3w
Reply to Does virtualizing macOS 27 require a macOS 27 host?
I'll be trying upgrading my broken Xcode 27 beta 2 installed host machine to 26.6b3 today once things calm down and and test it out. But yeah, testing the install 26.5, upgrade, then install Xcode 27 beta (without ever having it on the machine) is also a great idea
Topic: Core OS SubTopic:
Virtualization Q&A
Tags:
4w
Reply to Installed Xcode 27 beta 2, now I can't create any VMs
I tried sudo rm -rf /Applications/Xcode*.app sudo rm -rf /Library/Developer/CommandLineTools sudo rm -rf /Library/Developer rm -rf ~/Library/Developer rm -rf ~/Library/Caches/com.apple.dt.Xcode rm -rf ~/Library/Application\ Support/Xcode rm -rf ~/Library/Application\ Support/com.apple.dt.Xcode rm -rf ~/Library/Preferences/com.apple.dt.Xcode* rm -rf ~/Library/Saved\ Application\ State/com.apple.dt.Xcode.savedState sudo xcode-select --reset And I manually deleted the CoreSimulators. I restarted, but it's still broken. Even installed older Xcode 26 and ran XCODE_DESTINATION="/Applications" XCODE_APP="Xcode.app" sudo /usr/sbin/dseditgroup -o edit -a everyone -t group _developer sudo xcode-select -s ${XCODE_DESTINATION}/${XCODE_APP}/Contents/Developer sudo xcodebuild -license accept sudo xcodebuild -runFirstLaunch sudo DevToolsSecurity -enable for PKG in $(/bin/ls ${XCODE_DESTINATION}/${XCODE_APP}/Contents/Resources/Packages/*.pkg); do sudo /usr/sbin/installer -pkg "$PKG" -target / done echo "Checking Xcode CLI tools" sudo xcode-select -s "${XCODE_DESTINATION}/${XCODE_APP}" xcode-select -p &> /dev/null if [ $? -ne 0 ]; then echo "Xcode CLI tools not found. Installing them..." touch /tmp/.com.apple.dt.CommandLineTools.installondemand.in-progress; PROD=$(softwareupdate -l | grep "\*.*Command Line" | tail -n 1 | sed 's/^[^C]* //') softwareupdate -i "$PROD" --verbose; else echo "Xcode CLI tools OK" fi I must be missing something still...
Topic: App & System Services SubTopic: Core OS Tags:
4w
Reply to Does virtualizing macOS 27 require a macOS 27 host?
Just tested the latest Xcode, 26.6b3, and still can't create 27 VMs. failed to install macOS: Error Domain=VZErrorDomain Code=10007 "Installation failed." UserInfo={NSLocalizedFailure=An error occurred during installation., NSLocalizedFailureReason=Installation failed., NSUnderlyingError=0x7e001c1e0 {Error Domain=com.apple.MobileDevice.MobileRestore Code=-1 "AMRestorePerformRestoreModeRestoreWithError failed with error: 11" UserInfo={NSLocalizedDescription=AMRestorePerformRestoreModeRestoreWithError failed with error: 11, NSLocalizedFailureReason=An unknown error occurred during installation.}}}
Topic: Core OS SubTopic:
Virtualization Q&A
Tags:
Jun ’26
Reply to Enforced segregation between VM and Host
I found a way to do this in Anka. It allows me to prevent any communication at all between the host and VM. We're using it in a very frequently used public cloud that serves thousands of VMs a day and it's passed a ton of pen testing.
Topic: Core OS SubTopic:
Virtualization Q&A
Tags:
Jun ’26
Reply to open / libsystem_kernel.dylib slow on first run for any .img
Ultimately we didn't solve the first open delay. It's understandable that there will be a bit of a delay for the first call. But, we were seeing like 10+ seconds for a specific APFS image. It turned out after-all to be a problem with how we were reading only certain images that were created with older versions of Anka. Doesn't happen on newer images we were creating so we're going to just create and use newer ones.
Topic: App & System Services SubTopic: Core OS Tags:
May ’26
Reply to SIGSTOP logd causes panic?
We hit userspace watchdog panics (logd unresponsive on firehose I/O) during sustained writes into a large APFS file that was created with clonefile and then overwritten at many offsets. The workload was high-rate, relatively small pwrites with F_NOCACHE (and F_NODIRECT) on the clone. That behaved like sparse/COW allocation pressure even though the logical file size looked fully populated. Only reproducible on 2TB AP2048Z SSD. Couldn't get it to panic on any other type of hardware we have :( We understand clone + overwrite implies COW extent work similar to sparse writes. We’re mainly sharing this as a data point: under prolonged high-IOPS uncached pwrite into a clone, APFS metadata / system services (esp. logd/firehose) can fall far enough behind to trip the userspace watchdog, and the failure mode is easy to misread as a 'preallocated file should be fine' thing. Unfortunately we had to use customer hardware don't have any 2TB drives, but two customers did) and they won't allow us to share anything else. If it changes, I'll submit a feedback report. Otherwise we can just close this. I found a way to work around it. Thanks!
Topic: App & System Services SubTopic: Core OS Tags:
Replies
Boosts
Views
Activity
2w
Reply to SIGSTOP logd causes panic?
Is there a way to disable logd to test around it? We tried log config --mode "level:off" but it didn't seem to prevent the panic. We're actually using F_PREALLOCATE already which is what's so odd about this.
Topic: App & System Services SubTopic: Core OS Tags:
Replies
Boosts
Views
Activity
2w
Reply to SIGSTOP logd causes panic?
Some other logs to understand what we're seeing: com.apple.logd.store in TH_WAIT | TH_UNINT for ~120–200s com.apple.firehose.drain-io / io-wl blocked behind it then watchdog: no logd checkins for 120s userspace watchdog timeout: no successful checkins from logd ... 120 seconds unresponsive dispatch queue(s): com.apple.firehose.io-wl
Topic: App & System Services SubTopic: Core OS Tags:
Replies
Boosts
Views
Activity
3w
Reply to Does virtualizing macOS 27 require a macOS 27 host?
Both my dev and I confirmed 26.6b3 host OS with not only a pre-existing broken Xcode 27 beta 2 install upgraded to Xcode 27 beta 3, but also no Xcode beta 27 installed at all being installed after the 26.6b3 install. Both work fine now with Anka.
Topic: Core OS SubTopic:
Virtualization Q&A
Tags:
Replies
Boosts
Views
Activity
3w
Reply to Does virtualizing macOS 27 require a macOS 27 host?
I'll be trying upgrading my broken Xcode 27 beta 2 installed host machine to 26.6b3 today once things calm down and and test it out. But yeah, testing the install 26.5, upgrade, then install Xcode 27 beta (without ever having it on the machine) is also a great idea
Topic: Core OS SubTopic:
Virtualization Q&A
Tags:
Replies
Boosts
Views
Activity
4w
Reply to Installed Xcode 27 beta 2, now I can't create any VMs
This is more of a question about how to revert the changes 27 beta did to my host. The other thread seems related to the creation of 27 VMs. I'm just trying to use Virtualization for older 15 and 26 VM IPSWs, not 27.
Topic: App & System Services SubTopic: Core OS Tags:
Replies
Boosts
Views
Activity
4w
Reply to Installed Xcode 27 beta 2, now I can't create any VMs
I tried sudo rm -rf /Applications/Xcode*.app sudo rm -rf /Library/Developer/CommandLineTools sudo rm -rf /Library/Developer rm -rf ~/Library/Developer rm -rf ~/Library/Caches/com.apple.dt.Xcode rm -rf ~/Library/Application\ Support/Xcode rm -rf ~/Library/Application\ Support/com.apple.dt.Xcode rm -rf ~/Library/Preferences/com.apple.dt.Xcode* rm -rf ~/Library/Saved\ Application\ State/com.apple.dt.Xcode.savedState sudo xcode-select --reset And I manually deleted the CoreSimulators. I restarted, but it's still broken. Even installed older Xcode 26 and ran XCODE_DESTINATION="/Applications" XCODE_APP="Xcode.app" sudo /usr/sbin/dseditgroup -o edit -a everyone -t group _developer sudo xcode-select -s ${XCODE_DESTINATION}/${XCODE_APP}/Contents/Developer sudo xcodebuild -license accept sudo xcodebuild -runFirstLaunch sudo DevToolsSecurity -enable for PKG in $(/bin/ls ${XCODE_DESTINATION}/${XCODE_APP}/Contents/Resources/Packages/*.pkg); do sudo /usr/sbin/installer -pkg "$PKG" -target / done echo "Checking Xcode CLI tools" sudo xcode-select -s "${XCODE_DESTINATION}/${XCODE_APP}" xcode-select -p &> /dev/null if [ $? -ne 0 ]; then echo "Xcode CLI tools not found. Installing them..." touch /tmp/.com.apple.dt.CommandLineTools.installondemand.in-progress; PROD=$(softwareupdate -l | grep "\*.*Command Line" | tail -n 1 | sed 's/^[^C]* //') softwareupdate -i "$PROD" --verbose; else echo "Xcode CLI tools OK" fi I must be missing something still...
Topic: App & System Services SubTopic: Core OS Tags:
Replies
Boosts
Views
Activity
4w
Reply to Does virtualizing macOS 27 require a macOS 27 host?
Host: macOS 26.6 b3, correct. Xcode: I grabbed the latest beta from the site, but have since removed it. I can't remember what it was :( Just to confirm: I launched it, installed whatever it asked me to, accepted the license, etc.
Topic: Core OS SubTopic:
Virtualization Q&A
Tags:
Replies
Boosts
Views
Activity
Jul ’26
Reply to Does virtualizing macOS 27 require a macOS 27 host?
Just tested the latest Xcode, 26.6b3, and still can't create 27 VMs. failed to install macOS: Error Domain=VZErrorDomain Code=10007 "Installation failed." UserInfo={NSLocalizedFailure=An error occurred during installation., NSLocalizedFailureReason=Installation failed., NSUnderlyingError=0x7e001c1e0 {Error Domain=com.apple.MobileDevice.MobileRestore Code=-1 "AMRestorePerformRestoreModeRestoreWithError failed with error: 11" UserInfo={NSLocalizedDescription=AMRestorePerformRestoreModeRestoreWithError failed with error: 11, NSLocalizedFailureReason=An unknown error occurred during installation.}}}
Topic: Core OS SubTopic:
Virtualization Q&A
Tags:
Replies
Boosts
Views
Activity
Jun ’26
Reply to Enforced segregation between VM and Host
I found a way to do this in Anka. It allows me to prevent any communication at all between the host and VM. We're using it in a very frequently used public cloud that serves thousands of VMs a day and it's passed a ton of pen testing.
Topic: Core OS SubTopic:
Virtualization Q&A
Tags:
Replies
Boosts
Views
Activity
Jun ’26
Reply to IPSW for 15.7.7 missing
For the full list: FB22875951
Topic: App & System Services SubTopic: Core OS Tags:
Replies
Boosts
Views
Activity
May ’26
Reply to IPSW for 15.7.7 missing
Opened FB22869718
Topic: App & System Services SubTopic: Core OS Tags:
Replies
Boosts
Views
Activity
May ’26
Reply to open / libsystem_kernel.dylib slow on first run for any .img
Ultimately we didn't solve the first open delay. It's understandable that there will be a bit of a delay for the first call. But, we were seeing like 10+ seconds for a specific APFS image. It turned out after-all to be a problem with how we were reading only certain images that were created with older versions of Anka. Doesn't happen on newer images we were creating so we're going to just create and use newer ones.
Topic: App & System Services SubTopic: Core OS Tags:
Replies
Boosts
Views
Activity
May ’26
Reply to open / libsystem_kernel.dylib slow on first run for any .img
Moderators, please delete this. I think we figured it out.
Topic: App & System Services SubTopic: Core OS Tags:
Replies
Boosts
Views
Activity
May ’26
Reply to open / libsystem_kernel.dylib slow on first run for any .img
https://feedbackassistant.apple.com/feedback/22784807
Topic: App & System Services SubTopic: Core OS Tags:
Replies
Boosts
Views
Activity
May ’26