We're currently experiencing this as well. Our setup is similar to yours - we have one Xcode project with several swift packages. File operations are the most painful, since every addition or renaming causes Xcode to freeze for 10-30 seconds. This happens in Xcode 14.1.