What fixed it for me was:
Close Xcode
Delete those folders:
~/Library/Caches/org.swift.swiftpm
~/Library/Developer/Xcode/DerivedData
Restart Xcode
Right click and "update package"
Topic:
Developer Tools & Services
SubTopic:
Xcode
Tags: