Same issue here, Xcode 13.2 on Monterey 12.0.1. I was able to get it to go away for a few hours, not sure how exactly, may have involved deleting and re-adding the package manually. Also I cleared derived data, etc. Now it's back anyway, and I can't communicate with the package server, so I can't re-add it.

I'm curious to know this too.