Okay so it worked, I suggest moving the Xcode.app with cp -R <Xcode.app path> <destination>, I have the impression Finder struggles moving such a huge file.