Ah, if you unselect all additional downloads, then it downloads the main Xcode like you would expect. (Doesn't download xcdownloadmanifest)