By creating a new project and painstakingly moving the files/settings over one by one from my old one too see where it fails I found out that when I have more than 5 Configurations it starts to fail. It's rather random but it might give apple devs some clue on how to fix it. My project is using cocoapods.
When I removed the configurations, XCode continued to fail so it's not easy to reproduce. I also created a fresh project with the podfile from my big project but that one didn't fail on more than 5 configurations.
Only thing I know is that this is a pretty serious error and makes programming with XCode a complete nightmare.