Maybe my problem was that I used "CPU Profiler" instead of "Time Profiler". Anyways, 15.4 is out now and with that installed everything works fine :)

Is there a way to profile my software without having to use a beta macOS version? Can I for example download the old and working version of Instruments.app somewhere until 15.4 has been released and I've had time to install it?