Yeah, I just got bitten by the same issue! I set the installer to download overnight (fairly slow internet here) and woke up to see my MBP had been upgraded automatically! Why on earth has this process changed??
I can't even see how to downgrade back to Monterey.
I've removed the machine from the Beta access programme and checked software update, but it's not offering 12.4. Using the "Download" button on developer.apple.com/download simply opens the App Store on the page for Monterey. When I click the "Get" button, it just takes me straight back into Software Update where it shows "Your Mac is up to date".
I do have a backup (thanks CCC!)...but of course, they're not bootable backups these days, so I booted into Recovery mode, to see that the only option there is to "Reinstall MacOS Ventura". Talk about unlucky number 13!
I don't have any particular issue with Ventura, at least not yet anyway, but I certainly didn't want it to usurp my main laptop.