Just to close the loop, after some testing, I believe the issue was with my certificate. I downloaded it again from the Apple developer site and ran through the same process and it appears to have worked. So despite there being no flag when codesigning with a bad certificate, the app was seen to be from an untrusted developer and was deleted automatically.