As a workaround I tweaked my build script to call codesign again and again if it fails, and that seems to have solved my issue (at the expense of time). So it definitely seems that the timestamp server is flaking out.

Okay, I did all that. I used Feedback Assistant and was assigned FB9724101.