I can share pretty much anything that could be helpful, all of these dylibs are in-house compiled versions of open source projects so I'm going to link to a XZ archive containing a few of those that are most often involved in the crashes (in this case openCASCADE, specifically libTKBO.7.dylib). There is no difference between the working and the crashing case as I do not modify the dylibs other than applying the copy, delete, replace workaround if needed.
One thing that makes me believe it might even be some weird caching or APFS issue is the fact that, if I create a simple "Hello World" C program and link it against one of the dylibs it will normally run just fine except in the case where otool would also crash. If I get a crash from otool -L and try to run the binary linking to the affected dylib it will also crash pointing to a failure to verify the code signature of @rpath/NAME_OF_LIBRARY.
https://www.icloud.com/iclouddrive/0m1cKLZsmmdbXfK7xg9VORzFA#openCASCADE_sample it's shared for meaton3 apple.com, libTKBO.7.dylib is one of the libs mostly causing the problem.
A few more details to my working environment:
MacBook Air M1, late 2020
macOS Big Sur 11.5.2 (issue also happened on 11.5.1)
Xcode 12.5
all files are stored on an external USB 3 SSD formatted with APFS