I had the same problem using Pyinstaller and sklearn.
I successfully notarized such an app by first codesign the .dylibs/libomp.dylib file and then the remaining using the --deep option since I believe that you did the same (which is not recommended to be used - see here: https://developer.apple.com/forums/thread/128166 and https://developer.apple.com/forums/thread/129980).
Did you resolve the issue using a different approach?
Cheers
Topic:
Code Signing
SubTopic:
Notarization
Tags: