After hours of playing around with App Store and deleting everything I could I manage to fix it on my Mac. Here is what I did.
I created a new Apple ID, I signed out from the current one.
Logged in with the new Apple ID in App Store / Mac
Tried download again - still not working
Restarting my Mac
Downloading works!
Sign out from the new Apple ID
Log in with my Apple ID
Remove the app downloaded with the new Apple Id
Restart Mac
Downloaded the app with my Apple ID without any problem - What changed was that this time it ask me for verification.
Just for fun, I removed the app and re-download it again and it works now fine.
This is super annoying but it fixed my problem. I hope it helps.
Topic:
App Store Distribution & Marketing
SubTopic:
TestFlight