I have MacBook Pro M1 2021 15.1. I have similar issue. When I swipe right it scrolls back. I even tried Magic Trackpad it behaves same. It is very irritating. i will be happy if some one have the solution.