[Update 2022-01-29] macOS 12.2 seems to have a solution for this problem. I've done a complete clean re-install (with re-formating the entire drive). I've run a 24hr Blackmagic Disk Speed Test, Safari open (the app that most usually made it trigger this issue). Without the issue. Could more people try 12.2 too? and see if it solved their issue?

@HK2LS Go into System Preferences --> Network --> on press "the tiny wheel" (see attached image) --> Set Service Order --> Re-arrange "Ethernet" to be above "Wi-Fi" I had this issue too, where everything was showing that it's connected and working. But I was still getting slow Wi-Fi speeds. Re-arranging the Service Order, helped me. Hope it helps you too.

Update 2022-01-12: Filed *this in FBA: FB9842248 Filed a Feature Request with Objective Development (Little Snitch devs...): 1789897 /* I will try to keep this post updated, with any news (if someone is interested) ... */

@robnotyou The "iCloud Private Relay" is located under: System Preferences --> Apple ID --> iCloud --> iCloud Private Relay And know about it. but this "Limit IP Address Tracking" is located under the Network settings: System Preferences --> Network --> Ethernet --> Limit IP Address Tracking And I haven't seen it previously when I tested some of the macOS 12 Beta's. Don't know when they added this new option under the Network options. And why I'm wondering it's now there :-)