Same issue, tried everything but the entries of uninstalled apps remain

I had the same issue but I found a fix, run the following command: rm ~/Library/GroupContainersAlias/group.com.apple.controlcenter/Library/Preferences/group.com.apple.controlcenter.plist and log out and back in