To follow up on this, I now have an answer, which is that Apple did accept my app into the App Store. ADHG - Yes, in general, Mac Catalyst apps can be distributed outside the App Store. However apparently this is not possible for apps that link against HomeKit. These have to go through the App Store.

Anticro - Apple have added HomeKit support for Catalyst in Big Sur. If you get the latest Xcode beta and Big Sur beta you can start using HomeKit in your Catalyst app :) As per my original question though, what I'm concerned about is whether Apple will allow an app into the App Store that doesn't really have many features itself, but rather acts as a bridge between a macOS app and HomeKit. It seems like this should be allowed, but I haven't been able to determine this for sure.