Hey @secretchimp,
Thanks so much for posting this question! If you were to search the Apple Developer Forums for this capability, you would find years worth of requests for Apple to support this capability with not much interaction from Apple. Today, I don't know of a formal solution that exists from Apple for managing Enterprise content programmatically.
To get to the crux of your question, the solution today is to continue to use Fastlane or another solution that is leveraging the same approach.
However, we live in a world of technology, so there are many paths to take that could potentially speed up your workflow. I will go into some of them, but each organization will need to determine what works best for them.
As your organization continues to scale, I strongly recommend continuing to look at Custom Apps. I work at a large organization with hundreds of applications and there will never be a "right" time to make the move, but Custom Apps serves the organization better in a few key areas.
You no longer need to maintain your own infrastructure to make internal software available and provide updates. Apple does all of this for you for $100 a year. I don't know of another service where this is possible.
You get access to the API that does what you are looking to do and then some.
You do not need to respond to OS code signature changes for "stale" applications. When Apple updates the code signature minimum, you don't need to update your applications, the App Store does this for you.
Most importantly, you do not need to resign your applications on an annual basis or even create new certificates or provisioning profiles. Apple does all of this for you.
If all of your developers have the same requirements to meet while developing applications (Enterprise applications are the Wild West), and it is the same model that is used by the talent pool you are hiring from, collaboration and cross training becomes much easier.
Specifically regarding your challenge with internal applications and systems that live behind firewalls, these are all good things to bring up to Apple and request improvements to the App Review Guidelines. The App Review Guidelines do indicate that a fully featured demo mode is acceptable.
Provide App Review with full access to your app. If your app includes account-based features, provide either an active demo account or fully-featured demo mode, plus any other hardware or resources that might be needed to review your app (e.g. login credentials or a sample QR code)
Consider moving your development work to an Apple Developer Account instead of an Apple Developer Enterprise Account. By doing this, you once again gain access to the API and can automate more of your workflows. Doing this would also set you up for long term success in the event that you do migrate to Custom Apps. There is no difference in the development capabilities between an Apple Developer Program and Apple Developer Enterprise Program in terms of numbers of devices and profiles, so you could leverage Fastlane to create two artifacts instead of one and then only need to manage your "Universal Distribution" profile once a year (or at the rate you deem necessary based off user behavior and device management capabilities).
Many development tasks including running applications on devices no longer require a paid Apple Developer Account. Investigating whether or not your applications require a paid Apple Developer Account, and whether or not they are using Apple ecosystem services may reveal that with the proper Xcode project configuration, the majority of your developers can be working on software development while not needing to be on a team account and instead can leverage free personal accounts.
One of the possibilities above may help, or you may find another that can do so much more for your organization, but hopefully you can find something that further enables you in both the short and long run setting you up for continued success!
Hopefully this helps and happy coding!