I also spent weeks battling signing issues.
If you're using Xcode 14, you can simply select "Automatic" in the Signing and Capabilities tab. As it says in that tab, "Xcode will create and update profiles, app IDs and certificates". Choose "Development" for the signing certificate.
Delete any app IDs or profiles you may have made manually on the developer portal for your app or its extension, and let Xcode do all the work for you. If you don't, Xcode will try to use the manually generated profiles and you'll get a message like "Xcode 14 and later requires a DriverKit development profile enabled for IOS and macOS. Visit the developer website to create or download a DriverKit profile".
Automatic signing will let you develop and test on your own machine, or any other test machine entered on the portal under Devices.
You asked "Does the Code Signing Identity/Profile in the Apple Developer section need to match exactly, or can that have more than what is requested? "
If you look at the embedded profile that Xcode puts in your app, you'll see what it creates (for development). It looks at the entitlements you request, and puts those same entitlements in the profile. In my case I have a com.apple.developer.driverkit.transport.usb entitlement which specifies my company's USB vendor ID. The development embedded profile specifies "*" as the vendor ID, the distribution profile specifies my company's VID.
When you come to distribute your app, Xcode can't create everything for you. The Account Owner has to go into the portal and create profiles for any extensions which require managed capabilities (entitlements which are only provided in response to a specific request to Apple). An Admin can create other types of profiles. If you are not the Account Owner the Generate button will fail to generate a profile, but it won't tell you why.
In the distribution workflow, you choose the manually-generated profiles when you come to upload your product for notarization (not at build time). So you can keep the "Signing and Capabilities" setting in Xcode at Automatic.