I had similar problems to make this working. Finally, I found that it's not enough to add the swift package. Once added, I need to go in the app target => General => Frameworks and add it as any Apple SDK or local framework. In the framework picker, there is a section where my swift package appears. So, Xcode recognises the package as a framework but you still need to include it in the target.
Once this worked, I successfully tried to use a relative path in the .binaryTarget() and put my framework in a subfolder. Just needed to quit Xcode and remove the DerivedData folder to clear all caches.
Topic:
Developer Tools & Services
SubTopic:
Xcode
Tags: