I finally found a solution. According to this document, the user first needs to pick a folder to give the app access by returning a security-scoped URL (which you can then bookmark for later use) for it and all of its content. After the user picks the folder, they can do whatever they want on another picker for the specified file type.
It's still way too complicated for users to understand why they have to do this, so I made a user-friendly flow so that users are informed on how to continue. It still lacks some toast notifications for success but you'll get the idea. The code is on the repo I posted before.
If there's a way to make this in one step instead of having to prompt the user for a folder selection, it would be great. Please do let me know. Until then, this is what I could come up with.