I see this issue both on localhost and my remote server where I generate the key, so I think my issue is different. Thank you for the reply though!

I never found a solution. I migrated to musickit v1, but auth is still not working for my app. I'm not sure if it's somehow related to using reactjs for my front end. Any help is appreciated!

To anyone encountering an issue similar to this, the issue is your referrer-policy settings. I'm using strapi which uses helmetjs out of the box. My referrer-policy was configured to "no-referrer" by default, but setting it to "origin" fixed this auth issue.