I found myself in the same situation and I had the same conclusion as you, did you find any solution to it besides not adding an kSecAttrAccessGroup ?

Did you find the solution to this?