What does your app do if the user declines location permission? Is the problem that they require it to work in some sense even if location permission is denied? At a minimum it mustn't crash. Consider using a default location (e.g. your home town) if necessary.
Topic:
App Store Distribution & Marketing
SubTopic:
App Review
Tags: