You can also click 'remove from sale' at the bottom of the "Pricing and Availability" tab. And once that's done I was able to remove the app in 'App Information'