On Android, I have the mic, location, and camera blocked via the pulldown tiles menu. I turn them on when needed. The OS and some apps like to bitch about this sometimes but it seems to be working ok.
My iphone does not offer these blanket blocking options. It's a work phone, so I just leave it off unless I need it.