Hi WaterlooUncovered ,
First of all, welcome to the Buttonizer Community!
Currently it's not possibe to hide Buttonizer if the user doesn't want it. But it would be a nice feature request, a 'cross' to hide Buttonizer.
Another way would be to hide Buttonizer with page rules for specific user groups or pages.
Let me know if this helps!