Bartender Question

I'm using Bartender to manage my action bars and I've unchecked the "Enable" box for the Extra Action Button (I have it keybound, so I don't want it appearing in my UI). However, lately, sometimes the extra action button shows up anyway (it seems to always happen on Tortose). If, when it happens, I open up Bartender and enable and re-disable Extra Action Button, it goes away again...until the next time it happens.

Any idea what might be causing this?