Cleaner Shrimp & Anemone

I've got a RBTA and a cleaner. When I put a silverside in my anemone, the cleaner will agressively try to steal it, and often does. I usually have to give him a little piece of his own before feeding my RBTA. He's not bothered at all by the anemone. I also have a Singapore angel that will do the same thing.
 
I have a similar issue with my cleaner harassing my long-tentacle plate coral. He feels it all over and the tentacles shrink away. He does move on eventually, but he does pick at it a lot. Should I be concerned that the cleaner is disrupting the coral to the detriment of its health?

Thanks
 
AquariaCentral.com