I've had them for years and never had that problem.
Anubia is an algae magnet. They are beautiful and easy to grow, but they(especially the Barteri and other broad leaf varieties) are just prone to collecting algae.
I have tanks with a minimum of algae anywhere and the Anubia leaves are loaded with it. Every now and then I pull them out, driftwood and all, and give them a 19:1 bleach bath. They then look great for 2-3 months and start collecting again.
"Brown" algae sounds like too little light. How much are you supplying?
Len