Brown isn't necessarily bad, since that is the color of the photosynthetic zooxanthellae. White is bad.
It's hard to know what's going on without knowing the condition of your tank.
How big is the tank?
How long has it been set up?
Is there any ammonia or nitrite?
How's the nitrate?
ph, alkalinity?
How much light? Anemones need quite a bit, and becoming darker may be a sign that the zooxanthellae are multiplying to make up for inadequate light.
Are you feeding it?