If you are not adding phosphate via some chemical that is a pretty good build-up. What is your water change routine?, what is your nitrate level? What f any chemicals are you adding aside from dechlorinator.
If you aren't adding phosphate via chemical adjustments, the level is a result of the fish food you add most likely, and simply indicates a need for more frequent or higher volume partials.
If it's being added via chemical additions then I would look at ways to avoid those chemicals.
Phosphates aren't nearly as bad as most folks think, but will contriute to algae growth and high levels give no benefits to anything but plants and algae.
Dave