If they aren't live rock, you can take them out a few at a time and hit them with a 20% bleach solution (a cup or two of bleach to a gallon of water), soak for 12-24 hours, rinse well, and soak in a double-shot of chlorine remover (use the cheapest stuff you can find) for 12 hours. No hint of chlorine (bleach) will be left. Rocks will be clean and white, no scrubbing!
Don't do this to sand. If you have algae on the sand, are you *sure* it's not slime algae (blue-green algae, cyanobacteria)?