sand is better is one way because the goldfish really like to suck it in and spit it back out. Waste entrapment is of course the downside of it. Gravel, however, may cause problems with that same behavior - occasionally they grab more than they can deal with and choke on it. I've seen that posted several times here. Once the person managed to find the fish in time and remove the gravel with some tweezers, but the others weren't so lucky.
also - watch your comet and common goldfish - they might outcompete the fancy goldfish for food. Usually they shouldn't be kept together for that reason (same for regular fancy goldies being kept with sight-impaired goldies like celestials and moors) - but if you make sure that they all get enough food you'll be okay.
actually sand is not a great choice, since goldies do love to mouth things and a certain amount of sand would doubtless be passed through the gills and intestines causing irritation. mine do pick up and spit out gravel, and graze at the algae that grows on them. some feel that there is a risk of gravel getting trapped in a fishes mouth but in that instance, everyone that feels the risk is great opts for larger stones instead of sand. mine have not had this problem. i have a fantail and a ryukin in a 35 with an ovation 1000 and a cascade 100 and that is really pushing it, so as one other member posted, id can the idea of the common.