Just have my own experience to share. I have three goldfish tanks-- one has fine black gravel, one has natural sand and the other pea gravel. Of the three tanks, the easiest to take care of is the planted pea gravel tank. I have tried to do a bubble eye tank with large river rock. You will pull your hair out by the time the tank cycles-- nearly impossible to vacuum.
If you are chosing substrate for goldies, one thing to remember is that they like to "mouth" everything. If they can get it through their gills (like sand) it's fine. Otherwise, you have to be very careful with crushed aquarium gravel. Pea gravel, however, works like a charm and creates a very natural looking tank with some of the coarser plants that goldies won't tear apart. Good luck.