Every time you pass the tank, take a moment to watch your fish.
If you're seeing active scraping or scratching on rocks, it is likely a parasitic infestation. If you don't see any unusual behavior, I would suspect it's simply normal scale loss. Sometimes goldfish lose a scale or two on tank decorations while rooting around for food, etc. But do look for any scratching or twitching - after all, these are pond fish and much more likely to harbor parasites than indoor fish.