My rule of thimb on sponges is that the third rinse should be clear, clean or nearly clean water. If it needs more than 3 rinses, then I need to maintain the filter more often or to have more filtration. Either option can do the job.
All JMHO and IME, YMMV.
BTW, the dirty stuff from rinsing filters is called "mulm" and is great for starting up new planted tanks - well mineralized nutrient mixture which will add to tne base layer of your new substrate. If the material is fresh, it will assist in avoiding cloudy water outbreaks in the new tank. It has live heterotrophic bacteria to help establish the needed complement of mixed buggs in our tanks.