You definitely want to at least rinse out that prefilter sponge at least every week, as it is the first line of defense for all the crud that enters the filter. Like Byron, I rinse my sponges with every water change.
Just one example of numerous algae guides pointing to low flow as a culprit. I am not denying that high organics are a cause as well. It is usually multifactorial is my point. Given that it is present in only 1 filter…..