there are a variety of dissolved ORGANIC POLLUTANTS, substances such as proteins, amino acids, phenolic compounds, pheromones (hormones that affect the behavior of other fish) and other metabolic byproducts which are continuously being discharged into the aquarium water by the fish. If these DOCs are allowed to become concentrated in the water, the health of the fish will suffer.
for example, laboratory studies have shown that high levels of DOC are associated with reduced fish feeding rates, slower growth, decreased reproduction rates, lower immune system activity and blooms of pathogenic bacteria in the water. it is now believed that many of the fish health problems thought to be the result of high nitrates are actually caused by DOCs.
in addition to those organic compounds noted above, inorganic byproducts of the nitrogen cycle in any aquatic environment also buildup in an your water. ammonia and nitrite are taken care of by your biofilter and converted to nitrate. this latter product can only be removed by dilution through water changes.
so while you may be thinking your fish are just fine if your water chemistry seems fine, even without water changes, in fact they are not.