They said it. Ammonia from fish waste (and decaying matter like leftover food) gets turned into slightly safer nitrite by bacteria. Other kinds of bacteria turn that nitrite into even more safe nitrate. However, nitrate is still harmful. The exact effects are debated, but basically, your fish will be less healthy, shorter lived, and less likely to be fertile.
Anyway, nitrates can only be removed by water changes to plants using them for food. If you have a really heavily planted tank and/or plants that grow quickly (which uses up a lot of food in the form of nitrate, etc.), you can easily control nitrate buildup and even lower it to the point where you have to add it for the health of the plants.
HOWEVER, there are other pollutants that either are not easily or not able to be measured that build up when you don't do water changes, such as fish hormones which can cause serious health defects and stunting.