The point is that over time there will be a build of a "things" (whether that be nutrients, or hormones, or whatever) that the filtration in a conventional tank cannot take care of. As many have mentioned, the point of doing water changes is to maintain a relatively balanced level of "water quality" (and I use that term loosely). By replacing portions of the tank water on a regular basis with water that is filtered beyond the capabilities of an aquarium filtration system, that balance is being maintained.
The fact that the fish are stressed after a water change is a sign that water changes are necessary. If, within the timespan of 1-2 weeks, the water conditions have changed so much from the "water of the tap" conditions that the fish are stressed by the addition of "clean" water, that says that there is the beginning of a buildup of harmful substances in the water. Now, simply because the fish are capable of adapting to these conditions does not mean that they are ideal or even suitable conditions for a fish to be living in. As others have mentioned, effects such as stunted growth or shortened lifespan may not be something you see within a week's, a month's, or even a year's time, but over the lifespan of the fish not doing water changes will more than likely have adverse effects.
Also, (and I apologize for the logical fallacy but I feel the point needs to be reiterated) if IACUC, the organization in the US that is charged with the responsibility of maintaining the health of laboratory animals, requires water changes in labs using fish, I have to believe that they are a "necessary evil," as you labeled them. If IACUC did not deem them necessary, they wouldn't require it, simple as that. IACUC makes the lives of researchers hard enough, and I can only imagine the time/expense that large fish labs must put into water changes (I work in a rodent lab so I see first hand how their rules affect our operating procedures). If it wasn't necessary for the safety of the animals they'd have a pile of angry scientists coming after them.