Water changes will do no harm once you start doing them regularly. If you have not been doing regular water changes, you need to work your way into them gradually. Your fish may have become accustomed to high nitrates and the pH that results from no water changes. The solids content of your water may have also gone up quite a bit by you just adding water instead of making changes. If you go out and make a large change under those circumstances the result will be lots of stressed fish.
Like I said earlier, if you have not been doing frequent water changes, take it easy at first and gradually work up to the larger amounts. You might try 10% per week for a few weeks then move up to 20% and so on. Thaty way you will improve the water quality gradually and not shock fish that have had more stagnant water conditions.