What is the KH of your tank? Do you have live plants? Are you injecting CO2?
Check the 'true' pH and KH of your tap, I'm assuming that's the source of your new water.
Place a shallow bowl on the counter for a few hours/overnight then test pH and KH. Test if before too, it will help you see the difference.
pH will slowly creep downward as nitrifaction consumes bicarbonate, but if you're making frequent water changes, this shouldn't be an issue.
There are ways to increase pH, but I wouldn't want to recommend anything until I know more about your tank and the water that's going into it.