pH challenges
The thing about trying to change pH is that it always bounces back. It is the sudden swing that does great damage to fish. The pH scale is like the earthquake scale, 7.0 is 10 times worse that 6.0 on the Richter scale. Same with acidity/alkalinity. The chemical solutions that the store may sell you will only be a temporary fix, then the pH will bounce back.
If you must alter pH, do it by mixing in RO or distilled water. Know that distilled water has no oxygen in it, so pouring it in is bad, you should aerate it first, run an airpump and airstone for an hour or so.
First question, what is the pH of tap water that has set out overnight?
It might not be the tap water, but something in your tank that is making the pH so high. Fish stores sell sand and gravel that is meant for saltwater tanks that has shell and things that are meant to make the tank water near 8.2 pH which is great for satwater, not for freshwater unless you have chiclids.