As said, the effect on your water depends on your current water conditions; very soft or acidic water will be strongly affected, while moderately hard, neutral or basic water will be less affected. It also depends on the limestone. For example, Indiana limestone is much less soluble than Tennessee limestone. You can try a bucket test to see what it does to your water.
What's in the tank now, besides plants? Most plants do fine with hard water, and many fish do as well; they just don't appreciate sudden changes. I keep limestone in many of my tanks with no issues.