Once you add fish, waste and such will build up on the gravel dust and your tank will clear up FAST. I had the same issue with my gravel. Rinsed the heck out of it but it was still cloudy. Once i added fish, it was crystal clear in a few days.
I have found when adding new subtrate that is somewhat cloudy (sand) that if you use a python, when removing the water close the valve and make sure that water remains in the vacuum. reverse water flow and slowly let off of the valve to restrict waterflow to a minimum when filling. If there is sediment that needs to settle changing water in large quantities can sometimes stir it back up. I know they are two different animals but maybe that will help.