I'd definitely recommend cleaning it. The pool filter sand I got was quite dirty, but it was a pain to clean since it's so small. Not sure I can even describe how I managed it hehe
If you can find a strainer that the sand won't fit through, that would be easiest. I'm going to look for one next time I go to the store.
Even after cleaning it best I could without much to work with, my water was cloudy for a few days. I can imagine if I had used it without cleaning it would have been a real mess.