I have always understood that most of your bacteria will be in your substrate and decorations. It would make sense since if it were mostly on your filter, wouldnt you send ur tank into a mini-cycle when you change it?
Now you have changed your subtrate and decorations, thats fine. Jumpstart the "colonization" process by either keeping your existing filter in.
Heres an article i found in regards to biological filtration.
Biological filtration
Biological filtration is the term for fostering ammonia-neutralizing
bacteria growth. It is so important to the health of your aquarium
that we should look at how this process works more closely. (There are
other types of wastes that can cause problems, but the regular partial
water changes needed to control nitrates are typically enough to
control other forms of waste as well.)
Mother Nature provides several types of bacteria that break down
ammonia into progressively less toxic compounds, nitrite and nitrate.
These bacteria are not harmful and are quite abundant in nature. They
are so common that we do not need to add them to our aquariums; nature
does it for us.
In the presence of ammonia and oxygen these bacteria will naturally
multiply. The bacteria attach to the tank, rocks, gravel, and even
tank decorations. Note that we have not yet said anything about a
physical filter. This is because biofiltration bacteria require only
1. A surface upon which to attach,
2. ammonia for food, and
3. oxygen-rich water.
This sounds so simple; why do we need a physical filter?
Actually, if you limit the amount of fish to what the natural
biofiltration can handle, you do not need a physical filter.
Unfortunately, you cannot support very many fish with only the natural
biofiltration.
In the last few decades, the hobby has seen many new types of
biological filters invented which can vastly increase the capacity of
the bacteria colony to provide biological filtration to your aquarium.
In essence, all of these types of filters provide additional surface
area for bacteria attachment and increase the available oxygen
dissolved in the water.