You have bigger problems than rearranging decorations I'm afraid.
Knight gobys are brackish water fish meaning they need a fair amount of salt in the water to survive. More salt than the other fish in your tank would care for. He may survive, possibly even a while, in your fresh water tank but he will probably have a shorter life expectancy and will not thrive. Also, he's likely to eat the neons if they are small enough to fit in his mouth. Its sadly common for brackish water fish to be mislabeled as fresh in a lot of shops. I'd bring him back.
Both corys and danios like company. I usually say no fewer than three corys in a tank. The danio is less critical in a tank with no aggressive fish and since a 10g is a small tank, I would probably leave him alone. For corys, think about a pair of panda corys. They stay fairly small.