during your water change if you siphon the water from your gravel you should be able to suck up a majority of the poop and uneaten food without stiring it up in the water.
Also when you replace your water you should alway treat the water ahead of time or as you fill the tank. Finally - you ask if the water could have been too cold -i always match the temperature of the replacement water with that of the tank - its not good to dump in cold water - especially if doing a 40% water change - you could drop the temperature of the tank 10-20 degrees F and this is not a good thing.
Also when you replace your water you should alway treat the water ahead of time or as you fill the tank. Finally - you ask if the water could have been too cold -i always match the temperature of the replacement water with that of the tank - its not good to dump in cold water - especially if doing a 40% water change - you could drop the temperature of the tank 10-20 degrees F and this is not a good thing.