Hendrix,
I did the exact same thing in the beginning, just swished and then scooped the water out with the tube using it like a long cup. Then at the LFS they did a demo for me and it was great! The gravel vac really does such like a vacuum and boy is it fast! I filled up a pail real quick.
Here's what I learned.
1. Make sure to the tube as it comes out of the tank stays close to the tank, not looped high up over the edge. (hard to explain)
2. Tape or clip the end tube into the bucket. I think it helps if there is some water (from the tank--not from the sink unless dechlorinated) in the bottom of the bucket. Place the end of the tube in the water. They did not tell me to do this at the store but it's something I noticed. Make sure the bucket rests on the floor and not on a chair or table.
3. Move the tube up and down rapidly. It may take several attempts but it will suck. When you move it up and down don't move it side to side. Try to move it up and down only and right above gravel level.
Seriously, ask how they do it at the fish store. If you go near closing time they may be doing it anyway and you can watch and learn.