I also recommend a 10g tank. It can be much easier to start a 10g tank than a smaller one (more margin for error). You can use just about any type of filter, HOB, sponge, etc. The only precaution is to cover the intake so that juvenile shrimp are not sucked in. If noise is an issue, I would go with a HOB, and keep the tank water level even with the filter outlet. Sponge filters require an air pump, which can be silenced, but it takes more work to do it.
As for shrimp, you can start with about anything cheep. Ghost shrimp are commonly sold for less than 50 cents each. Triops can also do the job, eggs are very reasonable. However, if you mix triops with shrimp, they can hunt and kill shrimp, depending on size and species. I also recommend that you look at optimal water parameters for your 'target' setup. Different species of shrimp prefer softer water, while others like hard water. There are plenty of comparability charts, and other helpful tips in sticky threads here.
Good luck