OK, the age of the tank at 3 weeks tells me that you are probably still in the midst of establishing your "cycle" that is growing the bacteia necessary to process the fish wastes in the tank. If you have a water test kit, please tell us the ammonia, nitrites, and nitrates. Also pH of the tank, pH of fresh tap water, nitrates of the tap water. If you do not have a test kit, I use the Aquarium Pharmacuticals kit that is $11 online at PetSolutions.com and gives ammonia, nitrite, nitrate, pH, high range pH. Tetra kit is good also, those dip sticks are often in error.
Next, I need to know about your water changes. With a fully stocked tank, I'd suggest at a minimum 25%/week, 50%/week is much better and certainly while the fish are so unhappy will be a good idea, however you need to get there slowly if you have not done much water changing so far. If you have not done any water changes, then do a 10% water change, daily for 7 days then go to 50% water changes each week. Be sure to match water temperatures, and use the proper dechlorinator for chlorine or chloramine as your water dictates. If you are not sure, use one that removes chloramine, pay the few bucks extra to be certain. I like Prime, myself, for it handles chloramine and also helps with nitrites and free ammonia, which is very nice to have for your fish if the tank is cycling or power goes out for awhile and the filter is down. Follow directions on the bottle, and if the tank is having high ammonia, feel free to add enough to treat the entire tank, it will help the fish a lot to bind up the ammonia and nitrite that is bothering them now.
As for the brown algae, just leave the light off conpletely until you are there wanting to see the fish, assuming there is some light in the room normally. A dim tank is not a problem, you just won't see them too well. You may want to get a timer and put the light to come on at whatever hours you are around to enjoy seeing them. The brown algae will fade, or you can remove tank decor a bit at a time to scrub the algae off and then lay the decor in the sun to naturally bleach clean.
All the fish are sad, even aggressive, due to the likely poor water quality. The Prime and water changes will help this a whole lot. The algae is not affecting the fish so try to ignore it for now, it actually allows some micro critters to grow and this is what they fish are pecking at. If you scrape the glass of algae, do a water change right away to remove the floating bits as they will just start to grow again once they settle down. Best to hold a siphon in the palm of the hand that is scraping the algae if you can.
If you were going to get rid of any fish, I'd vote the platies out, for as live bearers they may overpopulate the small tank. Otherwise the rest are fine, even the shrimp.