New World and Pacific are probably your best bets in the city for plants and oddballs, though they're probably the two most expensive places you'll find.
For the most common of plants (java fern, java moss, anacharis, etc), I sometimes goto Petco on Union Square.