In my experience, it takes a long time (months) before baby mollies show any external sexual differentiation, so I don't know if you waited long enough. With my two large batches of babies (I still have never actually counted them, there were/are so many) I'd guess around 30 each time) there appear to be about the same number of males to females, maybe slightly less males. When their anal fin changes enough so I'm sure they're going to be a male, I move them to the male tank; usually they aren't really acting "male" yet (i.e. they're not yet chasing the females).
I have another sex ratio question: what proportion of female to male mollies do people think it's best to keep together? I know 1:1 is bad, because the male harasses the female too much.
Thanks! Karen in Ithaca, NY