The caudal peduncle (the bit the tail fin's attached to) tends to be wider in mollies; they "taper" less than platys do. The gravid spot in the females tends to be more prominent as well. Of course, the males often (if they have latipinna genes) have sailfins quite different from the hi-fin which a platy may sport. Extended fins in females are nearly always platys.
End on, mollies can be seend to be rather more barrelshaped in cross-section. The "platy" part of Platypoecilis, the old genus name for the platy (before it was found to be too closely related to the swordtails to be given its own genus), means "flat", and refers to the flat sides of the fish, which is therefore far more oval or elliptical in cross-section.