Thats the way it goes in nature, the stronger get more food.
I dont know if this is the case with swordtails, but if I dont keep my largest dwarf gourami fry well fed, they will turn and actually eat the smaller fry in the tank with them. Its a rough world!