Actually, Cardinals/Neons can live about 10 years. Danios are also a long living fish. On the other side of the coin, there are also some rather large fish which dont live for that long.
I'm not sure its a matter of the size of the fish, as much as it is the genetics of it all. There is definately science behind things, and maybe someone can give a better explaination.