I didn't own, but i did fool around with my friend's 3g. I finally got myself a 3gs and I'm very happy with it.
Not sure if it's worth upgrading, but it is significantly faster than the 3G... Camera works nicely, video option is great... And I love the new compass feature. Seems like a small thing, but it opens up many doors on developers to make programs that work with the compass.
One thing I'm not happy about is the new screen... It isn't as "antiscratch" ad the reg. 3g phone is .. I'm pretty sure it has something to do with that oil resistant thing they put on the screen. I never dropped the phone, but the other day I noticed a couple of small scratches on the screen. I was very surprised, as I haven't seen this on the 3G iphone.