Amen bro. I get the same thing. People hear me and instantly think Rocky and beg me to do "ADRIENNE!!" impressions. To which I reply "I'm not your freakin dancing monkey, alright!" before I cave and eventually comply.
During most of the day or when I'm excited or something my voice gets a little higher. But at night when I'm tired... or when talking to girls especially... I seriously turn it on and my vocal chords turn into a virtual subwoofer. It's weird if I tried to do it right now it wont sound as good. It just sounds like I'm
trying to make my voice deep. But around a girls it just clicks.
As far as your problem though, I get the same thing so I can't give you the perfect answer. Nothing is worse then saying a funny line and then having to repeat it becuase they didn't hear. Like you said louder and clearer would help. But try speaking slower. Mystery preaches that speaking slower emits a sense of mystery, but it would also give your listener a longer chance to hear and understand what you're saying.