What does it mean when you don't have trouble hearing, but you have trouble understanding what people say?

I've noticed over the years that I don't necessarily have a problem hearing, but understanding what people say when they talk. I often have to have subtitles when watching movies, have to have lyrics in front of me to understand what people are singing in songs, and I absolutely hate talking to people on the phone because I keep having to have people repeat everything they say because I can't understand them. I don't know if it's them or if I have a hearing problem, but it's starting to bother me, especially since I've started talking to a new guy who seems to prefer talking to me on the phone rather than on IM, which would be preferable since I can read what he's saying and understand it -- no repetition necessary. I don't know what to do -- should I see a doctor?