There are two general ways to deal with this.
Daffy pretty much covered the first.
I would only add that you might want to put some time into finding clubs, classes, or social groups that are a good fit for you. There are probably quite a few on campus. Chess clubs, book reading clubs (at least they're literate, right

), and wine tastings will be a good place to start, but ultimately you will want to play to your own tastes.
The other thing you can do... is try to look a little deeper into the conversations you're having now. Ask these football guys what it is they love about sports so much. Ask people what makes them so interested in the things they seemed to be obsessed with.
Why are you so interested in having intellectual conversation?
What do you find most interesting about it?
What does that do for you?
I would love to hear your answers, and I also present these questions because they are the kinds of questions you can ask anyone about virtually anything (filling in the topic as appropriate)... and if you are a curious person (most intellectuals are), then you can have a very interesting conversation even with a 'stupid' person or an 'immature' person.
Or you can always just have a bit of drunken sex yourself... and see what everybody's raving about...
keep smiling,
Ben