I know a guy from Vandy who is very nice, down to earth, and helpful.
I think Vandy may get the rep for undergrad because of its spot in the SEC. Isn't it the only private school in the conference? The state school kids are naturally going to amplify tales of the "spoiled little rich kids" in that situation. Add up all the student bodies, alumni, and fans from the other SEC schools and you have a lot of people potentially sharing one opinion.
I highly doubt it is any worse than any of the other schools in the top 25.