I visited HLS for 3 days back in September and loved it. Boston is a sweet town (as long as you don't mind the cold), and I was surprised how many chill people I met there. I just got accepted today (I can't sleep), and based on how enjoyable I thought the school, students, and setting were I'd pretty much made up my mind when I left boston.
Yes, there are people that are stuck up and annoying as hell - assholes are pervasive, but I don't think people are as academically/professionally insecure at places like Harvard, Stanford, & Yale as you might find at slightly less highly rated schools (which, I think, leads to a lot of that type of stuff).
Anyway, I'm 21 from near SF and go to UCLA, so my sensibilities when it comes to people might be different than yours, but I had exactly the same questions and fears before I checked it out myself - I hope that helps!