That's pretty much my understanding of the case, too. Admittedly, I haven't actually read the full opinion so I could be wrong.
It's one of those cases where I really can see both sides of the argument, and they both have merit. One the one hand, there is always going to be disparity in economic influence. I mean, Bill Gates and the Koch brothers are going to be able to promote their causes more effectively than most people because they're rich.
Is that fair? Probably not, but that doesn't mean it's unconstitutional.
On the other hand, there is a real danger of becoming a corrupt oligarchy where the government is simply a bought and paid for instrument of the rich. Like the old Roman Senate, a bunch of elites pretending to care about the common people.
Interestingly, this particular phenomenon crosses party lines. I read an article recently about how this is the wealthiest Congress in history. I believe both of my Senators (Feinstein and Boxer) are worth hundreds of millions. Darrell Issa is worth something like $600 million.
I think it's already at the point where you can't really win without huge personal wealth or the support of SuperPACS. It may not be unconstitutional, but I think the long term effects are probably negative.