I think if you don't want to go to the firm, you should cancel. Otherwise, you are wasting everyone's time. While the firm may have to eat the plane ticket, it is worse if you are wasting others time. At my school, our career services people tell us to cancel if we are no longer interested, even if a plane ticket is involved.
However, I guess there might have been something about the firm/city that let you book the trip in the first place. Maybe if you go, you will end up liking it? I think if you do end up going, you should keep an open mind.