Location does count. Make sure you know how it counts in order of importance.
We looked at 4 or 5 B&B's in this town before we decided on the one we chose. The reasons we liked it better - owners' apartment was not cramped, ocean view, house style, bargain price (cuz it needed lots of love.)
I actually would have preferred a location closer to the center of town, but the owners' apartment and view cancelled out proximity.
For us, the living space is very important. I've seen innkeepers live in a bedroom or even a basement for the summer. We were not willing to do that. Just something to take into consideration.
btw - in our town right now there are two inns for sale. The more expensive one is the ocean views. The less expensive is the close to town. Both are beautiful and will likely sell as single family homes or condo conversions. Which makes me sad..