It doesn't matter if you don't smoke, this does effect you. Especially when you consider that the lower the income and education level, the lower the likelihood of a person having health insurance...and the higher the likelihood of them being a smoker. I pay health insurance premiums, rather high ones, to subsidize the costs of those smokers who end up in the hospital with illnesses related to smoking.
So, while I am also uncomfortable with government forcing the issue, I can see the public good in a smoking ban. The spillover costs of all that smoking are coming right out of my pocket.