Thanks for all the feedback guys. I really feel I've made the right call to turn down the offer. I wouldn't want to go unless it was just crazy to turn it down, and it's just not.
Not with me thinking about retirement, and the house I own here, and the wife having to find new job.
And I love it here. And yes the no income tax is great, even if I pay it in other ways ($2750 property tax on 1800 SF house) and 6% general sales tax.
So, my plan is to pursue consulting with the Cal school and others around here as a way to supplement the income, and ease my way into retirement.