Do the golf thing first, see if you can land a teaching pro job. They're tough to get, but if you're willing to re-locate you can find one. It's a competitive field and they often take guys who are minor pros or played in college. But some guys haven't played in college, you need to be good at golf, however, real good not a hack, obviously. I'm guessing you're pretty good.
Always have a backup plan in life. I changed careers in my late 30's, I now work at a college after many years as a journalist. I got burned out on the former, enjoy the latter. Things change, that's why I say have a backup plan.
You don't have to do college right now, tons of guys and gals go back at later ages. Today I just met with a woman at 46 who is going back to school. So be it, better than doing something you don't like. And if college isn't for you, you can go to trade school or get an associate's at a community college, or build another career.
Just always have a backup plan in life, be it in career, dating/women, everything, because there are no guarantees. If golf doesn't work out, just have a backup plan. I've put all my eggs into certain baskets in the past and you know what? I wasn't rewarded for it.
Hit a hole-in-one! Go for the golf and in your off-time research possibly backup plans, be it college or another career in case things don't work out. In American society we're told to obsess about ONE thing, ONE career, ONE woman ... when things don't work out, we're left out in the cold. Always have options.