The Castro may have been Gay Heaven, but the rest of San Francisco was not. You might see an occasional hand-holding gay couple or rainbow flag, but mostly you were deluged by heterosexual power-couples and cooing Moms and Pops on holiday. Some neighborhoods were quite homophobic I have had slurs yelled out of passing cars at me only five times in my life: once in Maine, once in Texas, and three times in San Francisco.
So most gay people in San Francisco wanted to move to the Castro.
I had the opportunity once, in the fall of 1996, as long as I didn't mind having a slave boy.