Definately go to Rooky Ricardos 448 Haight St (between Fillmore and Webster) they are always getting new stock in, although they know prices on the known stuff. Ask to look behind the counter at the slightly more exspensive stuff (from 5 bucks and up) also if Dicks in the store he keeps a "good box" of stuff thats not been sold on the bay.......you may get lucky.
Unfortunatly Glynn the nice old lady behind the counter passed away a couple of years ago.
Amoeba Records is at the other end of Haight St near the park and is also worth a look lots of 45s from a couple of bucks
Theres also an LP shop just accross the road from Rookys (the name of which escapes me) specialising in soul/funk/jazz.
Chris if the shop you mean was in Mill Valley I think its closed now
You can catch the BART(underground) to Berkley (& Oakland) where there is another Amoeba and a shop called Rasputins in the same street, I found the Amoeba shop there was a lot cheaper for LPs than the one in San Fran
Near China town on Green St and Columbus is a store that sells instruments and audio eqiupment that also has a basement full of vinyl, again they're always getting stuff in so you may get lucky.
Good luck
Cheers Paul