Probably the best jazz record shop in ny is the "Jazz record Center". It is at 236th west 26th St. Most of what they have there is not new vinyl though.
Jaybo - I don't like the fact that the SF store moved the Jazz section way to the back corner. Kind of makes it inconvenient. Once, I went to all 3 stores in a weekend (doable since I live in SoCal), the Berkeley store had the best Jazz, the SF best rock, and the LA store was my least favorite. I was just at the SF store a couple weeks ago and for the first time since I can remember, I walked out witout any vinyl.