Mahler...OK I'm hooked

You have given me some good advice and I purchased (used) the 1st (Abbado) and 2nd (Bernstein). Now I am ready for more. What is next.

Remember...the sound is important and it must be LP.

keep-em in numerical order (except for 4 - you can do that anytime). my recommendations for #3 on lp, Bernstein again with the NYP or Horenstein and the LSO (if you can ever find it). I also have and enjoy Tennstedt, (an excellent Mahler conductor with less than the greatest technical support) and the LPO on EMI for a more idiomatic performance. enjoy........
Dietrich Fischer-Diskau singing Kindertotenlieder on DG - easy to find used and, in my book, essential Mahler. Heartbreakingly beautiful/sad - Mahler lost at least one child and this was his response... Also Das Lied Von der Erde (female vocal with gorgeous orchestration), lots of versions on lp, easy to find used.
For the BEST 2nd ----> Rattle's version has won nearly every award. I would put it at or near the top as the best CD I own.
ditto to drrdiamond,rattle's version is top shelf for #2 mahler symphony.true masterpiece and great liner notes to boot.
The following symphonies are good performances with good sound quality - all on LP...

Symphony 3 - Horenstein/LondonSO, Nonesuch HB 73023 or Unicorn RHS 302/3 (as Newbee suggests)

Symphony 5 - Barbirolli/NewPhilOrch, EMI SLS 785 or Solti/ChicagoSO London CSA 2228

Symphony 6 - Horenstein (again)/StockholmSO, Nonesuch HB 73029 or Solti/ChicagoSO, London CSA 2227

Symphony 8 - Solti/ChicagoSO, Decca SET 534/5 (or London if you can't find the Decca at a reasonable price)

Symphony 9 - Solti/ChicagoSO, Decca SET 360/1 (or London if you can't find the Decca at a reasonable price)

Alternatives for the 1st and 2d you have, both of which will be very different interpretations (not necessarily better, just very different) but in much better sound quality:

Symphony 1 - Horenstein/LondonSO, Unicorn RHS 301 or Guilini/ChicagoSO EMI ASD 2722

Symphony 2 - Solti/LondonPO, London CSA 2217

And, for a change from orchestral, make sure to experience another side of Mahler in his "Five Ruckert Songs." I've always been fond of the Janet Baker performance with Sir John Barbirolli on EMI SLS 785.

Best of luck as you continue to explore Mahler.
For the 4th, I'm partial to the Klemperer on EMI (available on reissued vinyl as well as a good CD), as well as the Solti. For the 5th, if you go for the Solti, I'd probably steer clear of the Super Analog reissue if you have speakers with deep bass response, as it is quite bass-heavy. Glad to see you're hooked, Rick!
Bruno Walter's recording of the 9th with the Columbia Symphony is very good. It is a 2 LP set Y2-30308.

For the 4th, look for Szell conducting Cleveland with Judith Raskin on CBS MY-37225.

If you can handle CD, then the new recordings by Ben Zander and the Philharmonia of London are also a pleasure. They come with a bonus CD by Zander with a lecture analysing the music.
So many interpretations, so little vinyl.

Regarding the 4th, I got to know Mahler with the Bernstein recording on (then) Columbia. It's important to me for the concluding soprano solo to sound unmannered and innocent which is why I prefer Reri Grist. Later, Bernstein went over the top (who, Lenny?) and used a boy soprano on DGG but it proved to be a case (for me, at least) of more being less.
I own almost all of the performances and alternate pressings referred to above on vinyl and CD. I started in 1962 and purchased most of the Mahler recordings as they were released and usually was able to get European pressings.

For Solti, the Decca pressings almost always sounded better than the London, especially on the 8th. The 5th 6th and the 8th have very noticable spotlight miking. I actually prefer the Decca Legends CD for the 8th. The Horenstein Unicorns were excellent pressings. I would also highly recommend the Horenstein 8th CD from BBC Legends.

For Bruno Walter, the Odyssey pressings were poor and I prefer the remastered CDs. The original Columbia Bernstein pressings were highly variable and I would rate most of them as poor. Again I prefer the CD remasters.

I would agree that the Zander CD's have excellent sound and rank with the top performances.

I didn't really care for the Tennstedt because I thought the orchestra and the recorded sound wasn't really that good.

I would add Klemperer's 2 and 4, again on EMI and not Angel, although the EMI Great Recordings remasterings on CD are quite good.

There are a fair number of interesting historical Mahlerperformances once you really get hooked and don't care as much about the sound.
Speaking of 'hooked', have you ever been to the Mahlerfest in Colorado in January?