Next on the list for Mahler....

Be careful with Bernstein and the NY Phil. Lenny's interpretation can't quite overcome the sloppy technique and bad intonation.

Check out Bernstein's recordings with the Vienna Philharmonic on DG. Symph #5 with Songs of a Wayfarer would be a wonderful place to start.

All of Mahler's symphonies have something wonderful to offer. Taking the time to explore them all will be very rewarding. 2, 3, 4, and 5 are my favs.

Also find the vinyl Mahler 7 with Otto Klemperer and the Philharmonia Orchestra on British EMI (i.e., don't get the US Angel or German Electrola vinyl pressings). The CD also sounds quite good.

While this may be Mahler's least popular symphony, it is a masterpiece. Klemperer's way is quirky and eccentric, but my favorite. Listen to it several times and you won't want to play the versions of other conductors. He brings out the beauty, angst and mystery of the first movement like no other. The "nactmusik" movements do indeed envoke the mystery and enchantment of the night. Other conductors tend to gloss over this music with too fast tempos, as if they can't quite figure out what Mahler was trying to say.

The sonics are simply knock out quality and come closer to a live concert hall experience than most classical recordings.

The orchestra is amazing in its virtuosity and tonal beauty. In its heyday, it was in a small handful of the world's finest orchestras.

Can't recommend highly enough.

I’ve been a Mahler buff for 40 years, have many Mahler recordings and have heard many Mahler live performances, so I’ll take a shot at your question. You’ve already got the First and Second, so we go from there. The most popular Mahler symphony, and the most accessible one, is the Fourth. It’s also the gentlest, most lyrical, and least bombastic, perhaps the most tuneful, and certainly very beautiful. (The First and Fourth are the shortest ones, and the only ones that could be gotten onto a single LP.) So I’d recommend the Fourth as your next acquisition. (The First, Second, and Fourth are normally recommended as the best introduction to Mahler.) Next I’d recommend the Ninth, and after that the Fifth. Beyond those outstanding symphonies, I’d say all the Mahler symphonies have their own beauties and power—with the exception, for me, of Seven and Eight, which I’ve always found the weakest, least inspired of the nine completed symphonies (the Eighth in particular to my ears seems to be trying to make up in elephantine scale and bombast what it lacks in true inspiration, but that’s just one man’s opinion).

I’m a CD guy so can’t comment on the best versions available in vinyl today, but two superb analog recordings of the Fourth that were issued on vinyl are the Szell/Cleveland from 1965, a classic, and the Solti/Concertgebouw from 1961 (not his digital remake with Chicago). The Szell/Cleveland always had considerably better sound than the norm from Columbia in the 1960’s, and the Solti/Concertgebouw was engineered by the great Kenneth Wilkinson and has wonderful sound; if you can find it, it’s a gem. I will simply add that the best-sounding Mahler symphony recording I’ve ever heard from an audiophile point of view is the Chailly/Concertgebouw performance of the Ninth on Decca, recorded in the Concertgebouw in 2004; stunning sound and also a magnificent performance. The Das Lied von der Erde is also indispensable; it and the Ninth are Mahler’s indisputable late, crowning masterpieces. The greatest recording of it is the Bruno Walter/Vienna Philharmonic/Kathleen Ferrier version from 1952 (monaural), unlikely to be surpassed as a performance, but inevitably showing its age sonically. (It’s been successfully remastered in Decca’s “Legends” series of remastered CDs.) Happy listening.

I agree with Texasdave that the 4th Symphony is a good next step. Along with the Szell he recommends, I suggest trying to find the Paul Kletzki/Philharmonia Orchestra version from 1958, last seen on LP on the Seraphim label. I used to own that disk and it sounded pretty good sonically (not as good as the Szell though), but it's gone now. Now I have it on CD (Royal) coupled with Mahler's 1st. Musically, they are both classic performances, but the 4th in particular is truly outstanding.

I would add on the list the cycle of songs Des Knaben Wunderhorn. I think, one of the best version is Szell, Schwarzkopf, Diskau and Cleveland SO. This cycle of songs is a key to understand motives of Mahler, and helps to understand also its first symphonies. Also a great introduction as i has a lighter texture than the symphonies.