+1 for Presto.
My music collection consists of about 2,000 CDs, 2,000 LPs and 1,000 downloads - it has been built over 50 years.
I do wish there were a term other than "classical", classical being, strictly, music coeval with Mozart, Haydn, early Beethoven, 1750-1820 are the usual dates. Western Art Music, perhaps, but that is a mouthful. My collection spans the period from Hildegard of Bingen (11th century) to Shostakovich (last work published in July 1975). To my ears most of the attempts since then are noise - sometimes explicitly so with tin plates lying on piano strings etc.
Bach, and other Baroque composers, are a good place to start, but, as I have aged, increasingly I listen to the Romantics (later Beethoven, Brahms, Rachmaninoff) and early 20th century.
To find works, and performances, that you like, have you considered streaming, from Presto for example? Presto sell an inexpensive streamer and the Bluesound Node is also Presto compatible. That way you can listen, compare, and add the CD to your permanent collection later.
As an example of Presto's thoroughness there are over 150 (!) versions of the Cello Suites (at least one of which is a MUST HAVE for a collection). Similarly, they have over 160 versions of Mahler's 2nd Symphony (at least one of which is a MUST HAVE for a collection)..
Since Bach is a through topic here it is interesting to listen to both "authentic, period" performances and those performed using modern practices and instruments. These comparisons show some of the richness of "classical" music, and how different artists find different musical expression in the same work.
For example, the Goldberg Variations. Listen to Angela Hewitt and Glenn Gould - the notes are the same but there the similarity ends. Then, add in Ralph Kirkpatrick (harpsichord).
For meticulously documented performances of Baroque & Early works DG's Archiv Produktion offerings are outstanding. I have many Archiv LPs.
For later works the reissues of RCA's Living Stereo recordings are famous. (BTW, many are available on SACD, if possible, get an SACD compatible player). The LPs are the fabled Shaded Dogs,
A subscription to Gramophone provides a fun and illuminating source.
Welcome to the world of "classical" music, I was brought into this as a 15 year old, 66 years ago, when the youth club in the small town where I lived would go to concerts at the Royal Albert Hall and the Royal Festival Hall in London.