I purchased the first edition of Robert Harley's book about Hi-Fi. I was hoping to find the kind of info you want (how amps work, crossovers, etc.). His presentation was at a very high level, however. It would be nice to know how capacitors, resistors, global feedback, etc. fit into the picture. Try to find a book about building a simple radio receiver. This will give you some education.
Jimmy, I think what you are looking for is one of the books written back in the 1950’s, by guys like Harry F. Olsen. A technical book, with electronic theory and principles for aspiring audio engineers. Don’t bother with an "audiophile" book---writers of them speak in subjective language, not technical.
By the way, if you’re in the San Francisco Bay area, Roger Modjeski of Music Reference is teaching a class in amplifier design. Taking that class would be a great way to both learn audio engineering and build your own tube power amp. If I still lived in my hometown of San Jose, I’d do it myself.
Douglas Self on amplifier design: http://www.douglas-self.com/ampins/ampins.htm
Floyd Toole on speakers and room acoustics: https://www.amazon.com/Sound-Reproduction-Psychoacoustics-Loudspeakers-Engineering/dp/0240520092
The bible is here and has never been equaled IMHO: Audio Cyclopedia by Howard M. Tremaine, 1969. A little expensive (~$120) and huge at 1,700 pages, but it has EVERYTHING including audio designs and history. Also found on eBay.