HDCD rescue - it's possible, but man...


I spent the better part of today sick, but working on finding a way to decode music I might have that is HDCD encoded.

As a refresher, HDCD was an enhanced CD format. In some ways the predecessor to MQA. HDCD was an engineer's toolbox, allowing the mastering engineer to select a number of features. This would then be decoded by a matching chip on a CD player or DAC. The most famous feature of HDCD was bit-compression. Getting a 24 bit signal encoded in a 16 bit music file.

As an aside, the Pacifics Microsonics AD converters were highly prized by engineers for their sound quality. Anyway, the format got bought by Microsoft and died.

Of the 670 CD's I have ripped only about 11 were HDCD encoded. But man, what a pain. I ripped everything to FLAC, but the HDCD decoder only does WAV. I had to download source, compile it, then write a script to go through every CD and decide if it's HDCD or not. Once found, I have to convert from FLAC (44/16) to WAV, decode the WAV file (now 24 bits) and convert back to FLAC to compress again.

The discovery process was pretty fast.  About 10 minutes to go through them all by cheating. :) More time was spent figuring out how to pass apostrophe's in file names than finding the files.  Nathalie Merchant was one author who consistently used HDCD by the way.
erik_squires
Erik, it's my understanding that the HDCD process squeezes 20 bits into a 16 bit WAV file for CDs.  I use dbPoweramp and it has an HDCD decoder built in.  It's (somewhat incorrectly) listed under the DSP effects section.  I've used it on my HDCD-encoded CDs and it seems to work well (the DAC identifies them as 24/44 files and they sound fine.  There's a check box in the decoder to apply "+6dB Amplification (align 20 bit result in upper 24 bit range)" and if you don't use that the volume comes out way too low (6 dB!).
Dick
Hi @djohnson54

I think you are right, it was only 4 bits more. However, since that's an odd bit size, it gets spit out as a 44/24 file.

Your approach is cool, but since I am on Linux only and don't have the MS approved HDCD decoder, I can't use it. Instead I convert all of my files, but I only have to do this one time. I think OS X people would have the same issues.

Thanks for the heads up on the level shifting. I'll have to compare a few files first. This is something quite easy to do with ffmpeg.

Best,


Erik
Sounds like a project for retirement or to while away the time during a long prison sentence.  Can you even enjoy listening to the music after so much effort?
@erik_squires Haha.  I should have guessed that when you mentioned downloading source and compiling.  I love Linux for server-side stuff but I'm past the point in my life where I want to figure out stuff like you just did because I don't have access to mainstream programs.  Don't get me wrong, I would love to tell M$ where to go (Comcast too!) but it's all about the applications for me at this point.  There was a time though ...
@mahler123

Well, there is more to this than just converting music files. I have been bed ridden on and off for a while, so brushing up my scripting skills is a plus. :)

Like a lot of things, the setup time is the most brutal part of this effort.

It turns out I don't need to level shift. I do however need to copy the metadata and cover art separately as that does not make it through the HDCD decoding.

I'm actually just really happy I found OS HDCD code to let me do this. Mostly because I've wanted to listen to HDCD since it came out, and this is the first time I can.

Best,


Erik
I remember when HDCD first came out. I was ready to get a new cd player and found one that decoded HDCD. When comparing the same non encoded cd with the HDCD encoded cd I found no difference in sound. I have heard MQA at home and I hear a difference. Just my opinion
Alan
Erik-

I want to buy a hi-rez download(FLAC). How do I go about the download, storing the music and burning to a CD-r?  I am not versed in the whole computer audio/hi -rez downloading thing?  The particular artist does not offer her music on physical media (CD/SACD).  Thank You for any assist.

You can PM me if as needed or post info here.
Happy Listening!
Hey @jafant

Start a new thread in the digital section. :)

The quick answer is, you use a laptop or PC as a music server, and use USB or S/PDIF output to your DAC. This is most convenient.

Of course, you can also burn a CD-R but I don't ever do that so I can't help you

Best,

Erik
There are many cd's that aren't labeled HDCD that are HDCD also, google HDCD list and you'll find some large list available.

Cheers George 
djohnson54 - Hat tip to you, man! I had no idea that dBPoweramp could do that! Looks like I will be re-ripping my HDCDs. I don’t have that many of them, so not a big deal for me.
@bondmanp You're welcome.  It took me a while to discover it too so I have several that also need to be re-ripped.  Actually, I'm surprised at how many I actually have.  Use George's suggestion above to Google a list and compare to your library!
Oppo 103 also has HDCD playback! Who knew??

I keep forgetting. I didn't think anything this new still had HDCD built in.
Oppo 103 also has HDCD playback! Who knew??
That's interesting.  It used to be quite expensive to license the HDCD encoder from Pacific Microsonics.  If that's still the case, it's even more impressive that it shows up in a relatively inexpensive piece of equipment.
HDCD was developed back in the 1990's. It is probably not expensive to license any longer (statue of limitations).
One of the reasons that I still keep my old EAD 7000Mk3 DAC is because it decodes HDCD. To my ears, HDCD is a very nice and easily heard improvement over the standard Redbook.
I actually believe that HDCD can still compete today with some of the higher rez formats. IMO, HDCD has a certain 'analog' sound to it that seems to do away with some of the digital nasties, that are to my ears always present in digital reproduction. 
@daveyf Yep, I am really liking my transcoded tracks, but I liked them a lot before. PM did nice things. :)

Yeah, the HDCD "license" holders actually had to buy little DIP chips with HDCD printed on them. I'm sure it's all in software now.
PCM HDCD and DXD/DVD-A rules when converted through a well implemented R2R Multibit converter with either PMD100 or the last to my ears better PMD200 HDCD chip.
Especially the 24bit pcm hdcd recordings from "Reference Recordings" done by the hdcd master Keith Johnson. Far better than any DSD crap, unless it’s native then it’s good, but sadly there’s no good music/artists for me on native dsd.

Cheers George
Wow, you packed a lot in that first sentence, @georgelofi Not sure what you meant.

AFAIK, the HDCD chip, was purely digital domain. The DAC implementation was left up to the licensees.

Interesting about Reference Recordings. I wonder if they are using any of the bit-compression or just the transient filters.

As far as I knew, there were no longer any HDCD ADC's available, so you had to use vintage gear, although I soppose you could do it purely in software, but then what's the point??
AFAIK, the HDCD chip, was purely digital domain.
Yes I know, I never said they were in the analogue domain, they were magnificent, especially when partnered with with well implemented Multibit d/a converters (I/V stages and buffers that weren’t just text book installed opamps) as many are.

Cheers George
So I've poked around Reference Recordings. I've not heard them, so I have no idea what they sound like, but I must say that their site is VERY light on technology. Big on adjectives.

Again, it may be fantastic, but I personally would love to understand which set of HDCD features they are taking advantage of.

Best,

Erik
Professor Keith Johnson who was the design man behind HDCD in 1996 also founded Pacific Microsonics Developments (PMD). And is now also the Technical Director, Recording Engineer and partner in Reference Recordings.

http://www.goodwinshighend.com/music/hdcd/hdcd_development.htm

http://referencerecordings.com/about.asp

http://www.stereophile.com/content/keith-o-johnson-reference-recordings#sbudxxeHz73RTH26.97

Cheers George
I've never heard one, but the owners "say" the Pacific Microsonics Model Two A-D/D-A Converter is the end game for listening to pcm with. And worth a small fortune s/h.
https://forum.audiogon.com/discussions/Pacific%20Microsonics%20Model%20Two%20A-D/D-A%20Converter

 Cheers George
I'm not questioning HDCD. I like it. I just was hoping they'd have something more specific about which features they used for RR.
No idea, he is very contactable.
All I know with my Linn CD12 which has 4 x PCM1704's and PMD200 his 24bit HDCD Reference Recordings rule.
The only time I heard it bettered was with the sound track from DVD-A, I believe the same can be got from the new DXD as well.  

Cheers George 
Thanks George!

When I scanned all of my library I assumed only 44/16 would be HDCD encoded. Now maybe I will look through my hi rez files too.

Best,

Erik
As I understand it, Microsoft bought the rights to HDCD and they elected to not do anything with it other than have decoding in Windows Media Center. As far as I know, they still own it.

I am pretty sure that all the current implementations are just the software implementations that dBpoweramp, foobar and others use. That was a reverse engineering project by an individual who posted it on doom9 forum and was discussed in detail on the hydrogenaudio forum. foobar uses that code to decode HDCD as does dBpoweramp. JRiver looked at implementing it but never did.

The software does the bit compression part but I do not think it implement the various filters. At least, that is my understanding. Not sure I could point to any documentation on that however. It has been a while since I looked for the source code. Pretty sure it is out there somewhere. I used dBpoweramp to rip my HDCDs.

There is a list of HDCD recordings on headfi and that also has a pointed to musicbenz.

http://www.head-fi.org/t/65414/hdcd-list

http://musicbrainz.org/search?query=format%3Ahdcd&type=release&limit=25&method=advanced


Hi @dtc

I found a very complete discussion of it here:

http://wiki.hydrogenaud.io/index.php?title=High_Definition_Compatible_Digital#Regarding_the_Playback...

According to that, the transient filters were part of the specification, but were never licenced and therefore never implemented.

So this leaves dynamic range expansion, a-la dbx, kind of. So it makes me wonder what good the RR HDCD encoding is doing? Expanding 24 bits to 28??

Best,

Erik
Following on the link, above, I found a very interesting section. It is true that PM equipment was pretty popular in recording and mastering studios. I quoteth the results:

Much of the time, the engineers turned all HDCD features off, but the HDCD control packets were still inserted into the discs anyway. For these discs, there is absolutely no benefit in decoding HDCD.

This explains why so many disks may be identified as HDCD even if not labeled as such. Chances are good the mastering engineers had no idea it was on, and therefore did not utilize any of it’s features. Meaning, HDCD is irrelevant for those disks. Though I imagine there’s probably a byte or two at the start of each track which gets the hidden HDCD markers removed. Otherwise they might as well be RedBook.

It's also interesting how they describe it as a scam, originally marketed with grandiose claims and very little information. I can think of at least one digital format that seems like that. Grandiose, unsubstantiated (to my ears) claims, but a ton of data about it. Not all scams are lacking in data. Some like the perpetual dark energy scams have tons of explanations. They're just not true.

Erik
Erik - You have to be a little careful of the hydrogenaudio comments. Many on there are so caught up on their methodologies and rules that I wonder sometimes whether they actual listen to music. I only have a few HDCD, but they are very good quality and I do believe the HDCD decoding improves the sound.

As to  disks that are not labeled as HDCD - no idea. Many of them may be a mistake, but it seems like an honest mistake since they were never labeled to be HDCD.  I just know the labelled ones I have I like.

RR is holding onto an old technology. WIth high rez available (including 24/44), I see no reason to continue with HDCD given the lack of proper decoders and the cheapness of disk space.
To me a HDCD disc, decoded using PMD100 or PMD200, far outperforms what MQA can do with the same disc without HDCD.

Cheers George
I sat through and heard a demo with and without MQA and all the spiel that goes with it.

" Neuroimaging tests show activity in different parts of the brain; transients trigger activity in Wernicke’s Area (in the boundary between the parietal and temporal lobe), where melody and timbre are associated with activity in the right frontal hemisphere of the brain known as Broca’s Area." ect, ect, ect.

All levels were checked, I couldn’t hear anything different all with or without MQA.
But I can with and without hdcd with levels checked, as I had a Cary 303/200 cdp that had the PMD200 or your switch on the fly via the remote to a DF1704. And just to prove another way I have a couple pairs of the same CD same but with and without HDCD, don’t ask me how they do that??
One of them being
Patricia Barba Nightclub
HDCD 90763 G
NON HDCD 7243 5 27290 2 9

Cheers George
Yeah, MQA reminds me of dark energy, a lot. :)

In addition to the unproven sound quality enhancements, I really like the data compression part of MQA. If I could losslessly compress my flac files to 1/8th the current size I would love it. However, it's not lossless. No remaining value.

Best,


Erik
Erik your technical knowledge is enviable. Do you also consider ways of improving the sound of various components. 
georgelofi

it appears to be a "gold" CD. These always fetch a little more at auction.
it appears to be a "gold" CD. These always fetch a little more at auction.
Yes it is gold.
What perplexes me is that both the HDCD version and the non HDCD version came from the same one off recording session.

Not that I'm a recording engineer, but they (the studio) must have done this recording master using two different methods, HDCD and Non HDCD??? Is this correct???

Cheers George   
@dtc

I'm sure the mistakes were honest. :) I only meant to point out that many think they've discovered unmarked HDCD disks (like a bonus track on an album) when in fact they are plain-Jane Redbook CD's. This is probalby why they don't have HDCD branded on the covers.

Best,

Erik
Hi @ptss

Well thank you kindly. I was very fortunate to be able to observe a true master, Dr. Marshall Leach Jr. when I was very young. That exposure sparked a life-long interest. My humble memorial to him is here. Later on I was able to work in circuit board and industrial design for motion picture sound systems, before digital sound for film existed.

While I have an interest in many things, my main focus right now is achieving high-end movie and music sound for apartment living. Meaning small and affordable but not quality limited.

Best,


Erik
I agree -George,
different mastering (codex) methods for the HDCD moniker.
I find a more interesting aspect, in that, there are HDCD discs that are not "branded" as such.

In the early 1990's before the big "remastering" boom in 1994 and forward to this day, there are plain CDs of various titles/genres that were "remastered" and not branded as such.  Weird or testing the waters by studios and record companies?
I find a more interesting aspect, in that, there are HDCD discs that are not "branded" as such.
Maybe the cover/label designer wasn’t told it was hdcd so it wasn't printed??

Cheers George
All the evidence I see so far points to:

- PM gear in mastering process turns HDCD flag on.
- Engineers don't use any features
- CD's get mastered with HDCD "data tag" but no features used.
- Label gets printed without HDCD because it's not real.
- Some one on the internet starts making list of all HDCD disks, whether real or just tagged.
- CD's get mastered with HDCD "data tag" but no features used.
- Label gets printed without HDCD because it's not real.
PB Nightclub, hdcd sounds far better than non hdcd, same session.
Don't know what I'm getting at here.
Cheers George
It’s quite possible some real HDCD disks also never did the labelling. :)

I could check my recordings, I suppose, but meh, it’s all automated.

That Patricia Barber CD with the HDCD label really IS HDCD though. :)

Sorry I think there’s some misunderstanding, let me be as verbose as I am not sober. :)

So, HDCD has a special data tag that digitally marks a CD track as being HDCD enabled BUT, that tag could get added unintentionally, without the engineers actually using any of the HDCD features, in which case it’s no different musically than plain old redbook.

HDCD requires manual intervention during the mastering, there's no "set and forget" about it.