When using a DAC does the cd player really matter?

Is there a difference between a relatively inexpensive cd player versus one that costs much more. All you are doing is feeding a digital signal that will be transformed by the outboard DAC, right? Or am i missing something?