What Almarg said is correct. Headphones and power amps differ enough that these should be considered different applications. Optimizing for one application will involve trade-offs for the other. Here are two big trade-offs to consider:
1) Output caps (for OTL tube preamps/headamps) - Headphone tube amps require much LARGER output capacitors than tube preamps. Bass roll-off is determined by cap size and load impedance. Headphones are much lower impedance than a power amp. You'll typically need about 440uF on the output of a tube headamp to prevent significant roll-off with 32 ohm headphones. In a preamp, ~ 4.7 uF output caps are fairly common. With the reduction in size, much higher quality capacitors can be used. For 440 uF @ 250V, you're stuck with electrolytic types. It's a necessary evil for an OTL tube headphone amp (many headphone aficionados go gaga for black gate electrolytics here - getting harder to find), but a sub-optimal choice in a speaker system.
2) Feedback - In order to make output impedance "small" compared to its load, some feedback will generally be required for a proper headphone amplifier. Preamps can often do without global feedback, if they choose. Some tube headphone amps use output transformers to step-down to the lower impedances, but again this is unnecessary for a preamp, and would just add another quality-critical component to the chain.
For these reasons - when you see a headphone jack on a preamp or integrated, there will often be dedicated headphone circuitry behind it (usually SS, and usually chips). Not great, but better than running from the main circuit, and better than making compromises to the speaker rig. My Rogue 99 Magnum has a headphone jack that is actually a direct tap into the preamp circuit - it's a great preamp, but HORRIBLE for headphones. I only use the jack to test new tubes before firing up the power amp. Other Rogue amps have dedicated SS circuitry behind their headphone jacks - not sure why they did the 99 that way.
A FEW preamps can do a good job at both. The Melos SHA-gold (tube/SS hybrid) was one such amp. The Eddie Current Zana Deux (tube OTL, 6sl7/6c33c with a bit of feedback) is freaking EXCELLENT at both, and the only one I can highly recommend in a dual-role - though no cool features like remote volume control, mono button, etc. The Manley Shrimp is apparently quite good too, though I've not heard it myself (50 ohm output with some feedback - not sure what cap size is used). Personally, I'd prefer to keep my preamp focused on and optimized for the speaker rig. I've had some great dedicated headphone amps, but now I'm fully focused on the speaker rig.
In short - keep your preamps and headphone amps separate :)