In any experiment, especially psychological, the devil is in the details. Even when elementary errors of logic have been avoided, and even when the statistics have not been abused, measurements are often irrelevant to the goals of the study.
The whole scientific literature is littered with claims not supported by the data.
@terry9 Statistics are invariably abused. Either deliberately or by sheer utter incompetence. Neither transgression is forgivable. Swaths of rants could be (and have been) devoted to this, perhaps what you concisely describe as the devil is in the detail..
Robust studies devote an abundance of resources defining exactly what variables need and can be be quantified and in a manner that is beyond reproach.
Sadly, many empirical studies spend 5% of the time thinking about the problem and 95% of time finding a solution. The opposite approach, often incorrectly attributed to Einstein, is the one with merit.
Edit - I do not limit this comment to any particular field being analysed. It applies generally.