Daubs MD, Patel AA, Willick SE, Kendall RW, Hansen P, Petron DJ, Brodke DS. Clinical impression versus standardized questionnaire: the spinal surgeon’s ability to assess psychological distress.  J Bone Joint Surg Am.  2010 Dec 15; 92(18): 2878-83. (PubMed ID: 21075972)

WHY did the authors undertake this study?
Psychological distress is a known factor contributing to the evaluation, treatment and prognosis of patients with musculoskeletal spinal disorders.  Despite this known information, the majority of physicians do not routinely assess patients for psychological distress during evaluation and prior to treatment initiation.  The aim of this study was to evaluate the ability of physicians to assess psychological distress in a group of patients. 

HOW did they attempt to answer this question?
The primary outcome measure of the study was patient classification with the use of the Distress and Risk Assessment Method (DRAM).  Four hundred new patients presenting for initial spine specialist consultation completed the questionnaire and were grouped into one of four potential categories:  Normal, At Risk, Distressed-Depressive, or Distressed-Somatic

At the end of the first patient visit, eight physicians (blinded as to the actual DRAM score) also attempted to categorize the patients using this classification, but without actually using the patient questionnaire. 

WHAT were the specific results?
Sixty-four percent (254/400) of patients were found to have some level of psychological distress as measured by the DRAM, but physicians only had a sensitivity rate of 28.7% in their assessment without use of the DRAM.  There was no difference in sensitivity rates between more experienced and less experienced surgeons, but non-operative spine specialists had statistically significant higher sensitivity rates compared to spine surgeons (41.7% vs 19.6%; p=0.03). 

HOW did the authors interpret these results?
From these results, the authors concluded  spinal surgeons had low sensitivity with respect to recognition of psychological distress in new patients, and should consider the routine use of a standardized questionnaire during evaluation.

Berrington de Gonzalez A, Hartge P, Cerhan JR, et al.  Body-mass index and mortality among 1.46 million white adults.  J Engl J Med. 2010 Dec 2; 363(23): 2211-9. (Pubmed ID#: 21121834)

WHY did the authors undertake this study?
Although obesity is a known risk factor for the development of health problems, the exact relationship between body mass index (BMI) and all-cause mortality remains somewhat cloudy.  Also unknown is the ideal range for BMI with respect to all-cause mortality.  The objective of this study was to determine the answer to both of these clinical questions. 

HOW did they attempt to answer this question?
The primary outcome measure of this investigation was mortality among a pooled analysis of data from >1.4 million patients and >150,000 deaths.  Regression analysis was performed to account for differences occurring from age, sex, tobacco use and physical activity. 

WHAT were the specific results?
A J-shaped relationship was noted between BMI and all-cause mortality (meaning that initially lower, and then higher BMI values were associated with increased mortality).  The lowest values of all-cause mortality occurred with a BMI range of 20.0 to 24.9. 

HOW did the authors interpret these results?
From these results, the authors indirectly concluded that all those New Year’s Resolutions to lose weight are probably a good thing. 

Let’s take a closer look at the topic of validity of outcomes measures, specifically as it applies to the Daubs et al psychological distress article.  The Journal of Bone and Joint Surgery has started a great new feature where they will occasionally provide “Commentary & Perspective” following a highlighted article.  It’s basically a well-founded critique of an article that is published immediately following the original publishing of the article, as opposed to months later through the “Letters to the Editor” section or critiques from shadowy guys like me on websites. 

I enjoy this feature for two reasons.  First, it seems incredibly ironic that a journal would publish an article, and then immediately publish a commentary about all the limitations of that article.  The second reason is that the people providing the critiques always go above and beyond to shower flowery praise on the authors of the article they have serious reservations about.  Immediately following the Daubs et al article, Dr. James Kang [https://www.ejbjs.org/cgi/content/extract/92/18/e39] provides a “Commentary & Perspective”, and starts it by saying “The authors of this interesting study should be congratulated on a well-designed and executed clinical study”, but then proceeds to tell us why it is not so well-designed!  Of course I understand the thought-process behind both above points, but it still makes me chuckle. 

Dr. Kang’s reservation is with respect to the validity of the outcome measure utilized by the authors of the study, and as this is a topic we’ve dealt with in this space before, I thought we could piggy-back his point to re-enforce the concept.

When critically evaluating a study, I like to think of outcome measures as the “tools” that the authors utilize to answer their clinical question.  As a basic example we could consider a study that wanted to look at the ability of a certain surgical procedure to correct the HAV deformity.  The “tools” that would be best suited to examine this question could be radiographic measures such as the first intermetatarsal angle, hallux abductus angle and metatarsal-sesamoid position for example.  These measurements each tell me something about the HAV deformity.  If, however, I used the cuboid abduction angle, calcaneal inclination angle and talar head uncovering as my outcome measures for this study, then my “tools” wouldn’t really have any validity in helping me answer the question.  These measurements certainly tell me something about the patient, but not really about the patient’s HAV deformity. 

Dr. Kang questions the use of the DRAM questionnaire as a “tool” to determine a surgeon’s ability to assess psychological distress in patients.  He points out that although the DRAM has been validated, it has not been validated for surgical patients.  In fact, the authors of the original DRAM study specifically excluded surgical patients from their analysis!  So although the DRAM is a potential “tool” for assessment of psychological distress, it probably wasn’t the best tool for >50% (210/400) of the patients who eventually ended up having surgery in this study. 

An analysis of outcome measures is an extremely important component of any critical evaluation of the medical literature.  As demonstrated above, it is not enough to simply ask if an outcome measure has been validated, but we should also be asking ourselves if the outcome measure has been validated for this exact clinical situation.