Practice Perfect 778
Can We Really Use the Research Literature?
Can We Really Use the Research Literature?
Recently, during a residency journal club, we discussed four articles related to a particular clinical question. The specific articles under discussion were chosen by one of the residents, and as the discussion proceeded, I realized that the articles were being interpreted incorrectly. We discussed the proper interpretation of these specific articles, but I can easily say that the residents made many of the same mistakes any practicing medical professional could make. In fact, some of the authors of those very studies made errors, which were, unfortunately, not caught during the peer review process and made it into publication.
This is definitely not the first time we have come across issues in the research literature, whether it being errors of interpretation (our individual faults) or research methodology problems. In fact, during a prior journal club with our students, almost every study that we discussed had major flaws. It almost became a joke that so few studies survived critical analysis.
On the other hand, I have heard (and made) the criticism of national speakers when they assert the success of certain treatments affirming, “In my hands this is successful.” This type of rhetorical device, resorting to authority, may have a place in one-to-one discussions, but anecdote is not scientific and is of little true value to the rest of us. If it’s good in your hands, then what about in others’ hands? What about my hands? Or his hands? Or her hands?
In some cases, the medical literature and experience don’t seem to match up well. I’ve heard Dr Douglas Richie give a very strong argument against the frontal plane bunion correction issue. He’s very convincing, and I respect everything he says. He cites research that argues against our need to create a varus rotation force on the first metatarsal during bunionectomies. On the other hand, my direct experience contradicts this argument. When I perform bunionectomies, I have found repeatedly that my outcomes are much improved when I place a varus rotation on the metatarsal until the 1st metatarsal head is sitting directly dorsal to the sesamoids which are themselves in a rectus position and the hallux is also rectus in the frontal plane. I don’t typically reconstruct the hindfoot deformity with these patients, so I may just be compensating for the valgus position of the entire foot. This is a difficult conflict to resolve when real-life experience contradicts well-made arguments and the literature.
As someone who takes the medical literature seriously – I’ve written literally hundreds of editorials that use the research literature – this dichotomy between flawed research on the one hand and anecdote on the other is troubling. How much of the literature can we really believe? How much of an individual’s experience can we believe? How much of either can we apply successfully to our own practices to help our patients?
Can we really use the research literature?
One of the primary problems for us in the more mechanical aspects of medicine is that contemporary research methods were created to study medications and not physical treatments such as surgery, bracing, etc. The randomized double blind controlled study, considered the top level of research quality, does not often work for surgery. For example, if I wanted to study the effects of resection of tarsal coalitions on patient outcomes, it would be unethical to group patients to coalition resection versus sham surgery. It’s also impossible to blind a surgeon to the procedure he or she will perform. And again, it’s not ethical to blind a patient to their surgical procedure; this violates the consent process. Additionally, it’s very difficult to gather a large enough patient cohort for something like this compared with studying a medication for a common condition like hypertension.
Let me say right here that there’s literally nothing else to use. It’s either our own experience or the research literature or both. My stance, then, is that we have to use both, but each of us should maintain some skepticism when considering both methods. Let’s talk about some examples and suggestions to hopefully help.
Experience Counts…From the Right Person
Experience comes in two forms: our own and that of others. When considering recommendations from others, I lend increased weight to those with deep and extensive experience with their topic. I’d prefer it if they could prove that experience before giving their recommendations. For example, I do not see large numbers of patients with tarsal coalitions. As such, if I lectured on that topic, I would not have authority, and my comments should be considered carefully and with due reservation. As a conference lecturer, if I were asked to give this lecture, I would first offer to speak on a different topic, but if it had to be coalitions then my lecture would be much more heavily weighted on the research literature. Personally, I would approach this topic as “I was asked to explore it, and this is what the literature tells us.” On the other hand, someone like Douglas Richie, Jr, DPM or Kevin Kirby, DPM can speak with authority on biomechanics of the lower extremity, and I give them more authority, which they deserve.
Interpreting the Research Literature
We can and should read the research literature. But we have to use this tool knowing its limitations. First, we in podiatry just have to accept the fact that some of our research has intrinsic shortcomings. This is also true in the orthopedic literature, by the way. Every study has flaws, and they may be intrinsic or situational. The limitations of a surgical study like the one mentioned above are intrinsic and very difficult to change. On the other hand, a study that takes place in Scandinavia may not be generalizable to my population in Southern California. This lack of generalizability (or external invalidity) is situational, and a different study may be more appropriate.
Next, we have to consider context and the specific clinical question of interest. Recently, our residents presented articles about arthroereisis devices with the question pertaining to the use of absorbable versus nonabsorbable implants. It became clear during the discussion that the residents were considering two simultaneous questions: should we use arthroereises at all and if we do, is absorbable a reasonable device instead of metal? Diverging to a different question is less helpful and definitely less efficient. Focusing on one question is a bit more manageable.
My suggested way to deal with these research limitations is to first approach the research from a question-based perspective.
- Don’t just randomly look for articles - Consider a specific clinical question generated from patient interactions. Keep the question in mind while looking for literature to answer the question and don’t deviate.
- Find more than one article - that looks at the question at hand. If one finds five studies that all point to one answer - even if those studies do not have the strongest methodologies – that’s reasonably strong evidence that you have discovered some truth. Sometimes an abundance of circumstantial evidence will work for real-life applications.
- Use an efficient and organized approach - when reading those studies that avoids reading the entire study unless you’re very interested and have the time. Skip the abstract (that’s for determining if you want to read the article in the first place). Look first at the end of the intro section and find the “purpose” statement. That will tell you the primary goal of the study. Then, go to the methods section and quickly determine if it is internally valid (are the methods reasonable to address the purpose statement). Next, examine the results, focusing on tables and graphs to gather information quickly. Finally, skim the discussion, looking to see if the authors’ conclusions match yours and if they make sense.
With all studies, each of us needs to analyze the results within the framework of our own experience and our patients’ needs and desires. This is where the combination of the research literature and experience come together to give us evidence-based practice. Good luck with your next clinical question.
Major Sponsor
Comments
There are 0 comments for this article