Last week, I asked the question, “Is podiatry scientific?” As a quick review, here’s what the scientific method looks like:

Which of these Major Medical Journals is Most Scientific?

Let’s do a quick quasi-experimental study. I looked through four journals that podiatrists commonly use to publish their research or read for continuing education. I then listed articles according to the general types of clinical/medical studies, they are:

In the table below, I’ve tallied the number of studies published in our two major podiatric and two major orthopedic journals over the last month (the most recent publication for each). As a control (since we’re talking science), I’ve chosen the New England Journal of Medicine (NEJM) since it has the second highest impact factor (72.406 in 2016)¹ of all medical journals in the United States. The highest impact factor is the Cancer Journal for Clinicians with an impact factor of 187.040.¹

---

“Impact Factor Score is a proxy for how influential a journal is”

---

As a quick aside, impact factor is a measure that describes the average number of times articles in a particular journal published over a 2-year period are cited in a particular year. This number is used as a proxy for how influential a journal is. For interest sake, here are the most recent impact factors of the journals we are discussing:

A couple of notes on my methods and choices. I counted meta-analyses, systematic reviews, and non-scientific reviews in one category. These fall in their own category and are not scientific. Technique tip articles are included in “reviews". Studies using surveys to test patient populations were included in the observational study category.

It’s important to note that the journals are published at different intervals (JAPMA bimonthly, JFAS bimonthly, JBJS biweekly, FAI monthly, NEJM weekly). This makes it difficult to accurately compare the numbers; a year’s worth of comparisons would be better. As a result, I made one significant assumption: The most current journal publication was indicative of the article types published for the year. Using this assumption, I multiplied each result by the number needed to show one year’s worth of publications. For example, JAPMA and JFAS are bimonthly, so I multiplied the surveyed results by six to determine the number of a study type in one year (which is noted in parenthesis).

Table 1. Numbers of article types included in 4 common lower extremity journals (one year average in parenthesis)

The Results

Looking at this table, we have the extrapolated numbers of article types in one year (Total of 924 articles from our experimental group and 912 from our NEJM control for a total of 1836 articles).

If we eliminate all of the obviously nonscientific studies (case studies, reviews, and editorials) there are 570 clinical studies.

We’ll now remove the observational studies since these don’t follow the scientific method (and can – if you recall – only determine if a correlation exists between two things and not causation with observational studies). That brings us down to 192 studies of experimental type (20.8% of the original total).

Since studies without a control do not correspond to the scientific method, removing them leaves 168 studies that most closely adhere. We’re now down to 18.2% of the original total. That’s 18.2% of articles in these five journals that are closest to following the scientific method in the sense that they are doing all they can to remove lurking variables and potential bias by having a control group. This is a rather free and loose method of defining a scientific study!

We should note that our control, the NEJM, despite its high impact factor, did not fare well against JBJS and was only a little higher than JFAS in the number of scientific studies published.

Only a small percentage of these journal statistics are close to following the precepts of the scientific method, so I’ll go out on a limb and say we’re mostly nonscientific.

Now, don’t despair. Keep in mind there are lots of limitations to my analysis. First, the analysis itself is potentially highly inaccurate. Better to do a survey of several years and several more journals rather than extrapolating as I did. Second, JBJS includes non-lower extremity studies, which I did not filter out. Third, there’s no comparison to other medical fields, which are only a little more scientific than we are. Keep in mind that medication studies are much easier to structure scientifically than surgical ones.

As a final note, almost none of the articles surveyed were biomechanical in nature, which is what I began this discussion with. If one were to look at these, one would find a dearth of good science mostly because the research methods at the time when they were done were not as sophisticated as they are becoming today. We’ve seen an upward progression of the technology available to learn about the foot and ankle, which will open up new vistas in our understanding of the lower extremity.

How Biomechanics is Studied

Originally, the only biomechanical studies possible were in vitro and kinematic in nature (describing positions of bones), using mounted cadaver extremities that did not approximate gait. Studies then moved to examining cadaver movement using X-ray stereophotogrammetry, a much better way to determine 3D relationships. However, these were also using mounted cadavers. We have since progressed to technologies that can study kinetics (forces acting on the extremity) in vivo during actual human gait and various camera systems. The newest technology, weightbearing functional MRI, stands to be the newest and potentially best method to study kinematics of the human foot during stance and gait.

In the end, the question “is podiatry scientific” is not that easy to answer. I would suggest the answer is “kind-a". We have significant methodological limitations due to the nature of the subject, but we don’t help ourselves by having so few well-controlled and methodologically sound studies. However, we’re only getting better, compared with the past, and with a continued desire to improve will come ever-better research and answers that come closer to reality.