Armstrong DG, Marston WA, Reyzelman AM, Kirsner RS.  A multicenter randomized controlled trial comparing negative pressure wound therapy with the SNaP™ Wound Care System vs. V.A.C.® Therapy System for the Treatment of Chronic Lower Extremity Ulcers. Unpublished data.  Presented at the 2010 Georgetown DLS Conference.

WHY did the authors undertake this study?
The value of negative pressure wound therapy (NPWT) has been readily established within the field of wound care, specifically in the treatment of lower extremity wounds.  The authors of this abstract aimed to compare a relatively new device for the delivery of NPWT (The SNaP™ Wound Care System) with the industry standard (KCI’s V.A.C®).

HOW did they attempt to answer this question?
The primary outcome measure of the study was complete wound healing at 16-weeks of therapy.   A secondary outcome measures of percent wound size reduction was also analyzed.

A multi-center, randomized, non-inferiority study design was utilized in patients with non-infected, non-ischemic, non-plantar lower extremity wounds.

WHAT were the specific results?
Interim analysis demonstrated no significant difference in the proportion of complete wound healing between the two groups.  There was also no significant difference in percent wound size reduction at the 4, 8, 12 and 16 week time periods, with non-inferiority demonstrated after 4-weeks of treatment.

HOW did the authors interpret these results?
From these interim results, the authors concluded  that there was no difference in wound closure between the two NPWT devices.

Slim H, Anwar A, Tiwari A, Ritter C, Edmonds M, Zayed H, Rashid H.  Distal versus ultra-distal bypass grafts; amputation free survival and patency rates in patients with critical limb ischemia.  Unpublished data.  Presented at the 2010 Georgetown DLS Conference.

WHY did the authors undertake this study?
The treatment of distal peripheral arterial occlusive disease remains a challenging clinical situation for vascular surgeons in the setting of diabetic limb salvage.  Outflow, or an appropriate site for distal bypass, is a major problem when considering distal anastomosis.  The aim of this study was to evaluate clinical outcomes following distal bypass grafting (defined as distal anastomosis distal to the popliteal artery) and ultra-distal bypass grafting (defined as distal anastomosis at or distal to the ankle) in patients with critical limb ischemia.

HOW did they attempt to answer this question?
The primary outcomes measures of this study included 1-year amputation-free survival rate and graft patency rates.  Other descriptive and comparative statistical analyses were also utilized.  One hundred and eighty three bypasses performed in 170 consecutive patients were analyzed.    

WHAT were the specific results?
Rates of amputation-free survival, primary patency and secondary patency were similar between the two groups.  The incidence of heart disease and end-stage renal disease was similar between the two groups, although there was a statistically significant higher percentage of diabetic patients in the ultra-bypass group.

HOW did the authors interpret these results?
Based on these results, the authors concluded that ultra-distal bypass represents a viable option in patients with critical limb ischemia.

Let’s take a closer look at the topic of non-inferiority study designs.  The Armstrong et al abstract provides an example of a study design which has become increasingly prevalent in the medical literature with the recent explosion in the number of available medical devices.  Usually when we are evaluating a study, we are looking for a statistically significant difference between two groups.  Maybe we are evaluating radiographic changes pre-operatively compared to post-operatively, or maybe Group A receives an anti-hypertensive medication and Group B receives a placebo.   In these comparative studies, our hypothesis is that the given intervention (or variable) is going to have demonstrable effect.  A non-inferiority study (also known as an equivalence study) really flips that hypothesis around.  The study design still allows for a comparison, but it’s usually between two different variables with a similar effect, and the hypothesis is that there will not be a demonstrable difference between the two.
 
Let’s take a closer look at the Armstrong et al abstract.  They are not attempting to determine whether or not NPWT delivered with the SNaP™ device has an effect on wound healing.  In fact, they assume or have already taken for granted that it does have an effect.  They are also not attempting to determine whether or not NPWT delivered with the V.A.C.® device has an effect on wound healing.  Again, the effect of V.A.C.® on wound healing has already been established and authors work from that assumption.  They are, however, attempting to determine whether or not NPWT delivered with the SNaP™ device has an equivalent effect on wound healing compared to the V.A.C.® device.
 
Why would we do this?  What does it matter it they essentially have the same effect?  Well, there are a couple reasons.  One therapy could be cheaper, or have fewer side effects, or an easier administration for examples.  This is the first important point when it comes to critical analysis of non-inferiority studies, and one that has nothing to do with statistics.  If non-inferiority is demonstrated, a critical reader should ask themselves, why does this matter to my patient?   How would this change the way that I practice?  This can be a difficult question to answer because it’s not based on any statistical formula and it’s usually not something that is directly studied in the investigation.  It’s almost as if you have to be a little bit weary of someone trying to sell you something!

A second important point with respect to critical analysis of non-inferiority studies is the importance of power.  We’ve covered this in previous PJC issues, but in this situation we have to think of it from the other perspective.  We know that statistical significance is not always clinical significance.  This is simplified, but in most study designs, prior to data collection the authors should figure out what they think is a clinically significant difference, and then calculate a power analysis to determine how many patients would be needed to statistically determine this clinical difference.  If a study doesn’t do this, and they inadvertently don’t analyze enough patients, they are likely to not demonstrate a statistically significant difference.  Think of this as a little like committing a type II error.  There may be a difference, but they didn’t design a study powerful enough to demonstrate it, so they conclude there is no difference.

Think about this situation however with a non-inferiority analysis.  In this case we are more concerned if there is a difference.  So if a study isn’t powerful enough, they will inadvertently not demonstrate a statistically significant difference and ergo demonstrate non-inferiority.  Along these same lines, if possible, you as a critical reader should ask yourself what you think is a clinically significant difference.  Do you agree with the clinically significant difference that the authors used to drive their power analysis, or would you have come up with another definition of a clinically significant difference for the situation?

A final point to consider is exactly which type of statistical test is being used to determine whether or not a difference exists between two groups.  This one is a little bit more complex.  Everyone appreciates the basic concept of a level of significance of 0.05 when looking for a difference.  What you may not appreciate, however, is that we usually use a two-tailed test and are in fact looking for a level of significance of 0.025!  In other words, the difference between two groups could be in the 2.5% at the high end OR 2.5% at the low end.  The total level adds up to 5%, but we are conservatively looking for half at the high end and half at the low end.  Sometimes with a non-inferiority study we only look at one end (or a one-tailed test).  So we only look for a 5% difference at the low end and ignore the high end for example.  This isn’t necessarily wrong, but it is a little less conservative from a statistical standpoint.  It’s not essential, but nice to know if a non-inferiority study is using a one-tailed or two-tailed test.