One of the new publications that appeared in my searches this week was this paper on “Next-Generation Sequencing vs Culture-Based Methods for Diagnosing Periprosthetic Joint Infection After Total Knee Arthroplasty: A Cost-Effectiveness Analysis”, published in The Journal of Arthroplasty. I wrote about it on this Twitter thread, but I just wanted to expand on a couple of points here.

There was a line in the abstract that caught my attention:

“At our base case values, culture was not determined to be cost-effective compared to NGS, with an ICER of $422,784 per QALY”.

This struck me as odd because culture-based methods for diagnosing infections have been around for a while now, and are not expensive. In addition, the comparison seemed back to front. On further investigation, this seems to be a good example of an economic evaluation paper that has been published in a clinical journal with little or no oversight from anybody with health economics training.

The main issue is that the authors have incorrectly framed their comparison, possibly because next generation sequencing (NGS) ends up in the south-east quadrant of the cost-effectiveness plane in their study (NGS saves $2,429 in costs, but produces 0.005 fewer QALYs – presumably these are mean figures, but this is not stated). The authors should have compared NGS to current practice (culture), but instead compared culture to NGS i.e. assumed that NGS was current practice. So their conclusion should have been that “NGS was not determined to be cost-effective compared to culture, with an ICER of $422,784 per QALY”.

This incorrect result is then used to frame the presentation of the rest of their results, and ultimately their discussion, implying that it is culturing that is not cost-effective, when in fact the opposite is true.

This is one of several issues that I have with this paper.

• Why have the authors done one-, two- and three-way sensitivity analysis (who does three-way sensitivity analysis?!), but not undertaken probabilistic sensitivity analysis? A cost-effectiveness acceptability curve would have been quite illuminating for this comparison.
• Why were such wide cost ranges considered in the sensitivity analysis for the two tests? Neither bacterial NGS not culture-based tests are ever going to cost $5,000 per test.
• In what way has this economic evaluation been conducted from a societal perspective, instead of a health service perspective?
• I strongly suspect that the transition probabilities have been lifted directly from clinical papers without actually converting them to annual transition probabilities.

So all in all, this is not a great paper, and I don’t think anybody should be making decisions about the use of NGS or culture-based tests in this context based on these results: the results are almost certainly wrong.