Despite an explosion of cancer research to exploit biomarkers for clinical application in diagnosis, prediction and prognosis, the impact of biomarkers on clinical practice has been limited. The elusiveness of clinical utility may partly originate when validation studies are planned, from a failure to articulate precisely how the biomarker, if successful, will improve clinical decision-making for patients. Clarifying what performance would suffice if the test is to improve medical care makes it possible to design meaningful validation studies. But methods for tackling this part of validation study design are undeveloped, because it demands uncomfortable judgments about the relative values of good and bad outcomes resulting from a medical decision.
An unconventional use of “number needed to treat” (NNT) can structure communication for the trial design team, to elicit purely value-based outcome tradeoffs, conveyed as the endpoints of an NNT “discomfort range”. The study biostatistician can convert the endpoints into desired predictive values, providing the criteria for designing a prospective validation study. Next, a novel “contra-Bayes” theorem converts those predictive values into target sensitivity and specificity criteria, providing the basis to design a retrospective validation study.
In the experience of this author, NNT-guided dialogues have contributed to validation study planning by tying it closely to specific patient-oriented translational goals. The ultimate payoff comes after completing and reporting a well-justified study. Readers will understand better what the biomarker test has to offer patients, because the study provides a biomarker test decision framework directly aligned with the targeted clinical decision challenge.
Suppose some biological characteristic of a patient would determine our choice between acting or waiting if we knew its status: either BestToAct or BestToWait. Initial knowledge or belief about the patient’s status is represented by a “prior probability” Pr(BestToAct), which could express a precise estimate or a subjective opinion. When there is treatment controversy, Pr(BestToAct) is too far from certainty (one or zero) to make the best clinical decision clear to most physicians.
The intention is that some biomarker test yielding a positive (Pos) or negative (Neg) result will reveal something about the patient status, so that knowing the test result updates our knowledge, expressed by moving Pr(BestToAct) up or down using Bayes Theorem. Now, one decision challenge is replaced by two: for Pos and for Neg patients. The hope is that they will have clear and opposite decisions.
To elicit the NNT discomfort range, we strip physician preferences down to the essentials. One guides the NNT respondent, typically the clinical principal investigator, to imagine a clinic schedule of patients, together with the certain knowledge that, if treated, exactly one will receive the benefit hoped for. With this scenario, all uncertainties are removed, and all outcomes are known; the only thing unknown is which of the NNT patients will be the sole beneficiary. Framing the problem in terms of a fixed number of patients with fixed outcomes rather than probabilities or proportions circumvents the well-known documented numeracy deficiencies that plague even medical researchers.
After choosing the discomfort range, the next steps are choosing desired values outside this range for NNTPos and NNTNeg in the test-positive and test-negative subgroups, and leveraging those values to guide the study design. This exercise helps the study team determine a relevant patient population and rules for selecting specimens, and ensure that the study’s eventual results will have utilitarian interpretability for guiding clinical decisions.
See the vignette Using_the_NNTbiomarker_package