Biomarker Discovery
Selection: these candidates are likely to be highly correlated, meaning that the information provided by any one of them is contained among some of the other ones
Reproducibility: we may need to perform so many tests that we are guaranteed to find statistically significant results at most common p-value levels (0.05, 0.01, 10-50), even when there are not real predictive biomarkers. On its own, a small p value is not evidence for anything**
Robustness of identification: small changes in the training or testing data set could lead to another (equally unreliable) biomarker proposal.
Non-optimal design: if data come from a source that was not experimentally designed for biomarker selection, we lose some statistical guarantees that makes the robustness of the results harder to establish.
How can we robustly pick a winner?
Is any candidate consistently a winner?
Main Issues during feature selection:
“The children of very tall parents tend to be tall but shorter than their parents. The children of very short parents tend to be short but taller than their parents”
This discovery is called “regression to the mean” with the word “regression” meaning “to come back to”
What is a biomarker?
What are biomarkers used for?
How do we develop a biomarker?
Considerations for the discovery of biological (molecular) targets
Taking information that exists on a continuous scale (such as blood pressure, lactate level, troponin) and transforming it into a binary or discrete cutoffs (e.g., “normal/abnormal”, “class 1/2/3”) for statistical and clinical convenience (Matt Sluba, 2021-02-28, link)
Categorization assumes that:
Risk profiles (or other outcomes you’re interested in) are constant in (usually heterogeneous) subgroups
There are sharp jumps at (usually arbitrary) cut points.
A common method of finding “optimal” cutoffs is the so-called “minimum p-value method (MinPVal)”
It take several candidate cutoff values and see which one provides the “most significant” difference in risk between the two sides, in terms of the smallest p-value.
Dichotomization needs more data to make the same level of statistical inference compared to a continuous predictor
One interaction (for a predictive biomarker, for example) will need at least 4 times more data for a dichotomized prognostic biomarker compared to a continuous biomarker.
Linearity also results in information loss -> inefficiency -> misleading conclusions.
But probably it is not as bad as dichotomization.
library(tidyverse)
library(tidymodels)
library(rms)
data(elas, package="OptimalCutpoints")
model <- glm(status ~ rcs(elas), data = elas, family = binomial)
a <- augment(model, newdata=elas, type.predict = 'response')
ggplot(a, aes(elas, .fitted))+geom_line()+theme_bw()+
labs(y = "Probability of CAD")
P-value for elas (leukocyte elastase) is \(1.8\times 10^{-7}\)
set.seed(2048)
library(rsample)
btstrps <- bootstraps(elas, times = 1000)
models <- map_df(btstrps$splits,
\(x) augment(glm(status ~ rcs(elas), data = analysis(x), family=binomial),
newdata=assessment(x), type.predict='response'),
.id = 'bootstraps')
ggplot(models, aes(elas, .fitted, groups = bootstraps))+
geom_line(alpha = 0.05)+ theme_bw()+
labs(y = "Probability of CAD")
