Hybrid elicitation and quantile-parametrized likelihood
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (PREreview)
Abstract
This paper extends the application of quantile-based Bayesian inference to probability distributions defined in terms of quantiles of observable quantities. Quantile-parameterized distributions are characterized by high shape flexibility and parameter interpretability, making them useful for eliciting information about observables. To encode uncertainty in the quantiles elicited from experts, we propose a Bayesian model based on the metalog distribution and a variant of the Dirichlet prior. We discuss the resulting hybrid expert elicitation protocol, which aims to characterize uncertainty in parameters by asking questions about observable quantities. We also compare and contrast this approach with parametric and predictive elicitation methods.
Article activity feed
-
This Zenodo record is a permanently preserved version of a PREreview. You can view the complete PREreview at https://prereview.org/reviews/6138664.
The manuscript presents an interesting elicitation method that allows the elicitation of the observable quantities to be accompanied by the elicitation of the uncertainty around them. Since quantile probability tuples (QPTs) are more natural to operate for humans than e.g. probability density functions, this work provides an excellent novel methodological opening how the elicitation method can infer based on QPTs provide by the expert.
The authors "describe and illustrate the hybrid elicitation approach for parameterizing the QDirichlet prior for describing the uncertainty in the {𝑝, 𝑞}𝑛 [quantile-probability-tuplets] parameters of the proper metalog". In simple words, …
This Zenodo record is a permanently preserved version of a PREreview. You can view the complete PREreview at https://prereview.org/reviews/6138664.
The manuscript presents an interesting elicitation method that allows the elicitation of the observable quantities to be accompanied by the elicitation of the uncertainty around them. Since quantile probability tuples (QPTs) are more natural to operate for humans than e.g. probability density functions, this work provides an excellent novel methodological opening how the elicitation method can infer based on QPTs provide by the expert.
The authors "describe and illustrate the hybrid elicitation approach for parameterizing the QDirichlet prior for describing the uncertainty in the {𝑝, 𝑞}𝑛 [quantile-probability-tuplets] parameters of the proper metalog". In simple words, the authors introduce an elicitation method for a Dirichlet (or Connor-Mosimann) prior in the context of meta-logistic distribution (Keelin, 2016) as a likelihood. In the world of quantile-parametrized distributions, their method can be regarded as a model-specific elicitation method for a meta-logistic model with a (Q)Dirichlet prior.
The manuscript is, at some parts, difficult to place side by side with the already existing elicitation literature. The adopted terminology is not consistent with the literature. For instance, in the first paragraph the authors write "Elicitation of parameters" is "known as the parametric elicitation (Garthwaite et al., 2005; O'Hagan, 2019)". Only the former of the referred works use the term "parametric elicitation", and it's used for a prior elicitation method that assume a parametric prior distribution (in contrast to non-parametric prior distribution). Similarly, in the second paragraph, the authors hint that the "predictive elicitation" is not about eliciting a prior distribution for model parameters but a distribution for "next observation", but this not true in the referred work (Akbarov, 2009, p.121), not to mention the early research on this research line in prior elicitation (Kadane and Wolfson, 1998; Kadane et al., 1980; Oman, 1985; Garthwaite and Dickey, 1988; Ibrahim and Laud, 1994; Bedrick et al., 1996; Chen and Ibrahim, 2003; Denham and Mengersen, 2007; Elfadaly and Garthwaite, 2011; Garthwaite et al., 2013; Elfadaly and Garthwaite, 2015).
I expect that the manuscript can evolve to an excellent paper, if (i) the connection to the existing elicitation literature is made clearer and the terminology is harmonized with that (a minimum requirement is that potential different usage of terms in references is explicitly stated), and (ii) the applicability of the proposed elicitation method is made more explicit (e.g. Is this elicitation method for any Dirichlet prior, or an elicitation method for a meta-logistic model with a QDirichlet prior, or both? How the method works with real human experts? etc..).
-
