Giovanni Parmigiani, Ph.D.
Associate Professor of Biostatistics
Bloomberg School of Public Health
The Johns Hopkins University
Baltimore, Maryland U.S.A.

Over the last decade Bayesian hierarchical models have been increasingly used in numerous areas, including clinical trials and epidemiological studies. This is now a well established methodology for handling study-to-study heterogeneity, small sample sizes, heterogeneous study designs, publication bias, and other complexities.  The necessary computations for fitting Bayesian hierarchical models in a wide range of situations can be carried out conveniently using standard software packages such as BUGS.  As results from Bayesian hierarchical models are increasingly used to support clinical and policy decision making, the issue arises of whether they provide a sound way for comparing treatments. In this case study we will consider a meta-analysis of 2x2 tables, each arising from a study comparing adverse event counts for a treatment arm and a control arm. Our analysis will highlight strengths as well as important potential limitations of Bayesian hierarchical approaches, and will emphasize approaches that ensure robustness of conclusions to modeling choices such as parameterization, distributional assumptions and prior hyperparameters.