Multiple Testing in Sequential and Non-Sequential Experiments

Join the Department of Statistics for a seminar featuring Michael Barón, professor of statistics at American University. 

Abstract:

The problem of multiple hypothesis testing often arises in sequential and non-sequential experiments such as clinical trials, change-point detection, acceptance sampling, etc. In such studies, it is necessary to reach a statistical decision for each individual hypothesis instead of combining them and giving one answer to the composite hypothesis.

Fixed-sample methods of multiple comparisons have been well developed. There are Holm, Hommel, Benjamini-Hochberg, Guo-Sarkar, and other methods that can control the familywise error rate (FWER) or the false discovery rate (FDR). We’ll start the talk with elaborating sequential counterparts of these tools. Proposed methods attain simultaneous control of both Type I and Type II FWER, similarly to the sequential probability ratio test of a single hypothesis.

Next, we develop techniques to reduce the expected sample size, and hence, the expected cost, under these FWER constraints. Methods include optimal error spending, weighted statistics, and equalizer rules. Asymptotic optimality is achieved under the Pitman alternative. Substantial cost reduction can be achieved by controlling generalized FWER, which is practical in applications that involve a large number of comparisons.

By introducing asymmetry into the error spending across hypotheses, we justify, formulate, and solve associated minimax optimization problems that result in more efficient error spending strategies. This leads to equalizer rules that minimize the overall risk, cost, or required sample size, while preserving strict control of FWER.

Proposed methods provide an extension of classical procedures, yielding gains in efficiency without compromising statistical accuracy. We illustrate this approach using examples of case studies from clinical trials.

About the Speaker:

Michael Barón is a professor at American University, where he arrived 12 years ago from the University of Texas at Dallas. He conducts research in sequential analysis, change-point detection and multiple testing, with applications in epidemiology, clinical trials, insurance, energy finance and semiconductor manufacturing. This last application brought him to IBM T. J. Watson Research Center, where he was a one-year academic visitor. Barón is credited for extending stepwise multiple hypothesis testing methods to sequential analysis and introducing asymptotically pointwise optimal stopping rules in change-point detection. He also elaborated several classes of sequentially planned statistical procedures. These are flexible group sequential sampling schemes with dynamically determined group sizes that result in substantial time and cost saving, when the cost is nonlinear.

Barón participated in the design and analysis of several clinical trials. He authored a probability and statistics textbook for computer scientists and co-authored a series of books studying applications of statistics in sociology and marketing. M. Barón is a fellow of the American Statistical Association, an elected member of the international Statistical Institute, and a recipient of Abraham Wald award for the best paper in Sequential Analysis.

Barón has a university diploma in mathematics from St. Petersburg State University, Russia, and a Ph.D. in statistics from the University of Maryland. In his turn, he graduated 12 doctoral students, all of whom are happily employed, mostly at academic and research positions.