BioPharmNet
Topics
BioPharmNet forum
 
Navigation
This page is maintained by the Multiplicity Advisory Board (Alex Dmitrienko).
Multi-stage/gatekeeping testing procedures
Multi-stage testing procedures play an important role in clinical trials with multiple families of tests (often termed gatekeepers) that are carried out in a sequential manner. The families can represent families of endpoints (primary, secondary and tertiary endpoints), tests (non-inferiority, superiority), dose levels (higher doses, lower doses) or various combinations of these.
Papers and books
Bauer, P. (1991). Multiple testings in clinical trials. Statistics in Medicine. 10, 871-890.
Bauer, P., Rohmel, J., Maurer, W., Hothorn, L. (1998). Testing strategies in multi-dose experiments including active control. Statistics in Medicine. 17, 2133-2146.
Chen, X., Luo, X., Capizzi, T. (2005). The application of enhanced parallel gatekeeping strategies. Statistics in Medicine. 24, 1385-1397.
This paper introduced Simes-based parallel gatekeeping procedures and Bonferroni-based parallel gatekeeping procedures with logical restrictions (with applications to clinical trials with multiple treatment arms).
Dmitrienko, A., Offen, W., Westfall, P.H. (2003). Gatekeeping strategies for clinical trials that do not require all primary effects to be significant. Statistics in Medicine. 22, 2387-2400 [SAS code and typographical errors].
This paper described testing procedures for clinical trials in which the primary family is a parallel gatekeeper, that is, at least one significant result needs to be found in the primary family before one can begin testing secondary hypotheses. The paper discusses parallel gatekeeping procedures based on the Bonferroni and resampling tests. Software implementation of these procedures is discussed in Chapter 2 of Dmitrienko, Molenberghs, Chuang-Stein, and Offen (2005).
Dmitrienko, A., Molenberghs, G., Chuang-Stein, C., Offen, W. (2005). Analysis of Clinical Trials Using SAS: A Practical Guide. SAS Press: Cary, NC.
Chapter 2 gave a detailed overview of gatekeeping strategies in clinical trials. Clinical trial examples and SAS code are included. A good way to get started in this area.
Dmitrienko, A., Offen, W., Wang, O., Xiao, D. (2006). Gatekeeping procedures in dose-response clinical trials based on the Dunnett test. Pharmaceutical Statistics. 5, 19-28 [Download paper] [Addendum, SAS code and typographical errors].
The paper introduced parametric gatekeeping procedures based on the Dunnett test. This approach was applied to dose-finding clinical trials with multiple endpoints. Dunnett-based gatekeeping procedures account for the correlation among the dose-placebo comparisons and multiple endpoints. They are more powerful than Bonferroni-based gatekeeping procedures described in Dmitrienko, Offen and Westfall (2003).
Dmitrienko, A., Wiens, B.L., Westfall, P.H. (2006). Fallback tests in dose-response clinical trials. Journal of Biopharmaceutical Statistics. 16, 745-755.
This paper extended the fallback testing method (Wiens, 2003) to the multi-stage case. Multiple testing procedures proposed in the paper can be used to carry over a predetermined fraction of the Type I error rate to the next family even if no significant results are found in the previous family. Applications included dose-finding studies with multiple endpoints.
Dmitrienko, A., Tamhane, A.C., Wang, X., Chen, X. (2006). Stepwise gatekeeping procedures in clinical trial applications. Biometrical Journal. 48, 984-991.
It was shown in this paper that parallel gatekeeping procedures (Dmitrienko, Offen and Westfall, 2003) have a simple stepwise structure that considerably simplifies their implementation and interpretation of the results.
Dmitrienko, A., Wiens, B.L., Tamhane, A.C., Wang, X. (2007). Tree-structured gatekeeping tests in clinical trials with hierarchically ordered multiple objectives. Statistics in Medicine. 26, 2465-2478 [SAS code and typographical errors].
Tree gatekeeping procedures serve as an extension of serial and parallel gatekeeping procedures and can be used in clinical trials with multi-dimensional objectives (multiple endpoints, doses, tests, etc) and clinical trials with logical restrictions. Note that the algorithm introduced in this paper may fail to meet the monotonicity condition, which may cause it to violate the gatekeeping property. It is shown in Dmitrienko, Tamhane, Liu and Wiens (2008) how the algorithm can be modified to rectify this problem.
Dmitrienko, A., Tamhane, A.C. (2007). Gatekeeping procedures with clinical trial applications. Pharmaceutical Statistics. 6, 171-180.
This paper gave a review of recent developments in the world of gatekeeping procedures, inclucing serial, parallel and tree gatekeeping procedures, with clinical trial examples.
Dmitrienko, A., Tamhane, A., Wiens, B. (2007). General multistage gatekeeping procedures. Northwestern University. Department of Industrial Engineering and Management Sciences. Working Paper No. 07-06 [Download paper].
This working paper introduced a class of general gatekeeping procedures that have an attractive multistage form (it extends the method proposed by Guilbaud (2007) who focused only on Bonferroni-based gatekeeping procedures). These procedures do not not require the application of the closed testing principle and are easy to implement. A modified version of the working paper will be published in Biometrical Journal (Dmitrienko, Tamhane and Wiens, 2008).
Dmitrienko, A., Tamhane, A.C., Liu, L., Wiens, B.L. (2008). A note on tree gatekeeping procedures in clinical trials. Statistics in Medicine. 27, 3446-3451 [Download paper].
It was shown in this paper that the algorithm for constructing tree gatekeeping procedures used in Dmitrienko, Wiens, Tamhane and Wang (2007) may fail to satisfy the gatekeeping property. A general algorithm based on the monotonicity condition introduced by Hommel, Bretz and Maurer (2007) was proposed to rectify this problem.
Dmitrienko, A., Tamhane, A., Wiens, B. (2008). General multistage gatekeeping procedures. Biometrical Journal. To appear [Download paper].
Guilbaud, O. (2007). Bonferroni parallel gatekeeping: Transparent generalizations, adjusted p-values, and short direct proofs. Biometrical Journal. 49, 917-927.
This paper proposed a straightforward approach for setting up multistage/stepwise gatekeeping procedures based on the Bonferroni test. This method was later extended in Dmitrienko, Tamhane and Wiens (2007) to construct a general class of gatekeeping procedures based on arbitrary separable tests.
Hommel, G., Bretz, F., Maurer, W. (2007). Powerful short-cuts for multiple testing procedures with special reference to gatekeeping strategies. Statistics in Medicine. 26, 4063-4073.
This paper introduced a class of Bonferroni-based gatekeeping procedures that includes popular tests (e.g., Holm and fallback tests) and gatekeeping procedures (e.g., Bonferroni-based parallel gatekeeping procedure, Dmitrienko, Offen and Westfall, 2003).
Maurer, W., Hothorn, L., Lehmacher, W. (1995). Multiple comparisons in drug clinical trials and preclinical assays: a-priori ordered hypotheses. Biometrie in der chemisch-pharmazeutischen Industrie. Vollmar, J. (editor). Fischer Verlag: Stuttgart, Vol. 6, 3-18.
The paper discussed serial gatekeeping strategies in clinical trials.
Westfall, P.H., Krishen, A. (2001). Optimally weighted, fixed sequence, and gatekeeping multiple testing procedures. Journal of Statistical Planning and Inference. 99, 25-40.
The paper described the fixed-sequence test which serves as an example of a serial gatekeeping procedure (all tests must be significant in the primary family before one can perform secondary analyses).
Presentations
Dmitrienko, A., Wiens, B.L., Tamhane, A.C., Wang, X. Tree-structured gatekeeping procedures in clinical trials with multiple objectives. JSM. August 2006. Based on Dmitrienko, Wiens, Tamhane and Wang (2007) [Download slides].
Dmitrienko, A., Tamhane, A.C. Gatekeeping procedures based on Simes test with clinical trial applications. ENAR meeting. March 2007. Based on Dmitrienko, Tamhane and Wiens (2007) [Download slides].
Dmitrienko, A., Tamhane, A.C. Multi-stage gatekeeping procedures with clinical trial applications. MCP conference. July 2007. Based on Dmitrienko, Tamhane and Wiens (2007) [Download slides].
Dmitrienko, A., Choi, S., Tamhane, A.C., Johns, D. Reverse fixed-sequence procedures in clinical trials. JSM conference. August 2008 [Download slides].