cshalizi + multiple_comparisons 9
The optimal discovery procedure: a new approach to simultaneous significance testing - Storey - 2007 - Journal of the Royal Statistical Society: Series B (Statistical Methodology) - Wiley Online Library
february 2012 by cshalizi
"The Neyman–Pearson lemma provides a simple procedure for optimally testing a single hypothesis when the null and alternative distributions are known. This result has played a major role in the development of significance testing strategies that are used in practice. Most of the work extending single-testing strategies to multiple tests has focused on formulating and estimating new types of significance measures, such as the false discovery rate. These methods tend to be based on p-values that are calculated from each test individually, ignoring information from the other tests. I show here that one can improve the overall performance of multiple significance tests by borrowing information across all the tests when assessing the relative significance of each one, rather than calculating p-values for each test individually. The ‘optimal discovery procedure’ is introduced, which shows how to maximize the number of expected true positive results for each fixed number of expected false positive results. The optimality that is achieved by this procedure is shown to be closely related to optimality in terms of the false discovery rate. The optimal discovery procedure motivates a new approach to testing multiple hypotheses, especially when the tests are related. As a simple example, a new simultaneous procedure for testing several normal means is defined; this is surprisingly demonstrated to outperform the optimal single-test procedure, showing that a method which is optimal for single tests may no longer be optimal for multiple tests. Connections to other concepts in statistics are discussed, including Stein's paradox, shrinkage estimation and the Bayesian approach to hypothesis testing."
to:NB
statistics
hypothesis_testing
multiple_comparisons
february 2012 by cshalizi
xkcd: Significant
april 2011 by cshalizi
... and this goes on the office doors of statisticians everywhere.
funny:geeky
funny:because_its_true
xkcd
hypothesis_testing
multiple_comparisons
cartoons
april 2011 by cshalizi
[citation needed]» Blog Archive » fourteen questions about selection bias, circularity, nonindependence, etc.
june 2010 by cshalizi
We don't appear to subscribe, but presumably I could write some of them for a copy... --- ETA: Received, thanks to T.D.
fmri
multiple_comparisons
neuroscience
neural_data_analysis
statistics
estimation
hypothesis_testing
june 2010 by cshalizi
"Neural correlates of interspecies perspective taking in the post-mortem Atlantic Salmon: An argument for multiple comparisons correction"
fmri neuroscience bad_data_analysis funny:academic funny:malicious statistics hypothesis_testing multiple_comparisons to_teach:data-mining salmon to:blog
september 2009 by cshalizi
fmri neuroscience bad_data_analysis funny:academic funny:malicious statistics hypothesis_testing multiple_comparisons to_teach:data-mining salmon to:blog
september 2009 by cshalizi
Why we (usually) don’t have to worry about multiple comparisons
march 2008 by cshalizi
My initial reaction is one of skepticism, despite my respect for Andy. To work through.
statistics
multiple_comparisons
hierarchical_models
gelman.andrew
yajima.masano
via:arthegall
have_read
hill.jennifer
march 2008 by cshalizi
related tags
bad_data_analysis ⊕ cartoons ⊕ data_mining ⊕ estimation ⊕ fmri ⊕ funny:academic ⊕ funny:because_its_true ⊕ funny:geeky ⊕ funny:malicious ⊕ gelman.andrew ⊕ have_read ⊕ hierarchical_models ⊕ hill.jennifer ⊕ hypothesis_testing ⊕ in_NB ⊕ markov_models ⊕ multiple_comparisons ⊖ multiple_testing ⊕ neural_data_analysis ⊕ neuroscience ⊕ salmon ⊕ statistics ⊕ to:blog ⊕ to:NB ⊕ to_teach:data-mining ⊕ via:arthegall ⊕ xkcd ⊕ yajima.masano ⊕Copy this bookmark: