Michael Last
National Institute of Statistical Sciences
Research Triangle Park
Pooled ANOVA
I will present Pooled ANOVA, an efficient method to screen for significant
factors. Problems in Computer Simulations, and in tuning computer programs,
often involve a large number of factors, not all of which have a significant
impact on the outcome or performance measure. Especially when interactions
are considered, possible spaces to explore for significant effects become
quite large. Our idea is to pool several factors together, and
simultaneously test them for significance. If the pool is significant, we
retain the factors for further testing. If not, we either declare none of
the factors as
significant, or retain them for follow-up tests, depending on our
assumptions and stage of testing. The rarer the significant factors, the
bigger the savings. Conducting fewer tests helps also mitigate multiple
testing issues. For instance, with 100 significant factors out of 1,000,
and keeping 9 degrees of freedom for estimating residuals, we were able to
find 98/100 significant factors with 3 false positives in 618 tests, whereas
we would need 1,009 tests to achieve similar results with a regular ANOVA,
and expect 45 false-positives. We also tested our procedure in a setting
that maximized collisions between factors with opposite effects, and found
97/100 significant factors with no false-positives in 751 tests.
Back to Colloquium Series