Safe and Efficient Algorithms for Monte Carlo Tests

Many recently suggested statistical hypothesis tests are so-called Monte-Carlo tests, meaning that the test decision (or p-value) is determined via Monte Carlo simulation. Examples of these tests include bootstrap tests and permutation tests. In practice, Monte Carlo tests are usually implemented via a fixed number of Monte Carlo samples. This has the problem that the test decision might be influenced by the simulation error. The project is concerned with the safe and computationally efficient implementation of Monte Carlo tests. Safe meaning that the test decision is affected by simulation error only up to a guaranteed (small) error bound. Furthermore, the project will consider algorithms for the evaluation of Monte Carlo tests, that is for level or power studies.A particular focus of the project will be on the implementation of double (or iterated) bootstrap tests. Theoretical and practical evidence shows that these tests are superior to simple bootstrap tests. However, they are rarely used in practice. This is because they are computationally (very) expensive due to the required nested simulation. This projects aims to remove these hurdles by developing more efficient and safer algorithms. The algorithms developed in this package will be made available as an R-package.

Applied statisticians in industry or medical research will benefit from a more dependable implementation of Monte Carlo tests. There is quite some theoretical and practical evidence that bootstrap or double bootstrap tests are superior to classical tests. The newly developed tests will enable the use of these better tests - hopefully leading to more reliable decisions. Ultimately the wider public will benefit from this improved decision making. To ensure that applied statisticians can use the newly developed algorithms, they will be made available as a freely available R-package. Furthermore, the aim is to publicize the result not only in the classical statistical journals but also in more applied journals such as Significance .