Test Statistics

Lastly, with your testable hypotheses and significance level stated, you'll need to figure out which statistical test is best suited to your study design (next section). All statistical tests calculate something called a "test statistic", and it is ultimately this quantity that is used to evaluate whether your criterion for significance is met. Specifically, based on the significance level you chose, and your sample size \(n\), you are provided - in "statistical tables" - a "critical value" of the test statistic that, if met or exceeded, leads to the rejection of the null hypothesis in favour of the alternative.

The critical value demarcates an outcome that is sufficiently convincing - based on your significance level chosen - that it is unlikely to have arisen solely from sampling error. After stating your significance level, and deciding which test is appropriate (see below), it is good practice to write down what the critical value of the corresponding test statistic is. Using your experiment data, you will then calculate (or the computer will provide) an observed value of the test statistic, and assess this against the critical value.

In BIOL116 you'll be using a Shiny app to conduct your analyses, and it will tell you whether you've exceeded the critical value or not. Thus, there is no need to write down the critical value in the planning stage.

P-value

When you conduct a statistical test using the computer or app, it will provide you with something called a "P-value". The magnitude of this P-value depends on how close or far your calculated test statistic value is to your critical value. They key thing to remember is that if you chose a significance level of 0.05 (the standard), then a P-value of 0.05 or lower indicates a "statistically significant" outcome, i.e. evidence consistent with your alternative hypothesis. Thus, this is complementary to evaluating your calculated test statistic value against the critical value of the test statistic.