An Example

Let's say our farmer is concerned primarily about fruit set. It seems reasonable then to test for fruit set. Again, fruit set could be defined in many ways - average biomass per fruit, average count per plant etc. And we may or may not be interested in each of these outcomes. In either case, a more concise, testable research question might then look like

Will the application of fertilizer x increase the quantity of fruit set of Vigna radiata?

Not only can we test this, it identifies exactly what we’re interested in testing, and it’s concise, which means that we can then readily propose a hypothesis and null hypothesis to address it:

• H_o: fertilizer x will increase the quantity of fruit set of Vigna radiata.
• H_a: fertilizer x will have no impact on the quantity of fruit set of Vigna radiata.

Reproducibility, meta-analyses, and the evidence base

When we reproduce a study, we always know that there is a chance of error or bias resulting from our sample not being truly representative of its population, for any number of reasons including sampling error, lack of power etc. This is why we should never rely on the findings of just one study.

A meta-analysis is a study of already conducted studies to try and determine if across a series of studies addressing the same research question there is enough agreement in the findings to accept one conclusion, even though this conclusion may be contradicted by individual studies.

Reproducibility enables this aggregation of findings, helping to sift through studies that have suffered from systematic error. To do this well, meta-analyses rely on documentation and homogeneity; studies that use similar methods, instruments, and techniques to address the same question and describe in detail how this was done. This is because comparing two studies of the same phenomenon with two different research questions and two different methodological approaches and data collection tools is extremely confounding and limiting.

Meta-analyses are based on extremely comprehensive literature reviews, reviews that attempt to uncover all literature – published and unpublished – addressing a given research question. Your research question not only informs your hypothesis and study design, it also frames your title and abstract, whether for a lab report, poster, or one day a manuscript. Expressing your research question in a way that clearly and succinctly outlines the variables you plan to test makes the inclusion of your results in a meta-analysis more likely, as your work will be more easily discovered and identified.

In fact, with this in mind, if you were conducting your mung bean research for a particular plot of land in a particular region, this might impact the variables you choose to work with, and you might end up with a still more concise research question that would allow for identification of potential homogeneity and then for comparing your data against other similar studies in a meaningful way. So, for example, in the Okanagan, your research question might be adjusted to

Will the application of fertilizer x increase the quantity of fruit set of Vigna radiata in a sandy loam soil of the BC Okanagan Valley?