Many struggle with discussions of null and alternative hypotheses. The logic behind phrasing research questions in that way can be a bit unintuitive. The logical involves what you can or cannot prove given an if-then statement, and I'll put a paragraph about that down at the bottom. In the meantime, here is a much easier way to understand what is going on:

(If you missed it, here is Stats Help Part 1: Independent and Dependent Variable)

**What Type of World do we Live in?**

*When you have a hypothesis that involves very large and ambiguous populations, you will never be certain about what is true for the entire population.*

What can you do then? You can gather a sample, and try to make an inference about the larger population. (That is why it is called "inferential statistics.") For example, you can gather 100 students who did poorly in English classes, and 100 students who did excellently in English classes, and you can have both group correct several documents, quantifying their success as editors. The groups will have different success rates, to be sure, because any time you gather data in enough detail you will find a difference between your groups. But remember, you don't care about your samples for their own sake, you care about using the results to guess what type of world you live in. What are the options:

1) You could live in a

*boring world*, a world in which English grades do

*not*relate to editing skills. In such a world your samples would still have different scores, because of "sampling error", but you would expect those differences to be very small.

2) You could live in an

*interesting world*, a world in which English grades

*do*relate to editing skills. In such a world your samples should have larger differences.

Alas, we have no idea how big a difference we would expect if we lived in an interesting world, so it is really hard to say what would expect to find if we were in that world. We can, however, predict how our samples should look if we were in the boring world. With the help of a few assumptions, we can say how likely we are to find group differences of different sizes given the size of our sample and the assumption that the world is boring. We call the probability of a given difference a "p-value", and small p-values indicate that it is very unlikely that we are living in a boring world.

**Interpreting Results of a Study**

If we see a large p-value, we say the result is not significant. The safe bet under such circumstances is to say that we "failed to reject the null hypothesis." We didn't prove there was no difference, but we really have no way of proving that there is absolutely no difference for certain. Rather, a high p-value means that even though we did find a difference between the groups (because you always do), the result is so small it could easily have been created by chance sampling error in a boring world. For example, if the p-value is .33, that means we would find a difference at least that big by random chance 1 out of 3 times when there was no relationship between the variables in the wider world. A better study in the future might reveal an affect, but our study did not find evidence for such.

If we see p < .05, we

*reject the null hypothesis*and

*conclude that we live in an interesting world*, where the variables we are interested in actually do affect each other. We call that a statistically significant result, which does not mean that our result is "important", it only means that it is unlikely to be due to random variation in a boring world.

We can never be completely certain of our conclusions, of course. No matter what our results are, we will still be making probabilistic claims, based on sample data. Also, having a significant result doesn't mean that the populations are different in exactly the same way our samples are. For example, if our good-grade students caught 10 more mistakes on average than our poor-grade students, we couldn't conclude that we would find an exactly 10 point difference if we measured hundreds of thousands of students, or millions of students. That said, if the result is statistically significant, we would conclude that good-grade students were some amount better, in the much larger world, which we could never completely measure.

**The Logic Behind Focusing on the Null Hypothesis**

*not*carrying an umbrella, you can conclude that it is

*not*raining.

The logic is not as air tight when we are making statistical claims, but its form is the same: If there was no effect, we will find a small difference between groups. We did not find a small difference between groups, and based on that we conclude that there is an effect.

## No comments:

## Post a Comment