Statistics In Plain English By Timothy Urdan

The p-value (while it is one of the most widely-used and important concepts in statistics) is actually widely misunderstood.

Today we’ll talk about what it is, and how to obtain it.

(If you’re in a statistics class, or using this stuff out there in the real world, consider ordering “Statistics in Plain English” by Timothy Urdan).

The P-Value: What it really means

The P-value is one of the biggest sources of confusion in statistics, and it’s not uncommon for researchers to use the wrong definition: is not when they compute it, but when they think about it. So read this definition carefully:

The P-value is the probability that our data would be at least this inconsistent with the hypothesis, assuming the hypothesis is true.

This sounds simple

If you fail to reject a hypothesis, it means that your experiment is inconclusive. To accept a hypothesis, you need to understand the statistical power of a test, something we’re not going to get into today.

So, in order to say something useful, we need to choose a null hypothesis and an alternative hypothesis. The null hypothesis is typically the accepted status quo. The alternative hypothesis is usually the one we’re more interested in. When dealing with P-values alone, the alternative hypothesis needs to be the only possible alternative. That way, if we reject the null hypothesis, we can safely accept the alternative hypothesis, and state a conclusive result.

For example, let’s say we wanted to know if a new drug had an influence on IQ. These are what we would want to pick as our null and alternative

- Alternative hypothesis – The average IQ of a population that uses the drug will be different from the average IQ of a population that does not use the drug.

These are the only two options, so if we reject the null hypothesis, we can accept the alternative hypothesis.

In order to reject the null hypothesis, you need to pick a level of statistical significance. By default, this is 5 or 1 percent. If you get a smaller P-value than your significance level, you can deny the null hypothesis.

Performing the Test

This is where things get muddy, and it’s why there’s really no predefined “formula” that will give you your P-value. The way you set up the test is going to depend on what your hypotheses are, and sometimes on how skewed your data is.

For the remainder of this article, we’re going to talk about some of the most common tests, how they work, and how you find the P-value. Let’s get started.

Z Test for a Population Mean

When to use the test: You want to test whether your sample average is statistically consistent with a hypothesized population average.

Conditions that must be