Statistics -- Hypothesis Testing

1. There are four steps in data driven decision making. First: Formulate a hypothesis, Second: Find the right test for your hypothesis, Third: Execute the test, Fourth: Make a decision based on the result.
2. Hypothesis: It is an idea that can be tested.
3. There are two hypotheses that are made. The null hypothesis denoted H0 and the alternative hypothesis denoted H1 or HA. The null hypothesis is the one to be tested and the alternative is everything else.
4. In statistics, the null hypothesis is the statement we are trying to reject. Therefore, the null hypothesis is the present state of affairs while the alternative is our personal opinion.
5. Significance level is denoted by Alpha and is the probability of rejecting the null hypothesis if it is true. Typical values for Alpha are 0.01, 0.05 and 0.1. It is a value that you select based on the certainty you need in most cases. The 0.05 is the most common used value.
6. The Z-test formula: Z equals the sample mean minus the hypothesized mean divided by the standard error.
7. How big should Z be to reject the null hypothesis? — There is a cutoff line since we are conducting a two sided or a two tailed test. There are two cutoff lines one on each side. When we calculate z we will get a value. If this value fall into the middle part then we cannot reject the null. If it falls outside in the shaded region then we reject the null hypothesis. That is why the shaded part is called rejection region.
8. If the test statistic is bigger than the cut-off z-score, we would reject the null otherwise we wouldn’t. If the test value falls into the rejection region, you will reject the null hypothesis. You will reject the test’ is a meaningless phrase, as the test is not accepted or rejected; It is executed. The outcome of the test is rejection or acceptance of the null hypothesis. Similarly, rejecting the significance level has no meaning.
9. In general, we have two types of errors. Type 1 error and type 2 error where sounds a bit boring.
10. Type 1 error is when you reject a true null hypothesis. It is also called a false positive. The probability of making this error is alpha. The level of significant since you the researcher choose the alpha. The responsibility for making this error lies solely on you.
11. Type 2 error is when you accept a false null hypothesis. It is also called a false negative. The probability of making this error is denoted by Bayda. Bayda depends mainly on sample size and magnitude of the effect.
12. One minus Bayda is called The Power of the test most often researchers increase the power of a test by increasing the sample size.
13. A level of significant after which we can no longer do it. This is the right moment to introduce a measure called the P-value. This is the most common way to test hypotheses instead of testing it pre-assigned levels of significance. We can find the smallest level of significant at which we can still reject the null hypothesis given the observed samples statistics.
14. Where and how are p-value used? — Most statistical software calculates p-values for each test. Researcher decides significance post-factum. P-values are usually found with 3 digits after the dot. The closer to 0, your p value is the more significant is the result you have obtained.
15. P value is an extremely powerful measure as it works for all distributions.
16. If the p-value was lower than the level of significance, you reject the null hypothesis. You would normally use the p-value in the presence of a digital medium.