Hypothesis Testing Steps

Hypothesis testing is the process of using statistics to determine the probability that a specific hypothesis is true. It evaluates two mutually exclusive statements about a population to determine which statement is best supported by the sample data. The process of hypothesis testing consists of four main steps:

Step 1: State the Null and Alternate Hypothesis

During this stage we formulate two hypotheses to test:

Null hypothesis (Ho): A hypothesis that proposes that the observations are a result of a pure chance and there is no effect relationship or difference between two or more groups.

Alternative hypothesis (Ha): A hypothesis that proposes that the sample observations are influenced by some non-random cause and there is an effect or difference between two or more groups. It is the claim you are trying to prove with an experiment.

Example: A local Italian restaurant has an average delivery time of 45 minutes with a standard deviation of 5 minutes. The restaurant has received some complaints from its customers and has decided to analyze the last 40 orders. The average delivery time for these 40 orders was found to be 48 minutes. Conduct the appropriate test at a significance level of 5% to decide whether the delivery times have increased.

Let μ be the average delivery time for the restaurant (population mean)

Null hypothesis: H0: μ=45

Alternate hypothesis: H1: μ > 45

Step 2: Set the Significance Level (α)

Is defined as the probability that the population parameter lies outside its confidence interval. If the confidence intervals 95%, the level of significance is 0.05, or there is a 5% chance that the population parameter does not lie within the confidence interval calculated from the sample

For our example, we will set a significant level of α = 0.05.

Step 3: Compute the test statistic

The test statistic is a mathematical formula that allows researchers to determine the likelihood of obtaining sample outcomes if the null hypothesis were true. The value of the test statistic is used to make a decision regarding the null hypothesis.

Examples of test:

T-Test: Compares the mean of two given samples.

ANOVA Test: Compares three or more samples with a single test.

Chi-Square: Compares categorical variables.

Pearson Correlation: Compares two continuous variables.

Select the appropriate Hypothesis Test

Number of samples: 1

Sample size: n=40 (Large)

What we are testing: Whether there is a difference between the sample mean (x = 48) and the population mean (μ=45)

Population standard deviation (σ=5) is known

We therefore select the one-sample z-test based on the preceding data.

Obtain the test statistic and p-value, with the help of the following

equation:

Substituting the values x = 48, μ=45, σ = 5, and n=40:

z=3.7947

Calculate the p-value corresponding to this z-value using the stats.norm.cdf function:

CODE:

import SciPy. Stats as stats

stats.norm.cdf(z)

Output:

p-value = 0.999

. Compare the p-value with the level of significance (0.05):

Since the calculated p-value is >α, we fail to reject the null hypothesis.

Step 4: Making an inference

Compare the calculated p-value with the given level of significance α. if the p-value is less than or equal to α, we reject the null hypothesis and if it is greater than α, we fail to reject the null hypothesis.

In the above example we fail to reject the null hypothesis thus there is no significant difference, at a level of 0.05, between the average delivery time of the sample and the historical population average.

Two-Decision Errors:

when we decide to reject or fail to reject the null hypothesis, two types of errors might occur.

Type I error: A Type I error occurs when we reject a null hypothesis when it is true. The probability of committing a Type I error is the significance level α.

Type II error. A Type II error occurs when we fail to reject a null hypothesis that is false. The probability of committing a Type II error is called Beta and is often denoted by β. The probability of not committing a Type II error is called the Power of the test.

References:

https://machinelearningmastery.com/statistical-hypothesis-tests/

https://www.analyticsvidhya.com/blog/2020/12/quick-guide-to-perform-hypothesis-testing/