Non-Parametric Tests

Non-parametric tests are applied in hypothesis testing when the data does not satisfy the assumptions necessary for parametric tests, such as normality or equal variances. These tests are especially helpful for analyzing ordinal data, small sample sizes, or data with outliers.

Common Non-Parametric Tests

Lets see some commonly used non-parametric tests,

1. Mann-Whitney U Test

• The Mann-Whitney U test is a non-parametric alternative to the independent t-test. It assesses whether there is a difference between two independent groups on a continuous or ordinal variable.
• If we want to compare two independent groups, the Mann-Whitney U test is an excellent choice as it assesses whether two groups have different distributions.

Lets see an example of customer satisfaction score,

import numpy as np
import scipy.stats as stats

region_a = [85, 88, 90, 92, 95]
region_b = [78, 80, 83, 85, 87]

stat, p_value = stats.mannwhitneyu(region_a, region_b, alternative='two-sided')

print(f"U-statistic: {stat}, p-value: {p_value}")

Output

U-statistic: 23.5, p-value: 0.02780296243469296

Since the p-value is less than 0.05, we reject the null hypothesis, indicating a significant difference between the two regions.

2. Wilcoxon Signed-Rank Test

• The Wilcoxon signed-rank test is used for comparing two related samples to assess whether their population mean ranks differ. It's the non-parametric counterpart to the paired t-test.
• When comparing two related samples, the Wilcoxon Signed-Rank test should be used, as it evaluates the difference in ranks for paired data.

Lets understand it with an example showing the evaluation of a Training program,

pre_training = [78, 82, 85, 88, 92]
post_training = [82, 85, 88, 90, 94]

stat, p_value = stats.wilcoxon(pre_training, post_training)

print(f"Statistic: {stat}, p-value: {p_value}")

Output

Statistic: 0.0, p-value: 0.0625

The output suggests there is no significant difference due to the training program.

3. Shapiro-Wilk Test

• The Shapiro-Wilk test is used to check if a sample comes from a normally distributed population. It's particularly useful for small sample sizes.
• To check for normality in small samples, we use the Shapiro-Wilk test, which is sensitive to deviations from normality.

Lets see an example of testing normality of data,

data = np.random.normal(loc=0, scale=1, size=100)

stat, p_value = stats.shapiro(data)

print(f"Shapiro-Wilk statistic: {stat}, p-value: {p_value}")

Output

Shapiro-Wilk statistic: 0.9837449032862893, p-value: 0.2569742070326989

The output suggests the sample follows a normal distribution.

4. Anderson-Darling Test

• The Anderson-Darling test is used to check if a sample comes from a specified distribution (e.g., normal distribution). It gives more weight to the tails of the distribution than other tests, such as the Kolmogorov-Smirnov test.
• A test for normality that is more sensitive to tail deviations than the Kolmogorov-Smirnov test, ideal for checking if a sample follows a specific distribution.

Lets see normality testing example,

data = np.random.normal(loc=0, scale=1, size=100)

result = stats.anderson(data, dist='norm')

print(
    f"Statistic: {result.statistic}, Critical values: {result.critical_values}")

Output

Statistic: 0.3747489555840531, Critical values: [0.555 0.632 0.759 0.885 1.053]

Since the test statistic is less than the critical value, we fail to reject the null hypothesis, suggesting the data follows the normal distribution.

5. Kruskal-Wallis H Test

• The Kruskal-Wallis H test is a non-parametric method for testing whether samples originate from the same distribution. It's used when comparing more than two independent groups.
• When dealing with more than two independent groups, the Kruskal-Wallis H test provides an appropriate non-parametric alternative to ANOVA.

Lets see an example of comparing test scores across multiple teaching methods,

method_a = [88, 90, 92, 94, 96]
method_b = [78, 80, 82, 84, 86]
method_c = [85, 87, 89, 91, 93]

stat, p_value = stats.kruskal(method_a, method_b, method_c)

print(f"H-statistic: {stat}, p-value: {p_value}")

Output

H-statistic: 9.620000000000005, p-value: 0.008147859697679966

Since p-value less than 0.05, at least one teaching method differs significantly.

6. Friedman Test

• The Friedman test is a non-parametric test for detecting differences in treatments across multiple test attempts. It's used for repeated measures on the same subjects.
• In cases of repeated measures, where the same subjects are tested under different conditions, the Friedman test is an ideal choice.

Lets see an example comparing ratings of different products by the same group,

product_1 = [4, 5, 4, 3, 5]
product_2 = [3, 4, 3, 4, 4]
product_3 = [5, 5, 5, 5, 5]

stat, p_value = stats.friedmanchisquare(product_1, product_2, product_3)

print(f"Statistic: {stat}, p-value: {p_value}")

Output

Statistic: 7.111111111111117, p-value: 0.0285655007845503

The output indicates a significant difference in ratings.

7. Kolmogorov-Smirnov Test

• The Kolmogorov-Smirnov (KS) test compares a sample with a reference probability distribution or two samples. It's often used to test whether a sample follows a specified distribution (e.g., normal distribution) or to compare two samples for differences in their distributions.
• If we're comparing two distributions or testing the goodness-of-fit of a sample to a known distribution, the Kolmogorov-Smirnov test is appropriate due to its flexibility in comparing sample distributions.

Lets see an example of comparing sample distribution to normal distribution,

import numpy as np
import scipy.stats as stats

data = np.random.normal(loc=0, scale=1, size=100)

stat, p_value = stats.kstest(data, 'norm')

print(f"KS-statistic: {stat}, p-value: {p_value}")

Output

KS-statistic: 0.08648372023957773, p-value: 0.4194177770213061

Since p-value is greater than 0.05, the sample follows normal distribution.

8. Lilliefors Test

• The Lilliefors test is a variant of the Kolmogorov-Smirnov test and is used for testing the normality of data. It's particularly useful when the parameters of the normal distribution (mean and variance) are unknown.
• If we have large datasets, the Lilliefors test might be preferred as it is more robust when parameters are unknown.

Lets see a Testing normality example,

from statsmodels.stats.diagnostic import lilliefors

data = np.random.normal(loc=0, scale=1, size=100)

stat, p_value = lilliefors(data)

print(f"Lilliefors statistic: {stat}, p-value: {p_value}")

Output

Lilliefors statistic: 0.11077802859590069, p-value: 0.00429933056976003

The output indicates the sample is not normally distributed.