The Mann–Whitney test (also called the Mann–Whitney–Wilcoxon (MWW/MWU), Wilcoxon rank-sum test, or Wilcoxon–Mann–Whitney test) is a nonparametric statistical test of the null hypothesis that randomly selected values X and Y from two populations have the same distribution.
The value of U calculated by the test can be converted to a measure of effect size by dividing it by the maximum value of U, which is the product of the sizes of the two samples being compared. This measure is the probability that the value of a random observation from the higher group will be greater than that of a random observation from the lower group.
Nonparametric tests used on two dependent samples are the sign test and the Wilcoxon signed-rank test.
Assumptions and formal statement of hypotheses Henry Mann and Donald Ransom Whitney developed the Mann–Whitney U test under the assumption of continuous responses with the alternative hypothesis being that one distribution is stochastically greater than the other. That is to say that the probability of a random observation from one group being larger than a random observation from the other group is 0.5. There are many other ways to formulate the null and alternative hypotheses such that the Mann–Whitney U test will give a valid test.
A very general formulation is to assume the following conditions:
All the observations from both groups are independent of each other, The responses are at least ordinal (i.e., one can at least say, of any two observations, which is the greater), Under the null hypothesis H0, the distributions of both populations are identical., See Table 2.1 of Pratt (1964) "Robustness of Some Procedures for the Two-Sample Location Problem." Journal of the American Statistical Association. 59 (307): 655–680. If the two distributions are normal with the same mean but different variances, then Pr[X > Y] = Pr[Y < X] but the size of the Mann–Whitney test can be larger than the nominal level. So we cannot define the null hypothesis as Pr[X > Y] = Pr[Y < X] and get a valid test. The alternative hypothesis H1 is that the distributions are not identical.
Under the general formulation, the test is only consistent when the following occurs under H1:
The probability of an observation from population X exceeding an observation from population Y is different (larger, or smaller) than the probability of an observation from Y exceeding an observation from X; i.e., or .
Under more strict assumptions than the general formulation above, e.g., if the responses are assumed to be continuous and the alternative is restricted to a shift in location, i.e., , we can interpret a significant Mann–Whitney U test as showing a difference in medians. This practice relies on strict assumptions about the data distribution. When this assumption does not hold, it is simple to demonstrate that the test produces a statistically significant result when the medians of the two groups are in fact identical. If a test of medians is required, quantile regression explicitly tests this.
Under this location shift assumption, we can also interpret the Mann–Whitney U test as assessing whether the Hodges–Lehmann estimate of the difference in central tendency between the two populations differs from zero. The Hodges–Lehmann estimate for this two-sample problem is the median of all possible differences between an observation in the first sample and an observation in the second sample.
Otherwise, if both the dispersions and shapes of the distribution of both samples differ, the Mann–Whitney U test fails a test of medians. It is possible to show examples where medians are numerically equal while the test rejects the null hypothesis with a small p-value.
The Mann–Whitney U test / Wilcoxon rank-sum test is not the same as the Wilcoxon signed-rank test, although both are nonparametric and involve summation of ranks. The Mann–Whitney U test is applied to independent samples. The Wilcoxon signed-rank test is applied to matched or dependent samples.
U statistic Although Mann and Whitney's original paper described the test as one of stochastic superiority – in other words, as a test of whether an observation from one group was likely to be larger than an observation from the other group, the test statistic, U, does not give this probability. This probability can be simply calculated by dividing U by its maximum value, which is the product of the two sample sizes.
Let be group 1, an i.i.d. sample from , and be group 2, an i.i.d. sample from , and let both samples be independent of each other. The corresponding Mann–Whitney U statistic is defined as the smaller of:
with being the sums of the ranks in groups 1 and 2, after ranking all samples from both groups such that the smallest value obtains rank 1 and the largest rank .Boston University (SPH), 2017
Area-under-curve (AUC) statistic for ROC curves The U statistic is related to the area under the receiver operating characteristic curve (AUC):
Note that this is the same definition as the common language effect size, i.e. the probability that a classifier will rank a randomly chosen instance from the first group higher than a randomly chosen instance from the second group.Fawcett, Tom (2006); An introduction to ROC analysis, Pattern Recognition Letters, 27, 861–874.
Because of its probabilistic form, the U statistic can be generalized to a measure of a classifier's separation power for more than two classes:
Where c is the number of classes, and the Rk,ℓ term of AUCk,ℓ considers only the ranking of the items belonging to classes k and ℓ (i.e., items belonging to all other classes are ignored) according to the classifier's estimates of the probability of those items belonging to class k. AUCk,k will always be zero but, unlike in the two-class case, generally , which is why the M measure sums over all (k,ℓ) pairs, in effect using the average of AUCk,ℓ and AUCℓ,k.
Calculations The test involves the calculation of a statistic, usually called U, whose distribution under the null hypothesis is known: In the case of small samples, the distribution is tabulated For sample sizes above ~20, approximation using the normal distribution is fairly good.
Alternatively, the null distribution can be approximated using permutation tests and Monte Carlo simulations.
Some books tabulate statistics equivalent to U, such as the sum of ranks in one of the samples, rather than U itself.
The Mann–Whitney U test is included in most statistical packages.
It is also easily calculated by hand, especially for small samples. There are multiple ways of doing this.
Method one:
For comparing two small sets of observations, a direct method is quick, and gives insight into the meaning of the U statistic, which corresponds to the number of wins out of all pairwise contests (see the tortoise and hare example under Examples below). For each observation in one set, count the number of times this first value wins over any observations in the other set (the other value loses if this first is larger). Count 0.5 for any ties. The sum of wins and ties is U (i.e., ) for the first set. U for the other set is the converse (i.e., ).
Method two:
For larger samples: Assign numeric ranks to all the observations (put the observations from both groups to one set), beginning with 1 for the smallest value. Where there are groups of tied values, assign a rank equal to the midpoint of unadjusted rankings (e.g., the ranks of are , where the unadjusted ranks would be ). Now, add up the ranks for the observations which came from sample 1. The sum of ranks in sample 2 is now determined, since the sum of all the ranks equals where N is the total number of observations. U is then given by:
where n1 is the sample size for sample 1, and R1 is the sum of the ranks in sample 1.
Note that it doesn't matter which of the two samples is considered sample 1. An equally valid formula for U is
The smaller value of U1 and U2 is the one used when consulting significance tables. The sum of the two values is given by
Knowing that and , and doing some algebra, we find that the sum is .
Properties
The maximum value of U is the product of the sample sizes for the two samples (i.e., ). In such a case, the "other" U would be 0.
Examples
Illustration of calculation methods
Suppose that Aesop is dissatisfied with his classic experiment in which one tortoise was found to beat one hare in a race, and decides to carry out a significance test to discover whether the results could be extended to tortoises and hares in general. He collects a sample of 6 tortoises and 6 hares, and makes them all run his race at once. The order in which they reach the finishing post (their rank order, from first to last crossing the finish line) is as follows, writing T for a tortoise and H for a hare: T H H H H H T T T T T H What is the value of U? Using the direct method, we take each tortoise in turn, and count the number of hares it beats, getting 6, 1, 1, 1, 1, 1, which means that . Alternatively, we could take each hare in turn, and count the number of tortoises it beats. In this case, we get 5, 5, 5, 5, 5, 0, so . Note that the sum of these two values for , which is . Using the indirect method: Rank the animals by the time they take to complete the course, so give the first animal home rank 12, the second rank 11, and so forth. The sum of the ranks achieved by the tortoises is . Therefore (same as method one). The sum of the ranks achieved by the hares is , leading to .
Example statement of results In reporting the results of a Mann–Whitney U test, it is important to state: A measure of the central tendencies of the two groups (means or medians; since the Mann–Whitney U test is an ordinal test, medians are usually recommended) The value of U (perhaps with some measure of effect size, such as common language effect size or rank-biserial correlation). The sample sizes The significance level. In practice some of this information may already have been supplied and common sense should be used in deciding whether to repeat it. A typical report might run, "Median latencies in groups E and C were 153 and 247 ms; the distributions in the two groups differed significantly (Mann–Whitney , , two-tailed)." A statement that does full justice to the statistical status of the test might run, "Outcomes of the two treatments were compared using the Wilcoxon–Mann–Whitney two-sample rank-sum test. The treatment effect (difference between treatments) was quantified using the Hodges–Lehmann (HL) estimator, which is consistent with the Wilcoxon test. This estimator (HLΔ) is the median of all possible differences in outcomes between a subject in group B and a subject in group A. A non-parametric 0.95 confidence interval for HLΔ accompanies these estimates as does ρ, an estimate of the probability that a randomly chosen subject from population B has a higher weight than a randomly chosen subject from population A. The median [quartiles] weight for subjects on treatment A and B respectively are 147 [121, 177] and 151 [130, 180] kg. Treatment A decreased weight by HLΔ = 5 kg (0.95 CL [2, 9] kg, , )."
However it would be rare to find such an extensive report in a document whose major topic was not statistical inference.
Normal approximation and tie correction
For large samples, U is approximately normally distributed. In that case, the standardized value
where mU and σU are the mean and standard deviation of U, is approximately a standard normal deviate whose significance can be checked in tables of the normal distribution. mU and σU are given by
and
The formula for the standard deviation is more complicated in the presence of tied ranks. If there are ties in ranks, σ should be adjusted as follows:
where the left side is simply the variance and the right side is the adjustment for ties, tk is the number of ties for the kth rank, and K is the total number of unique ranks with ties.