Test Statistic Type of Test t-statistic t-testRegression test F-statistic ANOVAMANOVAANCOVA z-statistic z-test x2-statistic Chi-square test Some of the most common teststatistics and their correspondingstatistical tests or models.
A statistical hypothesis test is a method of statistical inference used to decide whether the data provide sufficient evidence to reject a particular hypothesis. A statistical hypothesis test typically involves a calculation of a test statistic. Then a decision is made, either by comparing the test statistic to a critical value or equivalently by evaluating a p-value computed from the test statistic. Roughly 100 specialized statistical tests are in use.
Definition of terms
The goal of a hypothesis test is to establish whether certain properties of a statistical population are true by examining sample data. Typically, the population is modelled by a random variable whose distribution has unknown parameters. For example, a medical trial may wish to establish whether a particular drug is effective in treating high blood pressure, with "the change in blood pressure observed in a patient who takes the drug" being the random variable. An example hypothesis could be "the mean change in blood pressure is zero" or "the mean change in blood pressure is negative". In general, any statement about the parameters describing a population can be a hypothesis (but not a statement about the sample).
The test compares two hypotheses: a default null hypothesis (denoted H0) and its negation, the alternative hypothesis (H1). It is usually consistent with the research hypothesis because it is constructed from literature review, previous studies, etc. However, the research hypothesis is sometimes consistent with the null hypothesis. The null hypothesis and alternative hypothesis are two mutually exclusive statements.
"The statement being tested in a test of statistical significance is called the 'null hypothesis'. The test of significance is designed to assess the strength of the evidence against the null hypothesis. Usually, the null hypothesis is a statement of 'no effect' or 'no difference'."
If the sample data are consistent with the null hypothesis, then you do not reject the null hypothesis; if the sample data are inconsistent with the null hypothesis, then you reject the null hypothesis and conclude that the alternative hypothesis is true.
Typically the test will select a null hypothesis that the intervention being studied has no effect, or that the population parameter takes some "obvious" value. A test statistic is computed from the given sample data, and the tester calculates the conditional probability of observing a value at least this extreme, supposing the null hypothesis is true. If this probability (called the p-value) is less than, or equal to, the significance level of the test (denoted ), then the null hypothesis is rejected. The test does not conclude that the null hypothesis is false, or that the probability that the null hypothesis is true is less than .
Because it is usually impossible to definitely establish whether the hypothesis being tested is true or false from a sample, the conclusion of a hypothesis test is not certain to be correct. There are two possible classes of error:
A type I error, in which the null hypothesis is rejected despite the null hypothesis being true, with probability . This is the same as the significance level of the test. A type II error, in which the null hypothesis is accepted despite the alternative hypothesis being true, with probability . The quantity is called the power of the test. Some further definitions: : Any hypothesis which specifies the population distribution completely. Composite hypothesis: Any hypothesis which does not specify the population distribution completely. Positive data: Data that enable the investigator to reject a null hypothesis.
s of a statistical test are the boundaries of the acceptance region of the test. The acceptance region is the set of values of the test statistic for which the null hypothesis is not rejected. Depending on the shape of the acceptance region, there can be one or more than one critical value. / : The set of values of the test statistic for which the null hypothesis is rejected. Size: For simple hypotheses, this is the test's probability of incorrectly rejecting the null hypothesis. The false positive rate. For composite hypotheses this is the supremum of the probability of rejecting the null hypothesis over all cases covered by the null hypothesis. The complement of the false positive rate is termed specificity in biostatistics. ("This is a specific test. Because the result is positive, we can confidently say that the patient has the condition.") See sensitivity and specificity and type I and type II errors for exhaustive definitions. : A predecessor to the statistical hypothesis test (see the Origins section). An experimental result was said to be statistically significant if a sample was sufficiently inconsistent with the (null) hypothesis. This was variously considered common sense, a pragmatic heuristic for identifying meaningful experimental results, a convention establishing a threshold of statistical evidence or a method for drawing conclusions from data. The statistical hypothesis test added mathematical rigor and philosophical consistency to the concept by making the alternative hypothesis explicit. The term is loosely used for the modern version which is now part of statistical hypothesis testing. Conservative test: A test is conservative if, when constructed for a given nominal significance level, the true probability of incorrectly rejecting the null hypothesis is never greater than the nominal level. Exact test
In the case of a scalar parameter, there are four principal types of alternative hypothesis: Point. Point alternative hypotheses occur when the hypothesis test is framed so that the population distribution under the alternative hypothesis is a fully defined distribution, with no unknown parameters; such hypotheses are usually of no practical interest but are fundamental to theoretical considerations of statistical inference and are the basis of the Neyman–Pearson lemma. One-tailed directional. A one-tailed directional alternative hypothesis is concerned with the region of rejection for only one tail of the sampling distribution. Two-tailed directional. A two-tailed directional alternative hypothesis is concerned with both regions of rejection of the sampling distribution. Non-directional. A non-directional alternative hypothesis is not concerned with either region of rejection; rather, it is only concerned that null hypothesis is not true.
A statistical hypothesis test compares a test statistic (z or t for examples) to a threshold. The test statistic (the formula found in the table below) is based on optimality. For a fixed level of Type I error rate, use of these statistics minimizes Type II error rates (equivalent to maximizing power). The following terms describe tests in terms of such optimality:
Most powerful test: For a given size or significance level, the test with the greatest power (probability of rejection) for a given value of the parameter(s) being tested, contained in the alternative hypothesis. Uniformly most powerful test (UMP)
There are several types of hypothesis: Simple hypothesis Any hypothesis that specifies the population distribution completely. For such a hypothesis the sampling distribution of any statistic is a function of the sample size alone. Composite hypothesis Any hypothesis that does not specify the population distribution completely.. Example: A hypothesis specifying a normal distribution with a specified mean and an unspecified variance.
The simple/composite distinction was made by Neyman and Pearson.
Exact hypothesis Any hypothesis that specifies an exact parameter value is an exact hypothesis, also known as a point hypothesis. Example: . Inexact hypothesis Those specifying a parameter range or interval. Examples: ; . Fisher required an exact null hypothesis for testing (see the quotations below).
History
The history of the null and alternative hypotheses has much to do with the history of statistical tests. While hypothesis testing was popularized early in the 20th century, early forms were used in the 1700s. The first use is credited to John Arbuthnot (1710), followed by Pierre-Simon Laplace (1770s), in analyzing the human sex ratio at birth; see .
1778: Pierre Laplace compares the birthrates of boys and girls in multiple European cities. He states: "it is natural to conclude that these possibilities are very nearly in the same ratio". Thus, the null hypothesis in this case that the birthrates of boys and girls should be equal given "conventional wisdom".
1900: Karl Pearson develops the chi squared test to determine "whether a given form of frequency curve will effectively describe the samples drawn from a given population." Thus the null hypothesis is that a population is described by some distribution predicted by theory. He uses as an example the numbers of five and sixes in the Weldon dice throw data.
1904: Karl Pearson develops the concept of "contingency" in order to determine whether outcomes are independent of a given categorical factor. Here the null hypothesis is by default that two things are unrelated (e.g. scar formation and death rates from smallpox). The null hypothesis in this case is no longer predicted by theory or conventional wisdom, but is instead the principle of indifference that led Fisher and others to dismiss the use of "inverse probabilities".
The concept of an alternative hypothesis in testing was devised by Jerzy Neyman and Egon Pearson, and it is used in the Neyman–Pearson lemma. It forms a major component in modern statistical hypothesis testing. However it was not part of Ronald Fisher's formulation of statistical hypothesis testing, and he opposed its use. In Fisher's approach to testing, the central idea is to assess whether the observed dataset could have resulted from chance if the null hypothesis were assumed to hold, notionally without preconceptions about what other models might hold. Modern statistical hypothesis testing accommodates this type of test since the alternative hypothesis can be just the negation of the null hypothesis.
Modern origins and early controversy Modern significance testing is largely the product of Karl Pearson (p-value, Pearson's chi-squared test), William Sealy Gosset (Student's t-distribution), and Ronald Fisher ("null hypothesis", analysis of variance, "significance test"), while hypothesis testing was developed by Jerzy Neyman and Egon Pearson (son of Karl). Ronald Fisher began his life in statistics as a Bayesian (Zabell 1992), but Fisher soon grew disenchanted with the subjectivity involved (namely use of the principle of indifference when determining prior probabilities), and sought to provide a more "objective" approach to inductive inference.Raymond Hubbard, M. J. Bayarri, P Values are not Error Probabilities . A working paper that explains the difference between Fisher's evidential p-value and the Neyman–Pearson Type I error rate .
Fisher emphasized rigorous experimental design and methods to extract a result from few samples assuming Gaussian distributions. Neyman (who teamed with the younger Pearson) emphasized mathematical rigor and methods to obtain more results from many samples and a wider range of distributions. Modern hypothesis testing is an inconsistent hybrid of the Fisher vs Neyman/Pearson formulation, methods and terminology developed in the early 20th century.