Applied statistics

Applied statistics provide both a familiar source of information and a notorious source of error and misinformation. Errors commonly arise from misplaced confidence in intuitive interpretations, but some of the most serious have arisen from misuse by mathematicians and other professionals. Deliberate misinterpretation of statistics by politicians and marketing professionals is so much a popular commonplace that its genuine use is often treated with suspicion. To those unfamiliar with it, statistics can seem impenetrably arcane, but its pitfalls can be avoided given a grasp of a few readily understood concepts.

(terms shown in italics are defined in the glossary on the related articles subpage).

Overview: the basics

Statistics are observations that are recorded in numerical form. It is essential to their successful handling to accept that statistics are not facts and therefore incontrovertible, but observations about facts and therefore fallible. The reliability of the information that they provide depends not only upon their successful interpretation, but also upon the accuracy with which the facts are observed and the extent to which they truly represent the subject matter of that information. An appreciation of the means by which statistics are collected is thus an essential part of the understanding of statistics and is least as important as a familiarity with the tools that are used in its interpretation.

Although the derivation of those tools involved advanced mathematics, the laws of chance on which much of statistics theory is based are no more than a formalisation of intuitive concepts, and the use of the resulting algorithms and computer software requires only a grasp of basic mathematical principles

The collection of statistics

The methodology adopted for the collection of observations has a profound influence upon the problem of extracting useful information from the resulting statistics. That problem is at its easiest when the collecting authority can minimise disturbing influences by conducting a "controlled experiment". A range of more complex methodologies (and associated software packages) referred to as "the design of experiments" is available for use when the collecting authority has various lesser degrees of control. The object of the design in each case is to facilitate the testing of an hypothesis by helping to exclude the influence of factors that the hypothesis does not take into account. At the furthest extreme from the controlled experiment, no such help can be provided through the physical elimination of extraneous influences - and, if they are to be eliminated, it must be done after they have been identified by a purely analytical technique termed the "analysis of variance" For example, the rôle of the authorities that collect economic statistics is necessarily passive, and the testing of economic hypotheses involves the use of a version of the analysis of variance termed "econometrics" (sometimes confused with economic modelling, which is a purely deterministic technique).

The taking of samples reduces the cost of collecting observations and increases the opportunities to generate false information. One source of error arises from the fact that every time a sample is taken there will be a different result. That source of error is readily quantified as the sample's standard error, or as the confidence interval within which the mean observation may be expected to lie. That source of error cannot be eliminated, but it can be reduced to an acceptable level by increasing the size of the sample. The other source of error arises from the likelihood that the characteristics of the sample differ from those of the "population" that it is intended to represent. That source of error does not diminish with sample size and cannot be estimated by a mathematical formula. Careful attention to what is known about the composition of the "population" and the reflection of that composition in the sample is the only available precaution. The composition of the respondents to an opinion poll, for example, is normally chosen to reflect as far as possible the composition of the intended "population" as regards sex, age, income bracket etc. The remaining difference is referred as the sample bias, and undetected bias has sometimes been a major source of misinformation.

The use by statisticians of the term "population" refers, not to people in general, but to the category of things or people about which information is sought. A precise definition of the target population is an essential starting point in a statistical investigation, and also a possible source of misinformation. Difficulty can arise when, as often happens, the definition has to be arbitrary. If the intended population were the output of the country's farmers, for example, it might be necessary to draw an arbitrary dividing line between farmers and owners of smallholdings such as market gardens. Any major change over time in the relative output of farm products by the included and excluded categories might then lead to misleading conclusions. Technological change, such as the change from typewriters to word processors has sometimes given rise to serious difficulties in the construction of the price indexes used in the correction of GDP for inflation. Since there is no objective solution to those problems, it is inevitable that national statistics embody an element of judgement exercised by the professional statisticians in the statistics authorities.

National statistics also embody some further arbitrary or subjective adjustments that are intended to increase their usefulness, but constitute another possible source of error. The early provisional release of results involves the arbitrary imputation of figures in the place of late or invalid responses to enquiries, and substantial amendments to the published figures sometimes then continue for some months after their initial release. Decisions are also taken from time to time to exclude or include transitory changes such as whether the transfer abroad for repair of a multi-million dollar airliner is to be recorded as an export and its return as an import. Judgmental adjustments are also made to resolve conflicts between duplicate measures of national output. National statistics authorities employ research teams for the purpose of maintaining and improving the reliability of their output in those and other respects.

Politicians in the major democracies have seldom had any influence upon the collection and publication of national statistics, and most countries have sought to allay suspicions to the contrary by delegating those functions to public bodies that are free from possible government influence.

Statistical inference

Although statistics is sometimes thought of as a branch of mathematics, some of its findings can be successfully interpreted by verbal inference, and there are others that only require the use of a few simply-expressed rules (such as those set out in paragraph 1 of the tutorials subpage). However, there is evidence to suggest that most people confidently prefer an intuitive approach, unaware of the "probability blindness" that is characteristic of the humam brain. Educated professionals seem not to be immune from overconfidence in that respect, of which there have been several examples involving the medical profession> For example, the following question was put to the staff and students of the Harvard Medical School:

If a test of a disease that has a prevalence rate of 1 in 1000 has a false positive rate of 5%, what is the chance that a person who has been given a positive result actually has the disease.

- 45 per cent gave the intuitive answer of 95% when the true answer is 2% (see paragraph 2 of the tutorials subpage)
That was merely a test. Overconfidence by an eminent expert cost Sally Clark her liberty (see paragraph 3 of the tutorials subpage)