User:Matthias Brendel/Scientific method

Scientific method is the method of science. In most cases it refers only to the method of empirical sciences, which include natural and social sciences, but do not include mathematics. We do not describe here the method of mathematics, and use the word "science" for "natural sciences" in the following. There is also a different opinion if the followings shall be understood in a descriptive way, i.e. describing the actual method of science, or prescriptive, i.e. describing, how science should be done ideally. In the second case it may be disputed, how close actual science is to this ideal description.

Some scholar doubt that one or some methods of science exists, may exist or may be described. Most of those thinkers, who accept that there is a method of science describe it like it follows in this introduction. We will also point to different opinions later.

The method of empirical sciences describes processes for investigating phenomena and acquiring new knowledge, as well as for correcting and integrating previous knowledge. It is based on observable, empirical evidence, and subject to laws or reasoning.

Scientific researchers propose specific hypotheses as explanations of phenomena, and carry out several observations or design experimental studies that test these predictions. These steps are repeated in order to create better theories. There is no perfect consensus what exactly the definition of "better" is for theories.

Theories that encompass whole domains of inquiry serve to bind more specific hypotheses together into logically coherent wholes. This in turn aids in the formation of new hypotheses.

Another basic expectation is that of making complete documentation of data and methodology of measurements available for careful scrutiny by other scientists and researchers.

Scientific publication of measurement and thopries also allows statistical measures of the reliability of the results to be established.

Elements of scientific method

There are multiple ways of outlining the basic method shared by all of the fields of scientific inquiry. The following examples are typical classifications of the most important components of the method on which there is very wide agreement in the scientific community and among philosophers of science, each of which are subject only to marginal disagreements about a few very specific aspects.

Experiments

Once predictions are made, they can be tested by experiments. If test results contradict predictions, then this constitutes an anomaly, and explanations may be sought. Sometimes experiments are conducted incorrectly and are at fault. If the results confirm the predictions, then the hypotheses are considered likely to be correct but might still be wrong and are subject to further testing.

Depending on the predictions, the experiments can have different shapes. It could be a classical experiment in a laboratory setting or a double-blind study. Even taking a plane from New York to Paris is an experiment which tests the aerodynamical hypotheses used for constructing the plane.

Scientists assume an attitude of openness and accountability on the part of those conducting an experiment. Detailed recordkeeping is essential, to aid in recording and reporting on the experimental results, and providing evidence of the effectiveness and integrity of the procedure. They will also assist in reproducing the experimental results. This tradition can be seen in the work of Hipparchus (190 BCE - 120 BCE), when determining a value for the precession of the Earth over 2100 years ago, and 1000 years before Al-Batani.

Note that experiments are not a necessary part of scientific method. There are purelly observational sciences, or fields of science, like history and astronomy, where only observation is possible.

Hypothesis development

A hypothesis is a suggested explanation of a phenomenon, or alternately a reasoned proposal suggesting a possible correlation between or among a set of phenomena.

Normally hypotheses have the form of a mathematical model. Sometimes, but not always, they can also be formulated as existential statements, stating that some particular instance of the phenomenon being studied has some characteristic and causal explanations, which have the general form of universal statements, stating that every instance of the phenomenon has a particular characteristic.

According Hans Reichenbach's distinction of context of justification and discovery, there is no single method describing how to create hypotheses. Scientists are free to use whatever resources they have — their own creativity, ideas from other fields, induction, Bayesian inference, and so on — to imagine possible explanations for a phenomenon under study. Charles Sanders Peirce, borrowing a page from Aristotle (Prior Analytics, 2.25) described the incipient stages of inquiry, instigated by the "irritation of doubt" to venture a plausible guess, as abductive reasoning. The history of science is filled with stories of scientists claiming a "flash of inspiration", or a hunch, which then motivated them to look for evidence to support or refute their idea. Michael Polanyi made such creativity the centrepiece of his discussion of methodology.

Karl Popper, following others, notably Charles Peirce, has argued that a hypothesis must be falsifiable, and that a proposition or theory cannot be called scientific if it does not admit the possibility of being shown false. It must at least in principle be possible to make an observation that would show the proposition to be false, even if that observation had not yet been made. Popper was criticised for this demarcation of scientific and non-sicientific hipotheses.

In general scientists tend to look for theories that are "elegant" or "beautiful". In contrast to the usual English use of these terms, they here refer to a theory in accordance with the known facts, which is nevertheless relatively simple and easy to handle. If a model is mathematically too complicated, it is hard to deduce any prediction. Note that 'simplicity' may be perceived differently by different individuals and cultures.

For example Linus Pauling proposed that DNA was a triple helix. Francis Crick and James Watson learned of Pauling's hypothesis, understood from existing data that Pauling was wrong and realized that Pauling would soon realize his mistake. So the race was on to figure out the correct structure. Except that Pauling did not realize at the time that he was in a race.

Predictions from the hypotheses

Any useful hypothesis will enable predictions, by reasoning including deductive reasoning. It might predict the outcome of an experiment in a laboratory setting or the observation of a phenomenon in nature. The prediction can also be statistical and only talk about probabilities. Note, that generally a single hipothesis is not enough for a prediction, but many hipotheses and assumptions are needed for deducing it. Also note, that a prediction must not always point to the future. There might be a prediction refering to an event in the past, or a property of an object in the past, which is unknown, but may be measured or observed later.

It is essential that the outcome be currently unknown. If the outcome is already known, it's called a consequence and should have already been considered while formulating the hypothesis.

If the predictions are not accessible by observation or experience, the hypothesis is not yet useful for the method, and must wait for others who might come afterward, and perhaps rekindle its line of reasoning. For example, a new technology or theory might make the necessary experiments feasible.

For example,when Watson and Crick hypothesized that DNA was a double helix, Francis Crick predicted that an X-ray diffraction image of DNA would show an X-shape. Also in their first paper they predicted that the double helix structure that they discovered would prove important in biology writing "It has not escaped our notice that the specific pairing we have postulated immediately suggests a possible copying mechanism for the genetic material".

Testing and improvement

The scientific process is iterative. At any stage it is possible that some consideration will lead the scientist to repeat an earlier part of the process. Failure to develop an interesting hypothesis may lead a scientist to re-define the subject they are considering. Failure of a hypothesis to produce interesting and testable predictions may lead to reconsideration of the hypothesis or of the definition of the subject. Failure of the experiment to produce interesting results may lead the scientist to reconsidering the experimental method, the hypothesis or the definition of the subject.

Other scientists may start their own research and enter the process at any stage. They might adopt the characterization and formulate their own hypothesis, or they might adopt the hypothesis and deduce their own predictions. Often the experiment is not done by the person who made the prediction and the characterization is based on experiments done by someone else. Published results of experiments can also serve as a hypothesis predicting their own reproducibility.

For example, after considerable fruitless experimentation, being discouraged by their superior from continuing, and numerous false starts, Watson and Crick were able to infer the essential structure of DNA by concrete modelling of the physical shapes of the nucleotides which comprise it. They were guided by the bond lengths which had been deduced by Linus Pauling and the X-ray diffraction images of Rosalind Franklin.

Peer review evaluation

Scientific journals use a process of peer review, in which scientists' manuscripts are submitted by editors of scientific journals to (usually one to three) fellow (usually anonymous) scientists familiar with the field for evaluation. The referees may or may not recommend publication, publication with suggested modifications, or, sometimes, publication in another journal. This serves to keep the scientific literature free of unscientific or crackpot work, helps to cut down on obvious errors, and generally otherwise improve the quality of the scientific literature. Work announced in the popular press before going through this process is generally frowned upon. Sometimes peer review inhibits the circulation of unorthodox work, and at other times may be too permissive. The peer review process is not always successful, but has been very widely adopted by the scientific community.

Documentation and replication

Sometimes experimenters may make systematic errors during their experiments, or (in rare cases) deliberately falsify their results. Consequently, it is a common practice for other scientists to attempt to repeat the experiments in order to duplicate the results, thus further validating the hypothesis.

As a result, experimenters are expected to maintain detailed records of their experimental procedures, in order to provide evidence of the effectiveness and integrity of the procedure and assist in reproduction. These procedural records may also assist in the conception of new experiments to test the hypothesis, and may prove useful to engineers who might examine the potential practical applications of a discovery.

Note that it is not possible for a scientist to record everything that took place in an experiment. He must select the facts he believes to be relevant to the experiment and report them. This may lead, unavoidably, to problems later if some supposedly irrelevant feature is questioned. For example, Heinrich Hertz did not report the size of the room used to test Maxwell's equations, which later turned out to account for a small deviation in the results. The problem is that parts of the theory itself need to be assumed in order to select and report the experimental conditions. The observations are sometimes hence described as being 'theory-laden'.

Models of scientific inquiry

The deductive-nomological model

The deductive-nomological model describes theories as a set of hypotheses (Hempel & Oppenheim 1948), which are used, to deduce some particular statements, which can be compared to empirical data. If the empirical data is already known, then this way the theory explains them, if they are nort yet known, then this is a prediction of the theory, which can be tested.

There are however different views about how the result of testing imfluences the improvement of the theory.

Positivist Model

Most of the logical positivists focused to justification, or confirmation. Rudolf Canrap developed a model (Carnap 1950), where the degree of confirmation of a theory is defined. Carnap stated that a potive test result increases this degree of confirmation, and after a reasonable degre, the theory can be considered to be confirmed. There is no exact rule about the treshhold of degree of conformation. Carnap admitted that other factors also influence, if we accept a theory, like simplicity. In his views theory acceptance is a practical issue. This view resembles to the model of Thomas Kuhn, who is however not considered to be a positivist (see later).

Critical Rationalist Model

Karl Popper's model (Popper 1934) focuses on falsification instead of confirmation. Popper emphasized that no general statement can be justified wtih a finite number of tests without doubt. Because of this, he focused on falsification. In Poppers view, scientist just create some hipotheses (it is not a question of scientific method, how they do it), which have to be falsified. After testing we either succed, to falsify the theory, or not. All the theories, whihc are not falsified, are possible theoiries to accept. Popper's method is called critical ratinalist, because the focus is on the critique of the theory.

Later he defined the empirical content of the theories, whoich is the set of deducable statements. I.e. the empirical content of a theory is proprotional to the number of phenomena, which can be explained or predicted by the theory. Popper's norm was to accept the theory, which is not falsified and has the largest empirical content. In Poppers view, a main virtue of a theory is how many phenomena it is excluding by making it impossible.

If a theory is falsified, then the test do not define, how the theory shall be changed. Tere can be many ways to save a theory. Popper described that in some cases ad hoc changes are done, which are only for saving the theory, but do destruct the empirical content of the theory. For example to hipothise a sorcerer, who distuirbed an experiment is an ad hoc hipothesis, since this sorcerer is not described by rules, so a lot of things may be possible after accepting a hipothesis like this. And as we saw it this is not good in Popper's model.

It was shown that one problem with Popper's theory is, that existential statements are not falsifiable. On the opther hand, as we saw, universal statements are not proovable. Both kind of statements are used in a theory, and this way neither justification, nor falsification is possible for all the statements. This means that both conformation and falsification is used in the method of science. Moreover, they are the two sides of testing.

So, if Popper's and the logical positivist model is unified, it gives us quite a good model for the logical part of scientific method.

Pragmatic model

Charles Peirce considered scientific inquiry to be a species of the genus inquiry, which he defined as any means of fixing belief, that is, any means of arriving at a settled opinion on a matter in question. He observed that inquiry in general begins with a state of uncertainty and moves toward a state of certainty, sufficient at least to terminate the inquiry for the time being. He graded the prevalent forms of inquiry according to their evident success in achieving their common objective, scoring scientific inquiry at the high end of this scale. At the low end he placed what he called the method of tenacity, a die-hard attempt to deny uncertainty and fixate on a favored belief. Next in line he placed the method of authority, a determined attempt to conform to a chosen source of ready-made beliefs. After that he placed what might be called the method of congruity, also called the a priori, the dilettante, or the what is agreeable to reason method. Peirce observed the fact of human nature that almost everybody uses almost all of these methods at one time or another, and that even scientists, being human, use the method of authority far more than they like to admit. But what recommends the specifically scientific method of inquiry above all others is the fact that it is deliberately designed to arrive at the ultimately most secure beliefs, upon which the most successful actions can be based.

Historic Turn

After Thomas Kuhn's famous work, Structure of Scientific Revolutions (Kuhn 1962) the main focus shifted from methodology of science to history of science. Kuhn gave a description of scientific processed, based more on history than on logic. This means also a shift from prescriptive to descriptive philosophy. Kuhn's theory distinguishes between normal science and revolutionary periods. Normal science is made based on a so called paradigm, which is a big theoretical framework for developing theopries. For example Newtonian physics was a paradigm. Scientific revolutions are paradigm changes. According to Kuhn, there are some scientific values, like simplicity, which govern theory development, but scientific revolutions are like geschtalt switch. There is no exact methodology to compare theories before and after a revolution, since the different theories are like written in different languages, and noone can understand perfectly both languages. This is the incommensurability thesis of Kuhn. Kuhn was regarded as relativist because of this, but he himself rejected this. It is not clear, how relativist Kuhn was, since the mentioned constant values of science suggest also some ant-relativism. After Kuhn however the focus of history of science was permanent until the focus shifted even more to sociology of science. Kuhn himself take part also in this second shift.

In his 1958 book, Personal Knowledge (Polanyi 1958), the chemist and philosopher Michael Polanyi criticized the view that the scientific method is purely objective and generates objective knowledge. Polanyi considered this to be a misunderstanding of the scientific method, and argued that scientists do and must follow personal passions in appraising facts and in choosing which questions to investigate. He concluded that a structure of liberty is essential for the advancement of science -- that the freedom to pursue science for its own sake is a prerequisite for the production of knowledge.

Paul Feyerabend denied that science is genuinely a methodological process. In his book Against Method (Feyerabend 1975) he argued that scientific progress is not the result of applying any particular rules. Instead, he concluded almost that 'anything goes', in that for any particular 'rule' there are abundant examples of successful science that have proceeded in a way that seems to contradict it. To Feyeraband, there is no real difference between science and other areas of human activity characterised by reasoned thought. To be correct, we have to mention that Feyerabend's early view were close to Poppers, since hi was one of his students.

Sociological Turn

The strong programme is a variety of the sociology of scientific knowledge (SSK) particularly associated with David Bloor, Barry Barnes, Harry Collins, Donald MacKenzie, and John Henry (Bloor et al. 1996). The largely Edinburgh-based school of thought has illustrated how the existence of a scientific community, bound together by allegiance to a shared paradigm, is a pre-requisite for normal scientific activity.

The strong programme proposed that both 'true' and 'false' scientific theories should be treated the same way -- that is, symmetrically. Both are caused by social factors or conditions, such as cultural context and self interest. All human knowledge, as something that exists in the human cognition, must contain some social components in its formation process. The presence of social factors alone is not enough to falsify a scientific theory.

Models based on research programmes

At the end of tho era of logical positivism, philosophy of science focused not only on theories, but bigger parts of scientific research. They named them as research programmes, which are quite close to Kuhn's paradigm. Some post-positivistic philosophers started a detailed analysis of these research programmes. In a research program a main theory is developed and several variants are developed. We have seen models on acceptance of theories, but the new question was the acceptance or rejection of a research program. In the context of research programmes, rejection means that the scientist does not work any mor in the research program. As it was shown however, it is possible for a researcher to work in even rival research programmes, since this is not a question of beleive, but a mor practical question. Larry Laudan and Imre Lakatos tried to describe some criteria oiof how good a research program is. Lakatos defined progressive and degenerative research programmes. Lakatos (Lakatos 1977), as a student of Popper connected these concept to the empirical content of the theory. A research program is degenerative, if the empirical content of the theory gets lower. Lakatos stated taht there is no rule, of which degree of degeneration is a must for quit the program. That is why this is a practical issue.

Larry Laudan (laudan 1977) defined the degree of progress of a theory by how much problems it can solve over the time. Again, a scientist estimates this measure and decides which program to quit, which to maintain. It is again apractical decision.

Evolutionary models

In 1972, Stephen Toulmin published Human Understanding (Toulmin 1972) which asserts that conceptual change is an evolutionary process. In contrast to Kuhn’s revolutionary model, Toulmin proposed an evolutionary model of conceptual change comparable to Darwin’s model of biological evolution. Toulmin states that conceptual change involves the process of innovation and selection. Innovation accounts for the appearance of conceptual variations, while selection accounts for the survival and perpetuation of the soundest conceptions. The soundest concepts will survive the forum of competition as replacements or revisions of the traditional conceptions.

The later Karl Popper also developed a view (Popper 1984), which describes the growth of knowledge as an evolutionary process. He used Newtonian physics as an example of a body of theories so thoroughly confirmed by testing as to be considered unassailable, but which were nevertheless overturned by Einstein's bold insights into the nature of space-time. For the evolutionary epistemologist, all theories are true only provisionally, regardless of the degree of empirical testing they have survived.

Computational approaches

Many subspecialties of applied logic and computer science, to name a few, artificial intelligence, computational learning theory, inferential statistics, and knowledge representation, are concerned with setting out computational, logical, and statistical frameworks for the various types of inference involved in scientific inquiry, in particular, hypothesis formation, logical deduction, and empirical testing. Some of these applications draw on measures of complexity from algorithmic information theory to guide the making of predictions from prior distributions of experience, for example, see the complexity measure called the speed prior from which a computable strategy for optimal inductive reasoning can be derived.

Philosophical issues

Problem of demarcation

The problem of evaluating a system of thought with regard to its status as science is often called the demarcation problem. The criteria for a system of assumptions, methods, and theories to qualify as science vary in their details from application to application, but they typically include (1) the formulation of hypotheses that meet the logical criterion of contingency, defeasibility, or falsifiability and the closely related empirical and practical criterion of testability, (2) a grounding in empirical evidence, and (3) the use of scientific method. The procedures of science typically include a number of heuristic guidelines, such as the principles of conceptual economy or theoretical parsimony that fall under the rubric of Ockham's razor. A conceptual system that fails to meet a significant number of these criteria is likely to be considered non-scientific. The following is a list of additional features that are highly desirable in a scientific theory.

• Consistent. Generates no obvious logical contradictions, and 'saves the phenomena', being consistent with observation.

• Parsimonious. Economical in the number of assumptions and hypothetical entities.

• Pertinent. Describes and explains observed phenomena.

• Falsifiable and testable. See Falsifiability and Testability.

• Reproducible. Makes predictions that can be tested by any observer, with trials extending indefinitely into the future. (This may not be expected in all empirical sciences. )

• Correctable and dynamic. Subject to modification as new observations are made.

• Integrative, robust, and corrigible. Subsumes previous theories as approximations, and allows possible subsumption by future theories. See Correspondence principle

• Provisional or tentative. Does not assert the absolute certainty of the theory.

Cuurently, there is no consensus among scientists and philosophers about an exact formulation of criteria for the demarcation of science and non-science. Some of these criteria werre suggested by some philosophers, but rejected by other, significant philosophers.

For example Popper was in favour of falsifiability as a demarcation, while the Vienna Circle was in favour of confirmation as a demarcation not only for science, but for meaningful empirical statements.

Problem of foundation

Protocol sentence debate

The problem of foundation is a traditional problem of philosophy of science. The question is the basis of our scientifical knowledge, is it a set of statements, or just nonverbal empiria? Is it an absolute basis (foundationalism) or not (epistemological relativism? What kind of the statements are can be a relative or absolute basis for science? Statements about our observations (phenomenalism) or statements about physical objects (physicalism)?

This question is also connected with the different theories of truth. Since a correspondentist is more likelly to be foundationalist, while a coherentist can not be a foundationalist.

The most well known debate about the foundation of science is the so called protocol sentence debate in the Vienna Circle, which took place aproximatelly in the publications of 1928-1934.

Schlick proposed a foundationalist view, Neurath an anti-foundationalist one, and Carnap was in between. All of them argued against all of the others. Sclick took a phenomenalist and Neurath a physicalist standpoint, and Carnap as well known was more phenomenalist in the early ages, and more physicalist in the later ages, while being theretically neutral about this question. This was a practical question for him.

Theories of truth

Correspondence theories claim that true beliefs and true statements correspond to the actual state of affairs. It is a traditional model which goes back at least to some of the classical Greek philosophers such as Socrates, Plato, and Aristotle. This class of theories holds that the truth or the falsity of a representation is determined in principle solely by how it relates to objective reality, by whether it accurately describes that reality. For example, there is a true distance to the moon when we humans attempt to go there, and this true distance is necessary to know so that the journey can be successfully made. Among the philosophers who grappled with this problem is Alfred Tarski and Karl Popper.

For coherence theories in general, truth requires a proper fit of elements within a whole system. Very often, though, coherence is taken to imply something more than simple logical consistency. For example, the completeness and comprehensiveness of the underlying set of concepts is a critical factor in judging the validity and usefulness of a coherent system

Coherence theories distinguish the thought of rationalist philosophers, particularly of Spinoza, Leibniz, and G.W.F. Hegel, along with the British philosopher F.H. Bradley. They have found a resurgence also among several proponents of logical positivism, notably Otto Neurath and Carl Hempel.

There are also some other theories of truth.

Unity of science: is there a global method?

The unity of science is a thesis in philosophy of science that says that all the sciences form a unified whole. One aspect of unity is a unified method.

The unity of science thesis is most famously clarified and tentatively argued for by the logical positivists, Ludwig von Bertalanffy, Paul Oppenheim and Hilary Putnam. It is most famously argued against by Jerry Fodor. As Feyerabend is against method, he of course would also be against the existence of a unified method. As we already mentioned, Thomas Kuhn's standpoint is unclear.

Debate of induction versus falsification

Inductive reasoning maintains that if a situation holds in all observed cases, then the situation holds in all cases. So, after completing a series of experiments that support the Third Law, one is justified in maintaining that the Law holds in all cases.

One answer has been to conceive of a different form of rational argument, one that does not rely on deduction. Deduction allows one to formulate a specific truth from a general truth: all crows are black; this is a crow; therefore this is black. Induction somehow allows one to formulate a general truth from some series of specific observations: this is a crow and it is black; that is a crow and it is black; therefore all crows are black.

The problem of induction is one of considerable debate and importance in the philosophy of science: is induction indeed justified, and if so, how?

Another way to distinguish science from pseudoscience (e.g. astronomy from astrology), first formally discussed by Karl Popper in 1919-20 and reformulated by him in the 1960s, is falsifiability. This principle states that in order to be useful (or even scientific at all), a scientific statement ('fact', theory, 'law', principle, etc) must be falsifiable, that is, able to be tested and proven wrong.

It is easy to confirm or verify nearly every theory — if we look for confirmations. Confirmations are significant only if they are the result of risky predictions; that is, if, unenlightened by the theory, we should have expected an event which was incompatible with the theory — an event which would have refuted the theory.

References

1. Bloor, D., Barnes, B. and Henry, J. (1996), Scientific Knowledge: A Sociological Analysis, Chicago: University of Chicago Press
2. Carnap, R. (1950), Logical Foundations of Probability, University of Chicago Press
3. Feyerabend P. (1975), Against Method, Outline of an Anarchistic Theory of Knowledge Reprinted, Verso, London, UK, 1978
4. Hempel, G., and Oppenheim, P. (1948), Studies in the Logic of Explanation, Philosophy of Science
5. Kuhn, T. (1962), The Structure of Scientific Revolutions, University of Chicago Press, Chicago, IL.
6. Lakatos,I. (1977), The Methodology of Scientific Research Programmes: Philosophical Papers Volume 1. Cambridge: Cambridge University Press
7. Laudan, L. (1977), Progress and Its Problems: Toward a Theory of Scientific Growth, Berkeley/Los Angeles/London: University of California Press.
8. Polanyi, M. (1958), Personal Knowledge. Towards a Post Critical Philosophy, London: Routledge.
9. Popper, K. (1934), Logik der Forschung, Julius Springer Verlag, Vienna, 1935.
10. Popper, K. (1984), Evolutionary Epistemology, in Evolutionary Theory: Paths into the Future, (ed.) J. W. Pollard, London: John Wiley & Sons Ltd.
11. Toulmin, S. (1972), HUman Understanding, Princeton, N.J.: Princeton