Scientific inquiry is a process that enables us to gain new knowledge with greatest likelihood of it being correct. As a starting point, a hypothesis, i.e., a proposed explanation of a phenomenon based on limited evidence, is proposed. This hypothesis is then tested by experiment, by gathering data, or by reference to existing experiments and data. If this supports the hypothesis, then it is proposed as a theory, the research is written up, subjected to peer review or checks by other specialists in the field, and published.
Together, the accepted theories and the way in which they are developed form a paradigm. Wikipedia defines a paradigm as “a distinct set of concepts or thought patterns, including theories, research methods, postulates, and standards for what constitute legitimate contributions to a field.” An initial hypothesis is usually consistent with other theories in the prevailing paradigm, but sometimes not.
The philosopher of science, Thomas Kuhn, argued that science does not progress in a linear manner, but that the prevailing paradigm is relatively resistant to change, and undergoes periodic revolutions when the amount of contradictory evidence reaches a critical mass. In part, this resistance is brought about by human nature. I think it is fair to say that scientists and academic organisations are more objective than most. Nevertheless, they are still human, and the following traits will tend to introduce a bias in favour of the existing paradigm.
At the level of the individual, considerable effort is put into developing a theory. It becomes a part of his or her mental schema and is resistant to change. Furthermore, reputation and livelihood depend on any theory promoted as being correct. There will, therefore, be a personal vested interest in its acceptance and a resistance to any challenges. Obviously, acceptance is more likely if it is consistent with the prevailing paradigm.
Every organisation, including academia, is organised hierarchically, i.e., there are higher status individuals who command groups of lower status ones, and this assembles into a pyramidical structure. This structure is maintained by a process of trade or negotiation. Typically, a senior may delegate some of his power, influence, reputation, wealth, etc. in return for a subordinate’s support. Status is gained from the resources we control, and so, if one has many supporting subordinates, then one has greater status. With increasing status comes increasing power, influence, reputation, and wealth. Thus, there can be a personal vested interest in a theory and in the prevailing paradigm. Independence can be had by refusing to trade, but obviously this will result in a personal loss.
Experimental proof must be reproducible, and data must be readily accessible, so that others can check it. However, in the social sciences, there are several difficulties with this:
- Variables in the social sciences are often qualitative rather than quantitative, i.e., they are either true or not. Even if a variable is quantitative, there can be difficulties in measuring it. In physics it is relatively easy to determine the mass of an object and apply a number to it. However, in the social sciences, there are, for example, no objective ways of quantifying personality traits. These are probably more acquired than inherited, and so, more to do with the brain’s software than its hardware. This means, that they are unlikely to be revealed by MRI scans, etc. There are also issues with the reliability of questionnaires in which we report on our own personality.
- Experimental proof based on historical analysis can be flawed. History comprises reports and interpretations that can be highly subjective. In addition, there are often only a small number of examples to which we can refer.
- Experimental proof which relies on direct experience can also be highly subjective and is not reproducible.
- Experimental design is constrained by ethical considerations. For example, if one wished to prove that an event x always causes war, then it would be unethical to cause such an event.
- Finally, culture can have a strong influence on both hypotheses and the evidence used in their proof. For this reason, cross-cultural studies are becoming increasingly common. It is also the case that knowledge is a part of culture and so, new knowledge can alter culture. Thus, a feedback loop exists, i.e., culture affects social theory, which can, in turn, affect culture.
So, the paradigm for the physical sciences is very different to that for the social sciences. An analogy might be to regard the former as a criminal law case in which it is necessary to prove the defendant guilty. The latter can be regarded as a civil case, that is judged on the balance of evidence. This does, of course, mean that theories in the social sciences are less likely to be true than those in the physical sciences. Nevertheless, their pursuit is worthwhile because an understanding of human nature does, in general, and in the long term at least, appear to lead to an improvement in our circumstances.