Scientific principle 1: Pose significant questions that can be investigated empirically.
If questions are to be scientific ones, they must be significant and testable. In addition, scientific theories must be able to be disproved. According to Popper (1957/1996), a scientific theory is one that is falsifiable or refutable. In other words, a scientific theory cannot be proven, only disproved. The reason for not being able to prove a theory is that we will never know all of the variables that can affect the results of an experiment supporting a theory or from which the theory was developed. Simply stated, errors are always made in scientific research conclusions. We do not know where or when those errors will occur, only that they can occur at some point.
So, we set out to disprove theories, not to prove them; in order to disprove a theory, we must be able to test the theory. Many psychological theories do not fit within this criterion. For example, Popper (1957/1996) indicates that the psycho-analytic theories of Freud and Adler were not testable, not irrefutable. It is not possible, for example, to disprove the existence of the id, ego, and superego.
To ask, "What is the most effective approach to teach reading?" is a significant question. The answer to this question would further our knowledge regarding an important educational approach. This question can be answered empirically. Empirical refers to the gaining of information through our senses. Put another way, empiricism refers to the observation of the phenomenon under study. Thus, achievement scores can be collected and analyzed to determine if an instructional approach improved reading skills. With the use of an appropriate research design, this question can be answered. On the other hand, the question, "Should students be allowed to pray in school?" is not a scientific one. This is not to say anything about the societal importance of this issue. However, it is not one that can be answered through science; it must be answered in other ways, such as through philosophy or law.
Scientific principle 2: Link research to relevant theory.
Scientific theories are models that help explain a phenomenon. These theories help us understand the world around us as well as lead to new scientific questions. Well known scientific theories include the theory of evolution and theory of relativity. Theories are developed through one of two logic methods, or a combination of the two. First, theories may be developed after preliminary observations are made of the world around us. These theories then drive future research. Theory-driven research results in the development of research hypotheses that are essentially a prediction on what the results of an investigation will be. This logic method is termed deductive.
For example, the information processing (cognitive) theory, based on the computer model, indicates that sensory stimuli enter through our sensory registers. Once in our sensory registers, the information will either be lost (decay) within 1-2 seconds, or will move into our short term (or working) memory. The stimuli will move into our working memory due to attention to the stimuli and will also be transformed by our perceptions. Once in our working memory, the information will decay rather quickly unless we mentally manipulate the stimuli, such as through rehearsal (as when you repeat a novel phone number over and over again when making a call). These stimuli are then moved into the working memory via further rehearsal, mnemonic devices, etc. The theory indicates that the working memory holds a limited amount of information at a time (5-9 bits of information; e.g., g-d-o is three bits of information). Based on this theory, we could pose the following hypothesis: "Chunking of information (e.g., rather than teaching g-d-o in isolation, teach the letters as a whole word—dog--to take up one bit of information), we can increase what an individual can remember when learning new information." This type of theory development and research approach occurs through deductive logic [i.e., moving from the general (theory) to the specific (individual research hypotheses)].
Once research findings are gathered, the results will either provide confirmatory evidence for the theory or evidence that will refute the theory. In either case, the theory should evolve over a period of time based on the data generated. This type of theory development can be termed grounded theory development, since the theory is grounded in a great deal of research.
A second logic method is termed inductive. The inductive approach moves from the specific to the general. With this approach, rather simple basic phenomena are studied with a simple research question to guide the researcher. As research progresses, the research questions become more and more complex. After a time, very complex phenomena are studied and a general theory can be developed based on the knowledge accumulated.
This approach was used by B. F. Skinner. For example, we may have relatively simple research questions such as, "How do reinforcers and punishers affect our behavior?" We may also ask, "What are the side effects of the use of punishment?" We may also begin to approach more complex questions such as, "How do different schedules of reinforcement (e.g., reinforcement of every response or every other response) affect the response rates of an individual's behavior?" Finally, we may get to very complex behaviors such as how individuals acquire language. Over time, a theory will develop (in this case a behavioral theory) that will guide further research.
In between these logic forms is some combination of the two. Some researchers use this combination to guide their research. Think of logic forms on a continuum from pure deduction to pure induction. All research falls somewhere along that continuum. What determines where one falls on this continuum is the amount of research or observation that takes place before the theory is developed, and to what extent the theory guides early research.
Regardless of the scientific logic that is used, all scientific research is linked to relevant theory guiding future research.
Scientific principle 3: Use methods that permit direct investigation of the question.
This principle essentially involves two aspects. First, appropriate research designs must be used to answer the research question. A question such as, "Does reading method X result in improved reading performance?" requires a certain type of research method. The question, "How do students like reading method X?" may require still another investigative method.
Second, the measurements themselves must be appropriate for the investigation. To use improved reading test scores as a measure of how much students like the reading instruction is just as inappropriate as using opinion information from teachers or students (e.g., testimonials) as an indication of improved student performance.
Scientific principle 4: Provide coherent, explicit chain of reasoning.
This principle attempts to explain away alternative explanations for the obtained results. Additionally, the assumptions made before, during, and after the research was conducted should be known. The methods of explaining away alternative explanations and the assumptions made by researchers will depend on the type of research methods they use to answer the research question. For example, if the research question is one of cause-and-effect, an experimental design should be used. Additionally, there will be several possible alternative explanations for the results (termed extraneous variables or threats to internal validity--discussed more in a later chapter). The assumptions made must be understood. For instance, researchers using experimental designs will assume that objective data can and will be collected. These data should also be collected via reliable and valid measures. Finally, the assumptions made based on the statistical analyses of the data should be taken into consideration. Principle 4 rests on the notion that the assumptions and reasoning made before, during, and after the investigation can be exposed to public scrutiny.
Scientific principle 5: Replicate and generalize across studies.
A single and isolated research investigation is limited in terms of the overall conclusions that can be made based on the results. Interestingly, public policy and research funding have changed based on isolated results. For example, years ago, two researchers from the University of Utah claimed that they had demonstrated cold fusion. In essence, the researchers found that they were able to generate energy from water. The findings were made public; we were all excited about the possibilities. Essentially, if the researchers were correct, we would have a renewable source of energy and could reduce or eliminate our dependence on fossil fuels.
Unfortunately, no other research lab was able to replicate these findings. Essentially, it was found that the researchers made some mistakes in the analysis and interpretation of their data. This is the point of science that sets it apart from other methods of "knowing." Science is self-testing. It is self-testing through the replication process. We will not know if we have an adequate theory unless that theory resists alternative explanations over many attempts to refute it. Thus, if we can get consistent results from one investigation to another, we will gain confidence in our theory. Replication research will be described in more detail later in the course.
Generalizability is also a critical aspect of scientific research. Generalizability of an investigation's findings is termed external validity. The issue with external validity is whether the results will hold true in other settings and with other individuals. The question is essentially this: "Will these results hold true in my school with my students and when I provide the same instruction as that used in the investigation?"
Scientific principle 6: Disclose research to encourage professional scrutiny and critique.
A critical distinction of science over other forms of "knowing" is that science requires, and even encourages, scrutiny and critique. Once an investigation is conducted, it should be submitted for peer review. The peer review process involves other experts in the field reviewing the investigation and determining its scientific merits. If the investigation is determined to have scientific merit, it will likely be accepted for publication in a specialized (for that particular research approach or topic area) scientific journal. If the investigation is not determined to have scientific merit, it will likely be rejected for publication in the journal.
The critical aspect of this process is that it requires a peer review. These peer-reviewed journals are said to be refereed. Other outlets for investigations that undergo peer review are academic books and conference presentations. These outlets provide us with our evidence base.
It is also critical to distinguish between the term "evidence base" and another term common in education--literature base. An evidence base, as described, is one that requires stringent peer review. A literature base does not require such a review. Just because an investigation or opinion is published, it does not mean the investigation or opinion has been subjected to peer review. For example, a claim could be made that teaching children to read before the age of 6 can damage the child cognitively. This claim could get published in a book or journal. Practitioners could pick up this book or journal and tell others about the problem with teaching children to read at too early an age. Thus, the opinion of one individual will be taken as fact by a number of others.
However, this opinion may be just that--an opinion without any scientific evidence. This is an example of a literature base. The opinion has not gone through a rigorous review of any kind. In fact, such a claim would not likely get into print in a scientific journal unless it had an adequate research design, adequate procedures for conducting the investigation, and adequate methods of analyzing and interpreting the results. Therefore, an evidence base involves actually producing verification that a theory or opinion is supported by data. A literature base involves providing a theory or opinion without any supporting data.