| Common Threads in Research
Across Disciplines: Experiment The Experiment component is what most folk think of when “research” is uttered. Indeed, it is often the most prominent. In this phase the test is performed, the measurement taken, the data collected, the musical theme tried out, the draft written, the pen put to paper or the chisel to wood. The task of data collection may be a nearly life-long occupation of observing chimpanzees in the wild, or it can be a few hours when a spacecraft zooms past a planet after years of preparation followed by months of waiting after the rocket’s ascent. The process is often taxing, but as Sophocles mused, whatever the case there are principles that apply to valid experimentation: “Knowledge must come through action; you can have no test which is not fanciful, save by trial.” (Sophocles, 430 B.C.E.) Intellectual integrity also is essential to meaningful data acquisition. One must devise strategies that divorce the outcome from personal interest and bias. The Placebo Effect is well known in which the patient’s (and the experimentalist’s) expectations significantly affect the outcome. A similar effect has been reported in educational research in which simply telling a class that they are part of an innovative educational study will improve their performance. In a more sinister form, bias has clouded judgment in numerous scandals such as the “Cold Fusion” fiasco of the 1980s in which the standard protocol of disinterested and critical science was subverted in an infamous career-ending debacle. This and other such “bad” science is recounted by the physicist Bob Park, in his book Voodoo Science: The Road from Foolishness to Fraud (2000). Valid inference requires that trustworthy data be available. The product of inference cannot rise much higher than the foundation upon which it is built, especially when that floor is dubious, compromised, cracked and crumbling. Measurements filtered by expectation or polluted by excessive noise can be insidious. I have observed the gradual drifting of the measurement of a universal physical parameter from an initial (grossly erroneous) value to another more accurate value, the change coming not all at once but slowly, as if there were a social pressure not to deviate too radically from the previous results. Over time, however, a new consensus emerged and the measurements of the value stabilized. Thus, I suspect that experimentalists in the “hard sciences” are not immune to the sickness of bias. Moreover, ill-conceived experiments may obscure the reality with noise. I have witnessed such error in the efforts of novice researchers (middle school Science Fair participants) trying to measure the speed of sound using a short string with a stopwatch, only to produce data that measured their reaction time and told absolutely nothing about the phenomenon they sought to investigate. How an experiment is done is important. With what insight and skill the activity is performed is crucial. Here lies the art of the experimentalist, the beauty of imagination made material. How one asks a question can make the difference in what one learns. This point is dramatically illustrated by a classic Hungarian joke that recounts how a traveler’s auto broke down near the village of Hatvan, 20 km from Budapest. The stranded tourist stops a farmer driving his horse-drawn cart: “May I have a ride with you?” he asks. “Egan! Climb up here and sit beside me.” “Is it far to Budapest?” the visitor continues. “Nem, it is not far to Budapest.” So the visitor, relieved, climbs aboard. They ride for an hour or more in silence. The rider then remarks, “I thought you said it was not far to Budapest,” to which the farmer replies, “Oh, now it is very far to Budapest.” (P. Revesz, personal communication, 1977). The questions one asks are very important in determining what one learns. The wise experimentalist is one who continually performs a meta-experiment refining and re-refining the instrument he uses to make better measurements or more skillful tests of a concept. The goal of this stage is the discrimination of information, the sifting of data to uncover fact. We seek differences and magnitudes of effect with controlled parameters.
|