Read How We Know What Isn't So Online
Authors: Thomas Gilovich
Tags: #Psychology, #Developmental, #Child, #Social Psychology, #Personality, #Self-Help, #Personal Growth, #General
An additional problem with Rhine’s results was that the cards themselves were not manufactured with the strictest quality control, permitting an observant subject to identify the cards by certain irregularities like warped edges, spots on the backs, or design imperfections. The symbols on some cards reportedly could even be read through their backs under certain lighting conditions.
10
Questions about how well ordinary shuffling succeeds in truly randomizing a deck of cards also clouded Rhine’s findings. Without a randomly-ordered set of cards, the statistical analyses that are the centerpiece of Rhine’s research are meaningless: The beyond-chance results could stem from a pattern inherent in the cards being matched by a response bias on the part of the subjects. The seriousness of this problem is demonstrated by experiments in which results comparable to Rhine’s have been produced by simply shuffling two decks and comparing them to one another! Beyond-chance matching has been found in several such comparisons of “random” sequences, with the sequences produced either by card shuffling
11
or by comparing columns from published tables of random numbers.
12
These simulation experiments have also yielded various supplementary findings common to the field of parapsychology, such as
psi-missing
(significantly
fewer
guesses than chance expectation) and the
decline effect
(initial impressive success that trails off and eventually disappears over trials).
How much stock can be placed in Rhine’s data if strong results emerge when controls are lax but disappear when they are tightened? Can we infer the existence of psi from beyond-chance scoring when similarly impressive deviations from chance results can be obtained—without the intervention of a mind—by simply comparing seemingly random lists? The net result of these and other questions about Rhine’s research is that his work is rarely cited by parapsychologists these days as among the best evidence for the existence of psi.
The Soal-Goldney Experiments.
With the doubts surrounding Rhine’s findings, the burden of representing the definitive evidence for psi was passed to a set of similar experiments conducted by the British mathematician G. S. Soal and his assistant K. M. Goldney. Their procedure was as follows: An experimenter sat in one room with the “agent,” while the “percipient” was in an adjoining room. The experimenter, after consulting a randomized target list of the digits 1-5, would hold up the target number through a hole in a partition that separated the agent and experimenter. The agent would then pick up the one picture from a group of five that corresponded to the target digit. By concentrating on the selected picture, the agent would try to “send” the image to the percipient. The percipient would write down his or her guess, and the list of guesses was subsequently compared to the target sequence.
These procedures were conducted under much tighter experimental control than that which existed in the early days at Rhine’s laboratory. Independent witnesses were allowed into both rooms. Special precautions were taken to ensure that neither the agent nor the percipient knew the contents of the target sequence. Sophisticated tests of randomness were applied to check the target sequences, and copies of all records were sent after each session to a Cambridge University professor of philosophy, C. D. Broad.
Soal conducted these experiments during a four-year period beginning in 1935, during which time he examined the performance of 160 subjects on over 128,000 trials. The results of his efforts were tremendously disappointing: Absolutely no evidence for telepathy was obtained. Upon hearing these results, the parapsychological community was obviously disheartened.
Shortly afterwards, however, a colleague suggested that Soal check his results for “displacement” effects. Perhaps subjects’ guesses correlate significantly, not with the target cards, but with the cards that immediately preceded or followed the target cards. Beyond-chance matches to the subsequent cards might reflect precognition, for example. Soal initially considered this suggestion to represent somewhat questionable science and he was reluctant to try it. Eventually relenting, Soal discovered significant displacement effects for two of his subjects, Basil Shackleton and Gloria Stewart.
Recognizing the limitations of such a post-hoc analysis, Soal retested his two potentially-gifted percipients. Basil Shackleton was subsequently tested from 1941 to 1943 in over 400 sittings in which he made more than 11,000 guesses. As before, his responses were at chance level when compared to the target cards, but showed an excess of hits when compared to the card immediately following the target (2,890 hits compared to a chance expectation of 2,308; the odds of this happening by chance were estimated to be 1035 to 1). Gloria Stewart also provided additional tests. She made over 37,000 guesses in 130 separate sittings, and her responses—unlike Shackleton’s or her own earlier performance—were found to correlate significantly with the target cards, not those immediately following or preceding (9,410 hits compared to a chance expectation of 7,420; the odds of this happening by chance were estimated to be 1079 to 1).
Not surprisingly, these data were soon hailed as conclusive evidence for the existence of psi. Here were significant results produced under rigorous experimental control by an investigator whose integrity, by virtue of his earlier failures, was obviously beyond question. Parapsychologist R. A. McConnell stated that “… if scientists will read [Soal’s report] carefully, the ‘ESP controversy’ will be ended.”
13
Similarly, Whately Carington said that:
If I had to choose one single investigation on which to pin my whole faith in the reality of paranormal phenomena, or with which to convince a hardened skeptic (if this be not a contradiction in terms), I should unhesitatingly choose this series of experiments, which is the most cast-iron piece of work I know, as well as having yielded the most remarkable results.
14
The Soal-Goldney experiments stood as the cornerstone of the evidential foundation of ESP for the next twenty years. Skeptics were reduced to rather weak speculations about how Soal
might
have cheated.
15
Gradually, however, more substantial reservations about this work emerged, and today it seems clear that Soal faked his data.
Suspicions began to form when an agent in several of the Shackleton experiments, Gretl Albert, informed Mrs. Goldney that she had seen Soal altering the records during one of the sessions. Mrs. Goldney checked the score sheets soon afterwards, but she could find no evidence to support the accusations. Nor could a colleague that Mrs. Goldney had asked to do the same. When Mrs. Goldney brought the accusation to the attention of Soal, he apparently became indignant and insisted that Mrs. Albert not be allowed to participate in further experiments. Soal also persuaded Mrs. Goldney that they should not mention Mrs. Albert’s accusation in their reports of their work. In fact, Mrs. Albert’s charges were publicly acknowledged by Soal only when, more than fifteen years later, Christopher Scott pressured him to do so with the threat of publishing the accusations himself. Further suspicion was generated by Soal’s claim that he had lost the original score sheets in 1946, but that hand-made copies were available.
By themselves, these facts are mere curiosities. Scientists do occasionally misplace data, and the tampering that Mrs. Albert witnessed may have been a fastidious experimenter’s innocent efforts to tidy up the record. More damaging evidence eventually followed, however. To make his experiments as rigorous as possible, Soal had published detailed descriptions of how he had obtained the random numbers used to generate his target lists. However, computer analyses of the target lists provided by Soal indicate beyond any doubt that he either altered the lists or did not obtain his random numbers the way he said he did.
16
Thus, if fraud had been perpetrated, the most likely means would have been an alteration of the target list to match the percipient’s responses. This suspicion was bolstered when Mrs. Albert provided additional information about her initial suspicions of Soal. She claimed to have seen Soal changing 1’s into 4’s and 5’s on the target list. Guided by this claim, subsequent analyses of the full record confirmed that there were: a) an excess of hits when the target was a 4 or a 5, and b) a shortage of 1’s on those trials in which the guess was a 4 or 5. It thus seems that Soal began with target lists that contained an excess of 1’s, and that he subsequently changed some of the 1’s into 4’s and 5’s to create more hits.
17
The most conclusive evidence for this contention was obtained in a follow-up attempt to match Soal’s target lists with sequences contained in the tables from which he claimed to have selected them. Like the previous computer search, this attempt did not find any exact matches in the two sources. However, there were numerous close approximations—target sequences that matched those in the random number tables with only a few exceptions, “… as though digits had been inserted into one of the pairs of sequences (or omitted from the other).”
18
Predictably, these extra digits almost always corresponded to hits.
With this final bit of damaging evidence, what was once the strongest evidence for the existence of psi is now an embarassment to the field of parapsychology.
Remote Viewing.
Since Soal’s time there have been other examples of deliberate fraud in parapsychological research. Most notable in this regard were the actions of Walter Levy, J. B. Rhine’s top assistant and heir apparent, who was caught by staff members altering computer records of experimental results. Since Soal’s time as well, other research findings have been put forward as the definitive proof of the existence of psi. The “remote viewing” experiments conducted at SRI International were briefly thought of in such terms.
Remote viewing experiments were devised by two physicists, Harold Puthoff and Russell Targ. In these experiments, a percipient is seated in the laboratory while another subject proceeds to a nearby site that has been randomly selected from a small set of possible locations. While at the target site, the subject looks carefully at the surroundings and tries to send impressions of the locale back to the percipient. At a prearranged time, the percipient is asked to describe any impressions he or she has “received” by drawing a picture, writing a description, or both. This procedure is repeated over several days until all target locations have been visited.
Performance in these experiments is assessed by having a group of judges travel to each site, giving them the percipient’s pictures and descriptions (hereafter simply called “transcripts”), and having them rank the set of transcripts according to how well they describe each location. Are the rankings of the transcripts that correspond to each location higher than we can expect by chance?
The success rates claimed by Targ and Puthoff for their remote viewing experiments are among the most impressive ever reported in the field of parapsychology. For some of their more gifted subjects, the matching of transcripts to targets was almost without error. Moreover, Targ and Puthoff claimed that nearly everyone they tested was able to manifest this ability. Psi is not the exclusive property of the Uri Gellers of the world, but can be harnessed by anyone off the street. For some people, the publication of these results in the prestigious science journal
Nature
, rather than the usual parapsychology outlets, lent further weight to the evidence.
How well do these results stand up under close scrutiny? Careful examination of this work by New Zealand psychologists David Marks and Richard Kammann indicate that, like its predecessors as the bedrock support for psi, these experiments do not pass muster.
19
The problem in this case lies in the nature of the transcripts provided to the judges. Most of the material in the transcripts consists of the honest attempts by the percipients to describe their impressions. However, the transcripts also contained considerable extraneous material that could aid a judge in matching them to the correct targets. In particular, there were numerous references to dates, times, and sites previously visited that would enable the judge to place the transcripts in proper sequence. For instance, a comment such as “I’ve been trying to picture where you went yesterday on the nature walk” informs the judge that: 1) the correct target must not be the nature preserve; 2) this target, whatever it was, was not the first in the series; and 3) this target, whatever it was, came after the nature preserve. Knowing the target is not the nature walk obviously simplifies the judge’s task regardless of what additional information he or she has. However, information about where the transcript fits in the sequence of target sites is only helpful if the judge is aware of the target order. Astonishingly, the judges in the Targ-Puthoff experiments were given a list of target sites in the exact order in which they were used in the tests!
To determine whether it was these extraneous cues rather than remote viewing that produced the remarkable accuracy of the judges’ rankings, Marks and Kammann conducted a control assessment they call “remote judging.” In remote judging, the judges are given a set of transcripts containing
only
the extraneous cues and none of the legitimate descriptive material. Then,
without visiting a single location
, they are given a list of the target sites in the proper order and asked to match transcripts to targets. They did so as accurately as the judges in the Targ-Puthoff experiments! As an additional control procedure, Marks and Kammann edited the transcripts from the Targ-Puthoff experiments so that they no longer contained these extraneous cues, and then asked a group of judges to match them to the targets. Without the help of the extraneous information, the judges were not able to do so with any ciccuracy. So much for remote viewing.