Question

Discuss the criteria of a good experimental study using at least one example of a good study and at least one example of a poor study (Examples can be anything - website article etc. about anything. Be sure to demonstrate how to design the procedure and avoid potential biases and pitfalls.

Answer 1

Criteria of Good Experimental Study

1. Provides unbiased estimates of the factor effects and associated uncertainties
2. Enables the experimenter to detect important differences
3. Includes the plan for analysis and reporting of the results
4. Gives results that are easy to interpret
5. Permits conclusions that have wide validity
6. Shows the direction of better results
7. Is as simple as possible

But this is a general note. On the note of discussion in Statistics and probability

The term experimental design refers to a plan for assigning subjects to treatment conditions. Good experimental design serves three purposes.

• Causation. It allows the experimenter to make causal inferences about the relationship between independent variables and a dependent variable.
  
• Control. It allows the experimenter to rule out alternative explanations due to the confounding effect of extraneous variables (i.e., variables other than the independent variables).
  
• Variability. It reduces variability within treatment conditions, which makes it easier to detect differences in treatment outcomes.

We can also consider this outlook:

In an experiment, a researcher manipulates a variable (e.g., imposes some treatment) to study its effect.

• randomization – to ensure groups being compared are similar apart from the treatment
• control and compare – to study the difference with and without treatment
• replication – to minimize the variation that results from taking samples
• (in some cases) a matched pair/blocked design – to control for likely sources of variation in the data

Example experiment: A chemist wants to obtain the calibration curve of the ability of an instrument to detect the concentration of a chemical in a sample. They test the response of the instrument for a series of samples with known concentrations of the chemical in a random order.

Example of a good experimental research:

Stanford Prison Experiment

This infamous Stanford Prison Experiment has etched its place in history, as a notorious example of the unexpected effects that can occur when psychological experiments into human nature are performed.

The study created more new questions than it answered, about the amorality and darkness that inhabits the human psyche.

As a purely scientific venture, the experiment was a failure, but it generated some results that give an insight into human psychology and social behavior. The ethical implications of this study are still discussed in college and undergraduate psychology classes all across the world.

In the days of the Abu Ghraib and Guantanamo abuses, the Stanford Prison Experiment is once again becoming relevant, showing that systematic abuse and denial of human rights is never far away in any prison facility.

This study is so well known that a Hollywood movie about the Stanford Prison Experiment is going to be released in 2009. The experiment has also been the basis of many similar studies, over the years, but these have had much stricter controls and monitoring in place.

Method

To conduct the Stanford Prison Experiment, Zimbardo constructed a mock correctional facility in the basement of Stanford University.

Adverts were placed in local newspapers offering $15 per day for participants in this program. Of the 75 responses, the 24 male subjects judged to be most mentally and emotionally stable were selected. Mainly middle class and white, they were divided into two groups randomly, of 12 prisoners and 12 guards.

The group selected to be the guards were outfitted in ‘military-style' intimidating uniforms. They were also equipped with wooden batons and mirrored shades, to prevent eye-contact and make the guards appear less human.

In an initiation meeting, Zimbardo, who acted as the warden for the duration of the experiment, informed the guards that the only rule was that no physical punishment was allowed. Other than that, the guards were to run the prison as they saw fit, and would be divided into regular working shifts and patterns.

Prisoners, by contrast, were dressed in cheap smocks and were allowed no underwear. They were to be addressed by, and answer to, identity numbers only. They also had a small chain around one ankle to remind them that they were inmates in a correctional facility. Conditions were tough, with only basic sleeping mattresses and plain food being supplied.

The prisoners were instructed to wait at home "to be called" for the start of the experiment; their homes were raided without any warning, arrested by the real local police department and charged with armed robbery.

The Palo Alto Police had agreed to help with the experiment. As if they were real-life suspects, the prisoners were read their rights and had their mug shots and fingerprints taken. After being stripped, searched and de-loused, they were taken into the cells that would be their homes for the next two weeks.

Zimbardo, acting as a prison warden, would be able to observe and make notes about what happened during the course of the study.

Results

The Stanford Prison Experiment degenerated very quickly and the dark and inhuman side of human nature became apparent very quickly.

The prisoners began to suffer a wide array of humiliations and punishments at the hands of the guards, and many began to show signs of mental and emotional distress.

On the second day of the experiment, the prisoners organized a mass revolt and riot, as a protest about the conditions. Guards worked extra hours and devised a strategy to break up and put down the riot, using fire-extinguishers.

No prompt for this action was given by Zimbardo; the guards used their own initiative to formulate the plan.

Standard prisoner counts and roll-call became a trial of ordeal and ritual humiliation for the prisoners, with forced exercise and physical punishments becoming more and more common. Mattresses were confiscated from the prisoners and they were forced to sleep on cold, hard floors.

Toilet facilities became a privilege, instead of a basic human right, with access to the bathroom being frequently denied; the inmates often had to clean the toilet facilities with their bare hands. Prisoners were often stripped and subjected to sexual humiliation, as a weapon of intimidation.

The experiment showed that one third of the guards began to show an extreme and imbedded streak of sadism, and Zimbardo himself started to become internalized in the experiment. Two of the prisoners had to be removed early because they were showing real signs of emotional distress.

Interestingly, none of the prisoners wanted to quit the experiment early, even when told that they would be denied their participation pay. The prisoners became institutionalized very quickly and adapted to their roles.

A replacement prisoner was introduced and was instructed to go on hunger strike as a protest about the treatment of his fellow inmates, and as an attempt to obtain early release. Surprisingly, his fellow inmates viewed him as a troublemaker rather than a fellow victim trying to help them.

When the inmates were informed that, if the rest of their prisoners gave up their blankets, he would be released from solitary confinement, all but one refused to give up their blanket.

The Stanford Prison Experiment carried on for six days until an outsider, Christina Maslach, a graduate student who would later become Zimbardo's wife, was brought in to interview guards and prisoners and was shocked by the scenes that she was witnessing.Zimbardo terminated the experiment early and noted that out of over 50 external visitors, this lady was the only one to raise concerns about what was happening.

Conclusions

Zimbardo believed that the experiment showed how the individual personalities of people could be swamped when they were given positions of authority.Social and ideological factors also determined how both groups behaved, with individuals acting in a way that they thought was required, rather than using their own judgment.The experiment appeared to show how subjects reacted to the specific needs of the situation rather than referring to their own internal morals or beliefs.The results of the experiment have been used in many high profile court cases over the years, to try and show that a prison must have clear instructions and guidelines from higher level authorities, or prisoner abuse may occur.

Poor Experimental Design: Poor experimental design essentially contains confounding.Gene expression study comparing 60 CEU and 82 ASN HapMap individuals. 26% of the genes were found to be significantly differentially expressed (78% with less restrictive multiple testing correction) . But: all CEU samples were processed (sometimes years) before all the ASN samples.