Friday, September 13, 2013

Measuring Experimental Compliance

I came across this question on Cognitive  Sciences Stack Exchange and I thought it was pretty interesting.

How can experimental compliance (willingness of participants to participate) be measured?
I would like to have a measure for how willing and eager participants were during an experiment. I have the intuition that their eagerness to comply will profoundly alter results of psychological and even psychophysiological experiments, but I lack a good way of experimentally quantifying this. 

After giving this some thought, I felt that by applying a combination of techniques based on the learnings from obedience experiments as well as those on cognitive dissonance, it was possible to develop a technique that could reasonably measure experimental compliance.


Cognitive Dissonance at work.
I finally ended up with this:
A method that could help increase both experimental compliance as well as quantify it could be as follows: 
  • Take a quiz to measure subjects' self-perceived levels of obedience
  • Perform experiment normally
  • Take another test to measure current self-perceived levels of obedience
Although the initial test may significantly alter levels of compliance as questions on obedience will prime subjects' to be more compliant, the initial data, if looked at with the post experiment tests results, will give an accurate picture.

Why two tests?
The first would help in measuring their attitudes towards authority and general obedience that they believe, they hold. This may be highly inaccurate as has been observed in many experiments, most famously the Milgram Experiment, where less than 10% of the subjects were able to correctly predict how far they would go.
The second one would be a measure of actual behavior during the experiment. The results of both can be interpreted as follows:

1. If the attitudes and behavior did not match, that is, subjects first rated themselves as obedient but were less than compliant during the experiment, then they would be experiencing cognitive dissonance in the post-condition test. Hence, their second results would be higher and incorrect. In fact, their actual levels of compliance would probably have been even lower than initially predicted by them.

2. If attitudes and behavior match, then the post-experiment results could either be the same, or slightly lower than the pre-experiment results depending on the individual.
  • If they are the same, then the individual complied to the fullest extent possible with experimenter.
  • If lesser, then the subject complied with the experimenter to the maximum extent (s)he could justify to himself.
In either of these cases, the real measures of compliance would be the results of the second test.

What to test?
The tests should contain questions which are indirect and mixed with other questions. These should be asking about general situational obedience such as when being stopped by a cop, or being approached by a formally dressed executive in a company office.
The post-condition test could be more direct and ask specifically about the test.
Lastly, this would be most effective for experiments which involve an experimenter actively conducting it, thereby creating the maximum chances of dissonance in individuals who did not fully comply. This would allow for most accurate result collection.

Of course, hard to say how effective this would be without experimenting. I would love to hear back from anybody who gave this a shot!