Demand Characteristics and Response Bias [Integral Eye Movement Therapy (IEMT) Wiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Demand Characteristics and Response Bias ======

Demand characteristics are cues that help people infer what is expected of them in a study or other evaluative situation. Response bias is a broader term for systematic influences on the answers people give or the observations that are recorded. Both matter when interpreting therapeutic change, especially when the person delivering treatment also asks whether it has worked.

These concepts concern the conditions under which evidence is produced. They are not diagnoses of dishonesty. A client can answer sincerely while interpreting a question differently from the practitioner, recalling only part of an experience or feeling pressure to provide a reassuring answer.

===== The original idea =====

Martin Orne's influential discussion of psychological experiments emphasised that participants are active interpreters of a situation. They notice instructions, behaviour, setting and apparent purpose. As he observed:

> response to the demand characteristics is not merely conscious compliance.

— Orne (1962). His argument concerns the participant's understanding of the situation, including influences that need not involve deliberate cooperation. It does not justify assuming that every reported therapeutic benefit is a response to suggestion.((Orne, M. T. (1962). On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. American Psychologist, 17, 776–783. [[https://www.psych.upenn.edu/history/orne/orne1962amerpsychol776783.html|Read source]].))

Applying the concept beyond the laboratory requires care. McCambridge and colleagues' systematic review identified only seven eligible studies of demand characteristics in non-laboratory settings, with heterogeneous methods and limited evidence. The review supports investigating possible influences rather than treating “demand characteristics” as an all-purpose explanation for an inconvenient result.((McCambridge, J., de Bruin, M., & Witton, J. (2012). The effects of demand characteristics on research participant behaviours in non-laboratory settings: A systematic review. PLoS ONE, 7(6), e39116. doi:10.1371/journal.pone.0039116. [[https://pmc.ncbi.nlm.nih.gov/articles/PMC3378517/|Read source]].))

===== Distinguishing related influences =====

^ Term ^ Main issue ^ Fictional example ^
| Demand characteristics | The person infers the answer or behaviour the situation seems to require. | A client thinks a lower score is expected after a procedure. |
| Social desirability | Answers favour an acceptable or favourable impression. | A person underreports difficulties they consider embarrassing. |
| Recall bias or recall error | Retrospective accounts are affected by how events are remembered. | A difficult weekend dominates a rating intended to cover a month. |
| Acquiescence | A tendency to agree, regardless of the proposition's content. | A respondent endorses differently worded claims that conflict. |
| Observer bias | The assessor's knowledge or expectations influence judgements. | An ambiguous response is rated more favourably after a preferred treatment. |

These categories can overlap. Althubaiti reviews several forms of information bias, including problems in self-report and measurement. A useful distinction is between random imprecision and a systematic tendency that moves observations in a particular direction. Neither can be diagnosed from a single unexpected answer.((Althubaiti, A. (2016). Information bias in health research: Definition, pitfalls, and adjustment methods. Journal of Multidisciplinary Healthcare, 9, 211–217. doi:10.2147/JMDH.S104807. [[https://pmc.ncbi.nlm.nih.gov/articles/PMC4862344/|Read source]].))

===== Questions are part of the measurement =====

An answer depends partly on what the respondent understands the question to mean. Schwarz's work on self-report examines how wording, response options and the surrounding conversational context shape answers. This makes the design of outcome questions part of the assessment, rather than a neutral container for information.((Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54(2), 93–105. doi:10.1037/0003-066X.54.2.93. [[https://dornsife.usc.edu/norbert-schwarz/self-report/|Read source]].))

Consider three invented questions asked after a session: “How much better is it now?”, “Is the problem gone?” and “What, if anything, has changed since we began?” The first presupposes improvement; the second offers a narrow yes-or-no judgement; the third allows improvement, no change, deterioration or mixed effects. None is a substitute for an appropriate validated measure, but the differences show why wording should be considered when comparing responses.

Time frames matter as well. “How anxious are you?” might refer to the present moment, the previous week or the person's general identity. A rating of current distress while sitting in a quiet room cannot simply be relabelled as a measure of anxiety across daily life.

===== Measurement consistency and neutrality =====

When a questionnaire has validated instructions, changing them casually can change what is being measured. The same response scale, time frame and administration conditions should be used where comparison depends on their consistency. Neutral administration also means allowing the client to provide an answer without the practitioner visibly rewarding a preferred result.

The aim is not to create an emotionally cold encounter. A practitioner can respond respectfully to distress while avoiding evaluative signals about which score is desirable. An original example is: “Please use the same time period as before. Higher, lower and unchanged scores are all useful information.” The instruction explains the task without directing the outcome.

Repeatedly asking for a rating until it falls creates a different problem. The eventual score may reflect a real change, but it may also reflect changed interpretation, fatigue or a wish to end the exchange. Reporting only the lowest score conceals the sequence needed to evaluate these possibilities.

===== Assessor knowledge and allegiance =====

Knowledge of treatment allocation can affect judgements, particularly when an outcome requires interpretation. Hróbjartsson and colleagues reviewed trials that included both blinded and nonblinded assessors of measurement-scale outcomes. Their design addresses whether assessor knowledge can influence estimated effects; it does not imply that every unblinded assessment is wrong.((Hróbjartsson, A., et al. (2013). Observer bias in randomized clinical trials with measurement scale outcomes: A systematic review of trials with both blinded and nonblinded assessors. CMAJ, 185(4), E201–E211. doi:10.1503/cmaj.120744. [[https://pmc.ncbi.nlm.nih.gov/articles/PMC3589328/|Read source]].))

In psychotherapy, blinding the practitioner to the method they are delivering is often impossible. Other protections may still be available: an independent assessor, standard scoring rules, separation of treatment and data collection, or an outcome that can be checked against a defined behavioural criterion. Reports should state which protections were used and which were not.

An assessor who does not know allocation can nevertheless learn it accidentally. If a participant describes a recognisable procedure, the report should not continue to describe masking as perfect. It is more informative to explain how masking was arranged and any known limitations.

===== Expectancy is not the same as response bias =====

Expecting improvement could influence engagement or experience, influence what is reported, or do both. These are different pathways. A study measuring only self-reported improvement may not be able to distinguish them.

Boot and colleagues show why an active comparison does not automatically equalise expectations. This matters here because participants' beliefs about the comparison can affect how they interpret the study. Measuring expectations and explaining both conditions credibly can make interpretation more informative, but cannot guarantee that all reporting influences have disappeared.((Boot, W. R., Simons, D. J., Stothart, C., & Stutts, C. (2013). The pervasive problem with placebos in psychology: Why active control groups are not sufficient to rule out placebo effects. Perspectives on Psychological Science. doi:10.1177/1745691613491271. [[https://journals.sagepub.com/doi/10.1177/1745691613491271|Read source]].))

Equally, a physiological or behavioural measure is not automatically superior to a self-report. Measures answer different questions. A person may experience less intrusive distress while a laboratory response changes little, or behave differently while still reporting distress. The discrepancy should be investigated rather than resolved by declaring one type of evidence universally authoritative.

===== A worked example of discrepant outcomes =====

The following scenario is fictional. After an eye movement session, a client gives a current distress rating of 2 rather than 8. When asked whether they feel ready for an avoided journey, they say yes. At a later appointment, they report that they still have not made the journey.

There are several possible interpretations. The session may have changed distress during recall without changing behaviour. The person may not have had an opportunity to travel. A new practical obstacle may have emerged. The earlier answer may have expressed hope rather than a prediction. They may also have felt pressure to agree.

The discrepancy alone does not identify response bias. A useful review asks what the original rating referred to, whether the journey became possible, what the client understood the question to mean and what happened afterwards. Recording these answers preserves the evidence instead of forcing it into a success-or-failure story.

If the client says they wanted to please the practitioner, that statement is relevant, but it need not invalidate every earlier observation. The report can distinguish a lower in-session distress rating from an overoptimistic prediction about behaviour.

===== Strengthening routine outcome review =====

Practical safeguards include agreeing the outcome before treatment, retaining the same assessment time frame, recording the sequence of measurements and making space for unwanted effects. Where feasible, clients can complete forms privately before discussion. Independent follow-up can reduce some interpersonal pressures, although it introduces its own practical and interpretive limitations.

Using several relevant sources can help: the person's account, an appropriate scale and an agreed behavioural observation. Agreement across them can strengthen confidence; disagreement provides information about the scope of change. Collecting many measures and selecting only the most favourable afterwards has the opposite effect.

For research, specifying primary outcomes and analysis plans in advance makes selective interpretation easier to detect. For a case report, clearly separating contemporaneous records from retrospective recollection serves a similar purpose, though it does not turn an uncontrolled report into an experiment.

===== Relevance to eye movement therapy reports =====

Reports of IEMT, EMI or EMDR should describe who asked the outcome questions, when they were asked, what the ratings meant and whether the assessor knew the intervention. Statements about improvement become more useful when their measurement conditions are visible.

Demand characteristics should remain a testable methodological concern, not a label used to discredit a client or defend a favoured theory. The professional task is to improve the conditions for accurate reporting and preserve uncertainty where the evidence cannot distinguish among explanations.

===== See also =====

  * [[expectancy_contextual_effects|Expectancy and Contextual Effects in Therapy]]
  * [[common_factors_psychotherapy|Common Factors in Psychotherapy]]
  * [[statistical_clinical_significance|Statistical Significance and Clinical Significance]]
  * [[writing_interpreting_case_reports|Writing and Interpreting Case Reports]]
  * [[measuring_therapeutic_change|Measuring Therapeutic Change]]