Critically Reviewing Research (Literature Review)
•Why do this?
•Critically reviewing the literature to set the  context, explore theory, methods, experiences,
language, sampling and replicability/usefulness for your research
•
•


•1. Gain an overview – read introduction and summary (perhaps an abstract), plus conclusion – is it
clear and transparent what the research is about/theories/concepts?
•
2.- Identify and assess the methodological   approaches: quantitative? qualitative? Why was
particular methodological approach taken, plus strengths and weaknesses of it?
• - What is the focus and size of sample/or what is focus and sources of evidence?
• - Tools/methods?
• - Alternatives identified and examined?
• - Timescale clear?
• - Any ethical issues?
•

•3. Assessing findings (i)
•
• - Evidence
• - plausibility? (sources, use of evidence?)
• - reliability?
• - validity?
• - clarity?
•
•

Validity and Reliability
•Validity
• – covered/compensated for objectivity (or    lack of)?
• - credibility? (arguments, extrapolations, evidence)
•Reliability is a form of validity (quality control)
•How do we set up data collection so that it can be relied upon as having integrity?  Consider what
makes data collection unreliable?
•We can test to see if a questionnaire is reliable by using it on a pilot cohort/group (more than
once, with short gap in between): all conditions being equal, participant’s responses should remain
consistent

Threats to reliability
-participant error: fluctuation in responses due to external (changed) circumstances
-participant bias: social desirability effect (i.e. saying what participant thinks boss or
interviewer wants to hear)
-observer error: fluctuation in attention (to survey continuity) due to external circumstances
-observer bias: conscious or unconscious interpretation that has to do with researcher’s own
beliefs/experience (‘hearing/seeing’ what you want to ‘hear/see’ and conclude)

External validity (generalisability)
•The extent to which the findings are generally applicable.
•When generalisation from a properly constructed random sample to the research population from
which the sample drawn is a straightforward case of statistical inference
•Generalisations to other settings, to other (non-statistically controlled) groups not so
straightforward
-direct demonstration: achieving and identifying the same results with a different group, hence
that findings are transferable
-making a case: persuading that it is intuitively & logically reasonable to make generalisations
(based on qualitative evidence and factors)
•

Threats to external validity
•selection: findings specific to the group studied
•setting: findings dependant on context
•history: specific/unique historical experiences may determine or affect the findings
•construct effects; you may not be able to generalise because the effects studied are specific to
the group studied
• - BUT all can be accommodated if ‘threats’   are identified and ‘flagged up’/allowed for in   the
research and findings
•
•

Credibility
•Is there sufficient detail on the way the evidence was produced for the credibility of the
research to be assessed?
•The responsibility to give sufficient information and justification: transparency is especially
important with all research (and in particular in ethnographic and qualitative research) so that we
can weigh up the conclusions reached in terms of the contexts given

Establishing trustworthiness in qualitative data
•Dealing with the potential for bias as interpretative
• - Truth value: how do you establish confidence    in what is the ‘truth’ for the persons in the
  context and the truth of the context itself?    (allowing and accommodating for how they see
the situation, and how the situation is seen by    others?) Impacts upon the applicability of
 findings (accommodate in findings –       both/multiple interpretations)
•Consistency: repeatable with similar group within   similar context?
•Neutrality: can it be established that the findings are determined by the participants, responses,
contexts rather than perspectives, biases, motivations, interests of the enquirer?
•

Transferability
•Transferability is the qualitative version of external validity or generalisability
•Have to make a case for transferability of findings by establishing sufficient detail (about
similarities, contexts, factors/circumstances, etc. of cases?) so that the decision to transfer is
possible, and moreover reasonable, and academically acceptable and sustainable

•3. Assessing findings (ii)
•
• - Evaluation (strengths and weaknesses)
• - assessing conclusions - legitimate given findings in the research?
• - alternative interpretations?
• - relevance? (to your research and                       subject area overall) -
generalisability?
•