Critique Main

Critiquing Tools

• Enable nurses to systematically evaluate evidence for practice by using criteria of scientific merit as well as issues of practicality and feasibility.
• Criteria of scientific merit or guidelines for critiquing research articles are covered in most basic research texts, examples: Burns and Grove, 1997; Polit & Hungler, 1999.
• Tools addressing practice issues are more difficult to find and most shown here were developed in research utilization efforts.

The basic premise is that when critiquing research for application to practice a two-step approach is needed: (Titler, Mentes et al. 1999).

• This review of the scientific merit may follow general guidelines for critiquing research studies and should include the key aspects of:
  
  • Conceptualization and logical links between the sections of the study
  • Strength of the study design, sample, measurement instruments, data collection and analysis procedure
  • Representativeness of patients/clients studies along with setting
  • Number of replications of study (Burns and Grove 1997)


• Another method of determining the characteristics of studies most likely to provide valid results (randomized controlled trials) (Oxman, Sackett et al. 1993) that stresses design issues is:
  
  • Patients have been randomized to treatment.
  • All patients who entered the study are properly accounted for.
  • The control or comparison treatment is clearly described.
  • Patients included in the study include the types of patients you are considering changing practice for.
  • Follow-up of patients was long and complete enough to determine whether the treatment made a difference.

• One method that may be used follows from Appendix E, "Guide to Reading Research Critically" (Horsley, Crane et al. 1983).
• Problem studied. Does study address the current clinical problem/concern for this project?
• Study setting. Similar or different from current clinical setting?
• Number & type of patients/clients studied. Unique or typical? Similar or different from current clinical setting?
• Independent variable (variable assumed to cause the effect being studied; often a nursing intervention, patient characteristic, patient behavior).
• Dependent or outcome variables under study. How measured?
• Effects of independent variable on patients or clients (outcomes)?
• Any doubt that intervention (independent variable) led to outcomes?
• Issues of measurement reliability addressed sufficiently (reliability - if measurement was repeated on same patients, in same conditions, same results would occur)?
• Issues of measurement validity addressed sufficiently (validity - degree to which tools selected actually measure what they are intended to measure)?
• Conclusions of study are based upon statistically significant findings?
• Results of similar studies similar? Conflicting?
• Your interest in using the intervention in clinical practice? Why?
• Appraisal of findings requires asking different questions of different types of studies: (Brown, 1999) has appraisal forms for non-experimental (descriptive survey design, correlational designs); qualitative methods; experimental; and quasi-experimental designs: Each form is divided into the following categories of questions:
• Synopsis (leads to description of study)
• Credibility profile (leads to justification for validity and generalizability of findings; also compares individual study findings with other findings)
• Clinical significance profile (leads to statement of clinical utility of findings)
• Applicability profile (leads to statement of who/where findings might be useful, judgment of risk/benefits to clients/patients if change based upon findings were to be made, analysis of barriers to practice change)

Tables of evidence are helpful tools developed to visually display the findings of multiple studies. A table of evidence clearly tabulates features of individual research studies including those that affect utilization of findings. Columns in the table may be different according to topic of interest and purpose of the table (e.g., metholodologic factors or utilization factors) but can include citation information, study purpose, measurement tools, sample, design, procedures, outcome variables, results/findings, and implications for the project.

Example. A good example of a table of evidence is found in Table 1 Wakefield et al, 1998 (pp. 1507-1508). This table includes columns for study (citation information), purpose, sample, outcome variables (tools indicated), results. Additional columns may have been deleted for publication.

Warning. There is a tendency to put too much into each cell. This tendency can be diminished with practice.

Questions to Ask When Critiquing Findings

Once a table of evidence has been compiled, questions to ask include the following (Brown, 1999):

• Which findings describe the experience at one point in time, which describe it over time?
• Are there findings that describe the subtleties or fine points of an experience to allow you to get a better understanding of the experience?
• Which findings are supported by more than one study?
• If one study has a unique finding that is compelling, analyze why.
• Are there findings that are contradicted across studies? How do you account for this?
• If results were not consistent across studies, what could explain the differences in results?
• Which study findings come from studies using patients/setting similar to yours?
• Which findings do you feel confident are supported by the research evidence?

• What would I change in practice if I used the above findings?
• What type of change (degree - small, medium, large) would be involved?
• Is there a risk or burden involved in making this change?
• Is the change feasible in terms of resources and economics? If not, why?
• If I make the change, what will happen to patients? (what will be an expected patient outcome?)