Clinical Trials Smackdown

 

This activity teaches skills in critical assessment of the peer-reviewed published literature.  It focuses on analysis of clinical trials in mental health, but the principles and methods are readily generalizable to other scientific literature.  The “Smackdown” approach represents an augmentation of the traditional “journal club” mode of teaching critical scientific reading skills.

Goals: To develop skills in critical analysis of peer-reviewed scientific literature, including scientific justification, methodology, interpretation, and ethical and conflict of interest aspects.

Class: Research Methods, a seminar for post-graduate year II residents in psychiatry and nurse practitioner residents as part of the curriculum of the Harvard South Shore Psychiatry Residency Training Program at the VA Boston Healthcare System. Class sessions last 60 minutes and occur weekly. 

Background: This activity consisted of a five-session module in the Research Methods class designed for second year psychiatry residents and audited by mental health nurse practitioner residents. Participants come in with advanced clinical degrees (MD, DO, APRN), but with variable and often little to no familiarity with research methods. 

The seminar holds as an orienting premise that although a minority will “specialize” in research careers, all residents must be adept at reading and interpreting the scientific literature to prepare for a career of lifelong learning.  Moreover, we adopt adult learning principles which focus on active rather than passive learning, and targeting subject matter that is relevant to day-to-day duties, i.e., decision-making in the clinic.  

For several years the faculty member modeled the critique of an apparently high-quality study published in a premier peer-reviewed journal using a standardized critique template.  For the next four sessions, the faculty member assigned a pair of residents to read and critique a clinical trials article, with the other seminar participants discussing their views.  This is the time-honored “journal club” format that characterizes most similar faculty activities.  However, it became clear that while the assigned residents did their homework, there was little participation from the remaining participants, who played an essentially passive role (and likely didn’t read the paper beforehand, unless they were the assigned presenter).

We thus transitioned to the following “Clinical Trials Smackdown” format, which began with a modeling session by the faculty member as before, followed by four resident-led sessions.

Before Class:  The faculty member prepared an assignment grid that assigned a pair of residents and one nurse practitioner to each of the four major sections of the scientific article:  

  • Introduction 
  • Methods
  • Results
  • Discussion

One resident was assigned to take the “Pro” position for that specific section, emphasizing the positive aspects of that section of the article, while the other resident was assigned to take the “Con” position, emphasizing its shortcomings.  The nurse practitioner resident took a “Wild Card” position and could take either side.  The grid was developed so that across the four weeks of the module:

  • Each resident had at least one “Pro” week and one “Con” week.
  • Each resident had a week in which they were responsible for one of the four major sections.

This ensured that each participant took an active learning role in each session (not just the one in which they were the primary discussant), and that each had to be ready to joust in public with someone who would debate them during the session.

During Class: The faculty member gives a brief (5-minute) overview of the study.  The participants then break up into triads for 20 minutes of debate among themselves.  The groups then reconvene and present their sides of the “debate” for the Introduction, Methods, Results, and Discussion in order, guided by the critique template as modeled by the faculty member in the first session of the module.  These presentations and discussion last approximately 30 minutes.  At the very least, the three assigned to a section present—but typically other residents (who have typically read the entirety of the article each week) also contribute. The class closes with a 5-minute discussion led by the faculty member closing the class, addressing the Key Question: “Will this change your practice?” 

After Class: Participants are assigned the next article & their role in the critique. 

Materials: An electronic version of the critique template, assignment grid and the articles are distributed at the beginning of the module.

Comments from the Instructor: This format clearly enhanced breadth and depth of commentary, with all participants actively contributing to each class.  End-of-year anonymous evaluations revealed high marks for this module (4-5 on a 0-5 satisfaction/quality scale administered by the residency program).

Several specific comments for potential adopters of the Smackdown format:

  • The method is scalable, initially developed for 8 participants, but easily accommodating 12.  It is likely that 16 (4 groups of 4) would work as well.

  • The method can be used for 45-minute to 90-minute sessions.

  • The method is flexible as to study content.  Articles from any discipline (including undergraduate arts and sciences) could be utilized, with appropriate adaptation to discipline and level of learner.

  • It is important to pick an article from a highly respected and peer-reviewed journal, or equivalent source.  All studies have strengths and weaknesses, and it is important for residents to understand that this is true even of articles that are published in blue-ribbon journals. 

  • The hardest work for the faculty member is constructing the assignment grid, which must ensure that each participant has at least one assignment each to the Introduction, Methods, Results, and Discussion sections and at least one “Pro” and one “Con” assignment.

  • Preparing for and leading the seminar is actually easier than in more passive modalities, when the role of the faculty member can feel like “pulling teeth” or repeatedly “priming the pump.”

  • Because of the active role required of each resident, the faculty member is able to provide more accurate evaluation of resident competencies to the residency program than would be possible in a more passive activity.

Submitted by Mark Bauer, M.D., Professor of Psychiatry, Harvard Medical School

blank article.doc50 KB
dissection article.doc79 KB
clinical trials assignments.doc45 KB