McNulty Summer Scholar Studies Reliability of Concussion Test
Philanthropy supports innovation in faculty-mentored research and creative projects
Concussions are big news. Most every state has passed some sort of concussion legislation as the volume of research on traumatic brain injuries (TBI) escalates.
Junior biology major Kathleen Logan ’16 had a personal connection to the concussion concern which she was able to investigate as a Summer Scholar. Her sister Colleen Logan, now a Saint Joseph’s freshman, suffered a concussion a year ago after colliding with another player during a high school soccer game. “She played the rest of the game but by the next day was in a complete fog,” Logan said. “She missed two weeks of school and was on the sideline for the rest of the soccer season.”
As a McNulty Scholar, Logan had an opportunity to participate in the Saint Joseph’s University Summer Scholars Program through funding from the John P. McNulty Scholars Program for Excellence in Science and Math.
“Providing real-world research and leadership opportunities to young women with ability, work ethic and ambitiousness is the core of the McNulty Scholars Program,” said program benefactor Anne Welsh McNulty. “By the time they graduate, they’re ready to forge bold new paths in their fields. Kathleen has experienced great success in this program already, and I look forward to her development as a scholar and leader her field.”
Logan’s interest in concussion research took her across scientific disciplines to the office of Professor of Psychology and Director of the Behavioral Neuroscience Minor Philip Schatz, Ph.D. Schatz is a neuropsychologist with research interests assessing and monitoring the effects of concussions in youth and collegiate athletes and computer-based assessment. His research has been widely cited.
When Schatz agreed to mentor Logan, she was delighted for the opportunity to work with someone with his credentials and experience. “This was truly outside-the-classroom learning that exposed me to new ideas and scientific approaches,” she said. “I appreciated Dr. Schatz’s big picture thinking and the real-world applications of this work.”
Their research focused on Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT), the widely used and validated method of concussion assessment. ImPACT allows for an objective comparison of the individual before and after the injury. Current practice involves athletes undergoing a baseline examination before each season. After suffering a concussion, the injured athlete completes the ImPACT test again and the comparison to baseline performance assists in the diagnosis and management of a concussion.
The project investigated the reliability of the ImPACT test across three administrations (the novel aspect of this study along with the large volume of data available) as well as the utility of updated baselines when compared to post concussion test data. Logan prepared and organized the data set – on more than 700 athletes from a collaborative relationship with colleagues working with Louisiana high schools – for statistical analyses. Under Schatz’s mentorship, she also analyzed the data and interpreted results.
Preliminary results show moderate test/retest reliability in all five neurocognitive composite scores for athletes taking the ImPACT test three times across three administrations. “The results suggest that an annual update is an appropriate schedule,” Schatz said, “but my longer term interest is in documenting the utility or need for updated test scores.”
Of the more than 700 athletes, 71 completed two annual baselines, which were then followed by a concussion. Of the 71, only 4 percent (3/71) would have been diagnosed as “concussed” based on comparison to the second baseline instead of the first. “In further exploring the utility of these scores, we need to ask, for example: ‘If after sustaining a concussion, an athlete’s test scores drop considerably from baseline levels, is the small variation between annual baseline evaluations relevant in the context of such a large decrease in performance?’”
This research was phase one of a larger analysis that will ultimately be submitted for publication. It also triggered an offshoot project examining the correlation between high school data (state test proficiency scores and mean ACT scores) with ImPACT composite scores for verbal and visual memory. Preliminary results indicate a low correlation. However, results show a significant difference between public and private high school athletes’ composite scores on both verbal memory and visual motor speed.
According to Schatz, this is basic psychometric research that is the focus of only a small group of academic researchers. For Logan, undertaking this project required moving from her comfort zone in laboratory-based science to social science involving complex multivariate statistics. Soon Excel formulas, inclusion criteria and metrics such as “reliable change” became daily terms.
Logan rose to the challenge. “She brought intellectual curiosity, excellent academic preparation, a tremendous work ethic and strong communication skills to the task,” Schatz said. “Kathleen will certainly be appropriately listed as a co-author in the final paper.”