A Method for Characterizing Semantic and Lexical Properties of Sentence Completions in Traumatic Brain Injury

Academic
Paper Review
A (hopefully) accessible summary of my first first-authored paper.
Author

Matthew J. Kmiecik

Published

July 22, 2018


Short and Sweet

Last summer we published a paper in Psychological Assessment that presented methods to better characterize verbal responses on neuropsychological assessments. These methods were applied to a specific cognitive test called the Hayling Sentence Completion Test. We demonstrated that these new methods provide additional insights into the cognitive performance of individuals with mild to moderate traumatic brain injury (TBI). Furthermore, these new measures were related to other cognitive abilities, such as verbal knowledge, processing speed, inhibitory control, working memory, and task switching. These methods can be extended to other assessments that involve word generations and clinical populations that exhibit cognitive impariments.

To read our paper, see:

Kmiecik, M. J., Rodgers, B. N., Martinez, D. M., Chapman, S. B., & Krawczyk, D. C. (2018). A method for characterizing semantic and lexical properties of sentence completions in traumatic brain injury. Psychological Assessment, 30(5), 645-655. http://dx.doi.org/10.1037/pas0000510

You can also read it on ResearchGate.


If you are interested in learning more…

Long and Drawn (Out)

Traumatic brain injuries (TBIs) occur when the head receives a harmful blow that results in disruptions to normal brain functioning. Individuals can recover following these injuries; however, some individuals’ symptoms persist long after the injury (> 3 months post-injury) and become chronic TBI.

Individuals with chronic TBI often report difficulties in completing everyday activities, such as going to the grocery store or holding a job. In terms of what is required cognitively, these tasks are quite complex and require planning, blocking distractions, knowledge, and sustained attention. Neuropscyhologists call these complex cognitive abilities executive functions and a large focus of TBI research is understanding how these cognitive functions are impacted and how these deficits affect behavior, such as difficulties in holding a job.

Despite these self-reported impairments, it is difficult to capture these symptoms in a lab setting. In other words, an individual with TBI may report having difficulties in everyday life, but he or she may perform well on standardized neuropsychological assessments. Are the tests not sensitive enough? Are we measuring the right thing? Is TBI more difficult to assess?

The difficulties in understanding the effects of TBI is likely a mixture of the above concerns. The paper that this blog post covers addresses a specific concern: the sensitivity of existing tests. We took an existing neuropsychological assessment of executive functioning, called the Hayling Sentence Completion Test (Burgess & Shallice, 1996, 1997), and applied methods from the fields of cognitive science and linguistics to gain further insights about how TBI impacts performance.

When taking the Hayling Test, individuals are read sentences with the last word missing. In the first section, participants are instructed to complete the sentence with a logical word. In the second section, participants are instructed to complete the sentence with an illogical word. For example:

Section Completion Sentence Stem Correct Completion
1 Sensible He mailed a letter without a stamp.
2 Unconnected The captain wanted to stay with the sinking banana.

Performance on this test is guaged by the time to complete each section (faster is better) and the number of errors committed on section two (fewer errors is better). The frontal lobe is responsible for inhibiting distractions and following task instructions. The thinking behind this test is that those with damage to the frontal lobe will commit more errors on section two.

One potential source of improvement we saw was to introduce more objective methods in scoring errors, as unclear responses are often given.

We measured 2 different properties of the words:

  1. How similar the sentence stem (He mailed a letter without a) and the generated word (stamp) were to each other using a technique called Latent Semantic Analysis (LSA; Landauer, Foltz, & Laham, 1998)

  2. How frequently the generated words are used in everyday speech (word frequency) using the SUBTLEXus database (Brysbaert & New, 2009)

What we discovered was that these measurements provide additional metrics that can be used to guage performance on the Hayling test. More specifically, other measures of cognitive performance were related to performance on these additional measurements. These additional measures were verbal ability, processing speed, inhibitory control, working memory, and task switching. Taken together, these results suggest that these additional measurements are closely related to high-level cognitive functions and can be used to better understand solving strategies and response patterns in clinical populations.

Next steps in this line of research include further validating these measurements. Using our new measurements, we are currently conducting a study to compare those with chronic mild/moderate TBI to those without a history of TBI to further understand the effects of TBI on cognitive functioning. Our hope is that these methods can be extended to other linguistically based neuropsychological assessments and other clinical populations that experience cognitive impairments to better understand, diagnose, and formulate treatment plans.

References

  • Brysbaert, M., & New, B. (2009). Moving beyond Kucˇera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behavior Research Methods, Instruments and Computer, 41, 977–990. http://dx.doi.org/10.3758/BRM.41.4.977

  • Burgess, P. W., & Shallice, T. (1996). Response suppression, initiation and strategy use following frontal lobe lesions. Neuropsychologia, 34(4), 263-272. 10.1016/0028-3932(95)00104-2

  • Burgess, P. W., & Shallice, T. (1997). The Hayling and Brixton tests. London, England: Pearson.

  • Kmiecik, M. J., Rodgers, B. N., Martinez, D. M., Chapman, S. B., & Krawczyk, D. C. (2018). A method for characterizing semantic and lexical properties of sentence completions in traumatic brain injury. Psychological Assessment, 30(5), 645-655. http://dx.doi.org/10.1037/pas0000510

  • Landauer, T. K., Foltz, P. W., & Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 25, 259–284. http://dx.doi.org/10.1080/01638539809545028