News of Trump passing cognitive test may make it harder to detect dementia

Attachment 12269

News reports in January that President Donald Trump passed a widely used test that screens for mild cognitive impairment flung the little-known clinical tool into public focus. Google searches for the test—the Montreal Cognitive Assessment (MoCA)—spiked as dozens of media reports shared parts or all of the test and political commentators batted it around.

The president’s supporters proudly played up the test, boasting of Trump’s perfect 30-out-of-30 score and using it to laugh down those who questioned Trump’s mental state. Others snickered over the test’s seemingly straightforward components, such as asking test takers to correctly draw times on a clock and identify animals.

But the laugh may be on all of us, according to a research letter published Monday, July 16 in JAMA Neurology.

In the short communication, doctors led by Hourmazd Haghbayan of the University of Toronto assembled data on the extent of exposure to the public that the validated clinical tool received. And most importantly, they pointed out that several studies have suggested that such publicity may make the MoCA less reliable for detecting early stages of cognitive decline or dementia in test takers. Essentially, seeing or taking the test allows takers to learn the questions—and the answers—resulting in artificially improved scores. This could potentially obscure signs of early cognitive decline in common clinical assessments that rely on the tool, they warn.

“The presence of a learning effect could result in decreased sensitivity, eroding the MoCA’s chief usefulness in identifying patients with mild cognitive impairment,” they write. “Future research is warranted to validate test performance in populations with direct exposure to the questionnaire’s content or grading scheme.”

In the meantime, Dr. Haghbayan and colleagues recommend that clinicians ask patients if they’ve been exposed to the MoCA before giving it to them. They also recommend that clinicians use alternative versions than the one most commonly highlighted in news articles—i.e., version one as opposed to versions two and three.

Trumping the test

Before their warning and recommendations, the doctors tried to assess the scope of the test’s exposure. Simple Google News and Google Trends data suggested wide coverage. The doctors found 190 unique news articles that discussed MoCA in the context of Trump’s testing. Of those, 53.7 percent (102 articles) included all or some of the test’s questions, with 44 percent (84 articles) including the test in its entirety. Seventeen percent invited readers to take it, and nearly nine percent published the answer key. Outside of news articles, Internet searches for the MoCA skyrocketed relative to searches for other cognitive tests, the doctors found.

This is a problem, as previous research suggested that people can easily learn the test and score better after exposure to it. For instance, a 2015 study involving 468 participants found that those who repeated the test within a year of first taking it were able to improve their score. Another study from 2015 found similar evidence of a learning effect. Most importantly, the study indicated that those who initially scored low on the MoCA earned a significantly higher score when they took it again 12 months later.

The authors concluded:
News reports in January that President Donald Trump passed a widely used test that screens for mild cognitive impairment flung the little-known clinical tool into public focus. Google searches for the test—the Montreal Cognitive Assessment (MoCA)—spiked as dozens of media reports shared parts or all of the test and political commentators batted it around.

The president’s supporters proudly played up the test, boasting of Trump’s perfect 30-out-of-30 score and using it to laugh down those who questioned Trump’s mental state. Others snickered over the test’s seemingly straightforward components, such as asking test takers to correctly draw times on a clock and identify animals.

But the laugh may be on all of us, according to a research letter published Monday, July 16 in JAMA Neurology.

In the short communication, doctors led by Hourmazd Haghbayan of the University of Toronto assembled data on the extent of exposure to the public that the validated clinical tool received. And most importantly, they pointed out that several studies have suggested that such publicity may make the MoCA less reliable for detecting early stages of cognitive decline or dementia in test takers. Essentially, seeing or taking the test allows takers to learn the questions—and the answers—resulting in artificially improved scores. This could potentially obscure signs of early cognitive decline in common clinical assessments that rely on the tool, they warn.

“The presence of a learning effect could result in decreased sensitivity, eroding the MoCA’s chief usefulness in identifying patients with mild cognitive impairment,” they write. “Future research is warranted to validate test performance in populations with direct exposure to the questionnaire’s content or grading scheme.”

In the meantime, Dr. Haghbayan and colleagues recommend that clinicians ask patients if they’ve been exposed to the MoCA before giving it to them. They also recommend that clinicians use alternative versions than the one most commonly highlighted in news articles—i.e., version one as opposed to versions two and three.
Trumping the test

Before their warning and recommendations, the doctors tried to assess the scope of the test’s exposure. Simple Google News and Google Trends data suggested wide coverage. The doctors found 190 unique news articles that discussed MoCA in the context of Trump’s testing. Of those, 53.7 percent (102 articles) included all or some of the test’s questions, with 44 percent (84 articles) including the test in its entirety. Seventeen percent invited readers to take it, and nearly nine percent published the answer key. Outside of news articles, Internet searches for the MoCA skyrocketed relative to searches for other cognitive tests, the doctors found.

This is a problem, as previous research suggested that people can easily learn the test and score better after exposure to it. For instance, a 2015 study involving 468 participants found that those who repeated the test within a year of first taking it were able to improve their score. Another study from 2015 found similar evidence of a learning effect. Most importantly, the study indicated that those who initially scored low on the MoCA earned a significantly higher score when they took it again 12 months later.

The authors concluded:
These results suggest that the MoCA may be susceptible to practice effects, particularly between the first and second administrations. These practice effects should be taken into consideration when repeatedly employing the MoCA to screen for cognitive status in healthy older adults.




Contentious conjecture

Wide coverage of the MoCA and the president’s test results followed intense media attention on Trump’s potential mental state and health—and the debate over whether such attention was acceptable. According to the White House’s medical staff, Trump was the first president to undergo such cognitive testing, which was done at his own request amid all of the media attention.

In the past, such media attention has been strictly off-limits. Ethical standards, such as the Goldwater rule adopted by the American Psychiatric Association in 1973, prohibit mental health professionals from speculating publicly on the mental state of public figures whom they have not personally examined. However, in the current political climate, some doctors and journalists have flouted such rules. They argue that their speculation is for the greater good if they see a danger in a public figure’s behavior. Others have argued to uphold the ethical standard, pointing out that diagnoses at a distance are not reliable and that tossing around labels can increase the stigma of mental health conditions.

Journalists at FiveThirtyEight added to the argument, saying that media speculation on the mental state of public figures is “beyond the scope of our profession.” They also argued that in the case of Trump, it isn’t useful. Many of the characteristics and behaviors that have drawn media speculation were observable in the president prior to his election.

The new letter in JAMA Neurology may extend the argument even further, suggesting that focus on a public figure’s mental state can have additional widespread negative consequences—such as potentially weakening the usefulness of a common assessment tool.