NewsSource: EClinCloud official website historical archive

Inconsistent Scale Scoring? Understanding Neuropsychological Scale Assessment

8 min readEClinCloud Editorial Team
Article cover for "Scale scoring: a different score from every rater? An overview of the history of neuropsychological scale assessment"

The following article is from the " DXY NeuroTime (丁香园神经时间) " WeChat official account

MoCA评分标准化培训交流截图  

In neurology, neuropsychological assessment is an indispensable tool for clinical diagnosis and scientific research. However, when conducting clinical scale ratings, raters may apply varying degrees of subjective judgment and interpret scales inconsistently, leading to discrepancies in scoring results; during the assessment, leading language or questions from a rater can also introduce bias into the results. Particularly in the field of neuropsychology, neuropsychological assessment results often serve as the primary endpoint of a clinical trial, and bias in the results may reduce the reliability of a clinical trial and, in severe cases, lead to study failure . Therefore, standardized training on the use of neuropsychological assessment scales is necessary so that raters can assign reliable, accurate, and more consistent scores based on the patient's actual condition [1] .

 

On 5/6–7, the 4th Annual Academic Conference of the Chinese Stroke Association Brain Health Branch was successfully held, during which EClinCloud co-organized the "Neuropsychological Scale Assessment Training Session", aimed at training cognitive assessors and specialist physicians in cognitive impairment and promoting the standardized use and consistency evaluation of scales.

 

Standardizing Clinical Scale Assessments and Improving Rating Consistency

神经心理量表评估培训活动合影

Figure. Remarks by Professor Shi Jiong

 

At the start of the meeting, as the initiator of this training session, Vice President of the First Affiliated Hospital of the University of Science and Technology of China (Anhui Provincial Hospital) Professor Shi Jiong delivered wonderful remarks.

 

"Scales are an indispensable tool in the clinical diagnosis and scientific research of cognitive impairment-related diseases. Most drug development for cognitive impairment diseases—whether already approved or in clinical trials—uses scale outcomes as primary or secondary endpoints in protocol design, making them key to the success or failure of a clinical trial. However, due to the inherent limitations of scales themselves, the use of scales still requires further development and improvement; how to make the information collected by scales more accurate and objective is a primary focus in both clinical application and research.

 

"Currently, in clinical practice, teaching, and scientific research, the process of using and assessing scales faces major challenges, lacking unified assessment standards and standardized requirements. This theoretical and hands-on training on commonly used neuropsychological scales is organized to promote and implement the standardized application of paper and electronic neuropsychological scales in the field of neurology, improve the accuracy, consistency, and objectivity of scale assessments, and support the use of scales in clinical practice, teaching, and research , and advance and standardize the application of domestic Chinese scales to bring benefits to patients."

 

神经心理评估培训讲座MMSE量表培训讲座

Fig. Director Li Shiping (left) and Dr. Lu Xinyi (right) during on-site training

 

At the meeting, Director Li Shiping of Beijing Tiantan Hospital, Capital Medical University, first gave an engaging presentation on the general principles of neuropsychological assessment and the standardized application of the CDR (Clinical Dementia Rating). Subsequently, Dr. Lu Xinyi of The First Affiliated Hospital of the University of Science and Technology of China (Anhui Provincial Hospital) provided a comprehensive and accessible review of the standardized application of the MMSE (Mini-Mental State Examination), MoCA (Montreal Cognitive Assessment), and ADCS-ADL (Alzheimer's Disease Cooperative Study–Activities of Daily Living). The training content was detailed and practical, and participants widely remarked that they found it highly beneficial and rewarding.

 

CDR Scale

 

This scale is used to assess the degree of cognitive impairment in patients and has higher sensitivity and specificity than other scales. It primarily assesses areas such as memory, orientation, judgment and problem-solving, social functioning, housework and hobbies, and self-care.

 

MMSE Scale

 

The MMSE is one of the most widely used screening tools for cognitive impairment. It includes orientation to time and place, language (repetition, naming, understanding instructions), mental arithmetic, immediate and short-term auditory verbal memory, and constructional copying, among other items, with a total score of 30 points. It primarily provides a brief assessment of orientation, memory, language, calculation, attention, and other functions, taking 5–10 minutes. It can also be used to screen for mild cognitive impairment (MCI)[2] .

 

MMSE评估表第一页示例MMSE评估表续页示例

Figure. MMSE Scale

 

MoCA Scale

 

The MoCA scale is used for rapid screening of cognitive impairment. It includes short-term memory; visuospatial abilities; executive functions; attention, concentration, and working memory; language; and orientation to time and place. It takes approximately 15 minutes and is more sensitive than the MMSE. In addition to MCI, it can also be used to screen for Parkinson's disease dementia.[2]

 

ADCS-ADL Scale

 

This scale evaluates the ability to perform daily activities, with 23 items in total, including shopping, housework, using transportation, recreational activities, and travel. Completing these activities is a necessary condition for returning to society; the assessment results reflect relatively fine motor function and are commonly used for older adults and people with disabilities in the community.

 

 

Advancing Scale Digitization and Improving Data Collection Quality

 

For many years, paper scales have been used for patient assessment in both clinical practice and scientific research, but over time, their drawbacks have gradually become apparent: inconsistent scales across multiple sites and bias in retrospective analyses; increased workload at the data entry stage; delayed data synchronization across multiple sites; differing interpretations of scales among researchers; heavy burden of storing and managing paper data; high completion burden and poor compliance among trial participants; patient privacy breaches, and various other issues. Using paper scales entails high hidden costs and requires an even greater investment in labor.

 

In recent years, patient-centric clinical efficacy evaluation models have gradually gained recognition, and in China, patient-reported outcome (PRO) research has also begun to receive attention. Electronic scale data capture systems are developing rapidly in the pharmaceutical industry, yet there is still substantial room for their application in clinical settings. Advancing and refining the clinical application of electronic scales is of great significance for improving convenience for both physicians and patients.

 

Compared with paper COA, electronic COA (electronic clinical outcome assessment system, eCOA) has clear advantages in data collection efficiency, real-time capability, flexibility, compliance, security, and patient privacy protection.

 

Clinical value: The eCOA system can remind trial participants to enter data in multiple formats and across multiple dimensions, ensuring completeness, timeliness, and accuracy in data collection; its timestamping function can effectively identify and prevent behaviors that affect data reliability, such as backfilling or early responses.

Research value: In the eCOA back end, monitoring data can be reviewed in real time, allowing investigators and data managers to conduct online data management and remote data monitoring in real time, flag questionable data with queries, and follow up with trial participants promptly. Moreover, eCOA offers significantly higher efficiency of use than manual paper-based assessments and facilitates patient information management, thereby achieving more complete, accurate, timely, and compliant scale data collection. eCOA can support data standardization for clinical trial endpoints and improve data credibility and reliability, thereby improving study efficiency and quality.

 

However, most scales currently used in clinical research are introduced from abroad, which can easily lead to infringement and compliance issues. EClinCloud scale management provides end-to-end services including scale selection and design for clinical trial protocols, scale copyright acquisition, scale translation, and linguistic validation, and can also provide one-stop clinical endpoint assessment solutions such as rater training and an eCOA system to help support drug marketing approval.

 

Meeting Summary

 

As the training session drew to a close in an engaged and collegial learning atmosphere, Professor Shi Jiong delivered concluding remarks: "The standardized assessment of clinical scales requires the joint efforts of more professionals in the field. In the future, training sessions will be held from time to time to provide scale training at varying levels of depth and breadth based on the needs of clinicians across different levels and specialties. Through industry scale training sessions, exchange meetings, and similar activities, we will bring in accomplished industry experts and invite scholars from various fields to share their experience and lessons learned in working with clinical scales, promoting knowledge sharing and experience exchange; we will also invite relevant speakers to share upstream clinical research knowledge from the sponsor perspective, such as considerations for using scales in clinical protocol design. The scale training sessions aim to transfer broadly applicable experience into scale assessment and use across different levels, specialties, and dimensions, achieving consistency and standardization and thereby raising the quality of both clinical practice and scientific research to a higher level."

 

What is learned on paper alone remains superficial; true understanding comes only through practice. The experts' insights at this training session deeply influenced every participant, fostering growth, stimulating new thinking, and tangibly improving the reliability and validity of scoring results in clinical practice. Upon passing the training, a training qualification certificate in cognitive impairment assessment is issued by the Brain Health Branch of the Chinese Stroke Association. Going forward, EClinCloud will also maintain close collaboration with core committee experts of the Brain Health Branch of the Chinese Stroke Association to actively organize online or in-person training sessions focused on scales, which, in addition to issuing training certificates, will grant continuing medical education (CME) credits, jointly advancing innovation in clinical practice and scientific research in neurology.

 

 

References:

[1] Zhang Zuoji. Handbook of Behavioral Medicine Scales (行为医学量表手册) [M]. Chinese Medical Multimedia Press, 2005.

[2] Hoops, S., Nazem, S., Siderowf, A.D., Duda, J.E., Xie, S.X., Stern, M.B., Weintraub, D., 2009. Validity of the MoCA and MMSE in the detection of MCI and dementia in Parkinson disease[J]. Neurology 73, 1738–1745.. https://doi.org/10.1212/wnl.0b013e3181c34b47

 

 

 

About EClinCloud  

EClinCloud is a technology company focused on clinical research and development that provides an integrated, interoperable, all-in-one clinical trial cloud platform (including eCOA, CTMS, eTMF, EDC, RTSM, RM, and other technology products and offerings, with solutions such as scale design, scale copyright management, linguistic validation, rater training, and data quality analysis, among others) as well as clinical trial endpoint assessment technologies across different therapeutic areas (including central nervous system and ophthalmology, among others), combining cutting-edge technology with life-science data to build a globally leading "patient-centric" SaaS cloud platform supporting clinical evidence, helping domestic and international pharmaceutical companies and research institutions improve R&D efficiency, accelerate the market launch of drugs and medical devices, and benefit countless patients.

 

Related reading

Archived article cover: With Rising Pharmaceutical Submissions and Tightening Investment, How Can Digital Solutions Help Clinical Research Reduce Costs and Improve Efficiency?News7 min readWith Rising Drug Filings and Tightening Investment, How Can Digital Solutions Help Reduce Costs and Improve Efficiency in Clinical Research?As the biopharmaceutical industry continues to develop, innovative drug R&D has entered a period of rapid growth, and the number of clinical drug trials is trending upward. Alongside this, improving clinical trial efficiency and reducing the costs and risks of clinical research have become urgent priorities for a growing number of pharmaceutical companies and research institutions. Using digital tools to advance the digital and intelligent development of clinical trials is undeniably the prevailing trend. To better meet the needs of digital and intelligent clinical development and share the latest achievements and solutions, EClinCloud recently convened in Chengdu its first User Conference, "Digital and Intelligent Empowerment: Symbiosis and Win-Win," bringing together pharmaceutical industry leaders for discussions.Archived article cover: EClinCloud eCOA system passes CFDI inspection, supporting marketing approval of Raynovent's COVID-19 treatment leritrelvir tabletsNews4 min readEClinCloud eCOA System Passes CFDI Inspection, Supporting Marketing Approval of Raynovent's COVID-19 Treatment Leritrelvir TabletsOn March 23, 2023, in accordance with the relevant provisions of the Drug Administration Law and under the special drug approval procedures, the National Medical Products Administration (NMPA) granted conditional marketing approval to leritrelvir tablets (brand name: Leruiling), a Category 1 innovative drug submitted by Guangdong Raynovent Biotech Co., Ltd. (hereinafter referred to as "Raynovent"). The drug is an oral small-molecule therapy for novel coronavirus infection, indicated for the treatment of adult patients with mild-to-moderate novel coronavirus infection (COVID-19).Archived article cover: Committed to serving global clinical research, Microsoft supports EClinCloud's accelerated global expansionNews6 min readCommitted to Serving Global Clinical Research: Microsoft Supports EClinCloud's Accelerated Global ExpansionIn 2015, drug review reform began, and innovation in China's pharmaceutical industry saw explosive growth. With steadily rising R&D investment, innovative drug pipelines expanded rapidly and the number of new drug launches continued to increase. Studies show that over the recent 10-year period (2010–2020), both Investigational New Drug (IND) applications and New Drug Applications (NDAs) in China achieved rapid growth. However, the key metric of "clinical trial phase duration" lengthened from 2572 days to 2688 days. This upward rather than downward trend has made improving clinical trial efficiency one of the most pressing pain points in new drug innovation today. Guided by a core mission centered on digital tools...