Artificial intelligence (AI) has the potential to transform the screening of dementia, a disease that affects millions of people worldwide. A project based in Singapore, Project Pensieve is developing a digital screening tool that utilises AI to analyse drawings to detect early signs of dementia in users.
In collaboration with the Department of Psychiatry at Singapore General Hospital, GovTech has developed a tool that incorporates machine learning technology and local clinical expertise to accurately estimate the risk of dementia in just 10 minutes. This initiative is part of GovTech’s ongoing effort to use AI to detect dementia earlier, improve healthcare and enable seniors to live active, fulfilled lives.
The research and development phase of Project Pensieve will continue through 2023. By utilising AI, the tool is expected to facilitate more efficient and effective early screening for dementia within the community.
According to Associate Professor Liew Tau Ming, Principal Investigator of Project Pensieve and a senior consultant psychiatrist at Singapore General Hospital, “The timely detection of dementia enables those with the condition to access medications that can slow down the progression of the disease and interventions that help manage related behavioural symptoms. Additionally, caregivers will also have access to important community resources for dementia care.”
The OpenGov Asia Recognition of Excellence Award recognises Project Pensieve for its significant contribution to the AI and dementia screening field.
Utilising AI for early detection of dementia
“Singapore is experiencing its silver wave now,” says Tan Congyuan, Product Manager, Data Science and Artificial Intelligence Division, Government Technology Agency (GovTech), Singapore. “Today, one in seven Singaporeans are seniors. By 2030, we expect one in four Singaporeans aged 65 and above. That’s a projection of more than 900,000 seniors.”
Presently, one out of every ten senior citizens in Singapore has dementia. By 2030, it is anticipated that Singapore will have 152,000 individuals suffering from dementia. Unfortunately, up to 70% of dementia patients in Singapore remain undiagnosed until they have reached a later stage of the disease.
Despite the glaring gap in dementia diagnosis, there has been no efficient and scalable solution to address it at the population level. However, Singapore General Hospital and GovTech saw the potential of AI in developing a solution as the technology matured. Thus, they embarked on this journey in 2020, leveraging each other’s expertise and strengths in dementia care and machine learning (ML).
According to Congyuan, while several cognitive tests such as Min-Cog and Mini-Mental State Examination can assist in the early diagnosis of dementia, they have deployment limitations since each test requires 20 to 30 minutes to administer.
They are labour-intensive and frequently necessitate specialised training before they can be used by a healthcare professional. Moreover, some of these tests are dependent on the respondent’s literacy skills.
Drawing tests, on the other hand, do not rely on the respondent’s literacy level, making them more inclusive and accessible to a wider population. The test is self-administered via an app on a tablet using prerecorded audio and video prompts. It typically takes between five and ten minutes to complete and the AI instantly and automatically scores the test results.
One of the obstacles to early diagnosis of dementia is the current healthcare system, in which the resources required to diagnose dementia, especially the availability of trained professionals to administer cognitive tests, are concentrated at specialised memory clinics in tertiary hospitals. However, many community-based seniors rarely visit tertiary hospitals, limiting their access to professional cognitive evaluations.
Project Pensieve’s objective is to enable non-experts, such as volunteers, caregivers, or administrative staff from the community, to administer the exam, broadening the reach and scope of dementia screening.
The journey towards a minimum viable product
Choosing and adapting the AI algorithm to enhance the accuracy and robustness of identifying dementia cases was a critical technical consideration. Consequently, the Project Pensieve team tested the concept against a small set of synthetic data using few-shot learning techniques, which gave them the confidence to develop a minimum viable product (MVP).
The minimum viable product (MVP) gathered a dataset of 1,000 senior participants’ drawing strokes to enhance the AI model’s accuracy using actual data. Initial results showed promising inference accuracy rates of over 80%. As the project is ongoing, the precision of the AI model will improve as more data is collected.
Using a digital tablet, a participant is required to complete four drawing tests on the app which record the sequence of strokes made by the participant. The AI model then evaluates the stroke sequences and drawings to estimate the risk of dementia.
The project’s AI engineers Jessica Foo and Howard Yang employed multiple AI techniques for this project, including meta-learning, which helped them overcome the lack of data in the project’s early stages. This technique teaches a model to “learn to learn”, allowing for the rapid generalisation of new information.
To better capture temporal information, the engineers explained that they used long short-term memory (LSTM) and gated recurrent units (GRUs) to process images as stroke sequences. By analysing stroke sequences rather than the conventional pen-and-paper drawing, the AI can detect anomalous drawing sequences or prolonged hesitation when drawing, which may indicate cognitive difficulties.
More than 600 seniors participated in the project’s pilot and underwent memory assessments conducted by dementia specialists. Several individuals with dementia and mild cognitive impairment, which are early indicators of dementia, were picked up by the pilot.
“By recruiting community-dwelling seniors as volunteer participants, Project Pensieve was able to identify positive markers of dementia in individuals who might have gone undiagnosed otherwise,” Congyuan says excitedly. “These individuals were then referred to the memory clinic for follow-ups and further evaluation.”
Designing Project Pensieve with the user in mind
Designing for user-friendliness was one of the considerations, particularly for elderly users who are not digital natives and have little experience with interactive screens. Users needed to become familiar with the app so that the inputs collected were accurate and not influenced by the friction of drawing in a new medium.
The team conducted user experience research with clinicians and patients in the early stages of development, using tools such as affinity diagrams, user personas and emotional journey mapping to extract insights and themes to guide the app’s development.
Rachel Shong, Pensieve’s UX designer, engaged seniors at void decks and Residents’ Committees (RCs) to demonstrate mock-ups and gather feedback. The team had to design the user experience from the perspective of seniors with varying cognitive abilities, keeping in mind that they may have little experience with tablets and interactive screens.
The team was mindful throughout the process that inclusive design stems from empathy and that the human element must not be overlooked in the design process.
Rachel revealed that when designing the test setup, they considered a range of factors: “For example, selecting a stylus that resembles the classic hexagonal 2B pencil without any side buttons to recreate a familiar feel and reduce apprehension arising from tech novelty.”
The team intentionally chose a tablet that was appropriately sized and not too heavy for the participants to handle. They also paid attention to the user interface, including the visual contrast of the app and the size and style of the fonts, to ensure readability for seniors.
Project Pensieve maintains a human-centred approach by inviting citizen co-creation and participation. The team engaged the public through various events and platforms during the product development phase, such as GovTech’s Tech Kaki group and IMDA’s Digital for Life event.
Supporting existing dementia initiatives with Project Pensieve
Project Pensieve serves as a significant addition to the current dementia support initiatives, as the detection and intervention process is complex and various factors can affect the outcome. For example, even if a person is identified as being at risk for dementia, they may choose not to pursue follow-up due to stigma, anxiety, financial constraints, lack of knowledge, family circumstances, or other reasons.
“The healthcare system also needs to be robust enough to support the additional load of picking up more people with early dementia,” Congyuan opines.
It is important to note that AI should only be used as an aid in the diagnosis of dementia, and not as a replacement for a professional diagnosis. Therefore, it is recommended that individuals identified as potentially having dementia through AI testing be referred to a trained clinician for a more comprehensive diagnosis and appropriate intervention.
Adjustments will be made to the sensitivity and specificity of the AI as more data comes in. Sensitivity represents the true positive rate, and specificity represents the true negative rate. A test with high sensitivity will reliably detect the presence of a condition, resulting in a high number of true positives and a low number of false negatives. This is especially crucial when the repercussions of missing the condition are severe. A test with a high specificity reliably excludes individuals who do not have the condition, resulting in many true negatives and a small number of false positives. This is especially crucial when people with a condition are subjected to additional testing, expense, stigma, and anxiety, among other things.
AI can and is already being used for a variety of tasks such as interpreting imaging results, ensuring medication adherence, and detecting abnormal behaviour that could be a symptom of a problem, such as a fall detection device.
The team is actively investigating the optimal application of AI to healthcare needs.
About the team
The Project Pensieve team is committed to achieving its goal of providing public benefit through its collaborative work. Clinical experts from Singapore General Hospital develop the drawing tests and administer and interpret them to provide ground truth data to train the AI.
GovTech has software developers building the app, AI and data engineers who build the AI engine to improve the accuracy and robustness of detecting dementia and user experience designers who conceptualise features and the user journey to deliver a good experience.
The team demonstrates empathy towards the user’s experience, strives for continuous innovation and delivers a superior product by putting themselves in the shoes of all stakeholders.
Interacting with seniors whose cognitive abilities span a broad spectrum is a good example of empathy. The team must demonstrate sensitivity and patience when approaching, understanding and communicating with the elderly.
The project team must be able to communicate its business case and the potential value of the project to stakeholders to gain their support. Stakeholders can evaluate the cost-benefit ratio and determine the level of support for the project if they are informed of progress, obstacles and successes.
“For example, challenges arising from the COVID situation led to a slower outreach and uptake of recruitment at the start, necessitating some schedule adjustments,” Congyuan explains. “Being transparent in the recovery action and demonstrating successful delivery of intermediate milestones gave the stakeholders confidence in their support to overcome the hurdles.”
The Project Pensieve team includes subject matter experts from various fields, including clinical, user experience, and artificial intelligence. Open communication within the team is vital as it ensures that everyone’s domain expertise is shared and not withheld and members know whom to approach when confronted with a complex problem.
For instance, the AI engineer must comprehend how clinicians interpret features within a drawing to optimise the AI’s ability to zero in on areas of interest.
Congyuan shared that the team uses Kanban to track work transparently and keep each other informed of upcoming tasks or backlogs. This allows the team to pursue incremental, evolutionary improvements throughout the phases of the project, as app development progresses concurrently with participant usage.
“Team members are empowered to be leaders and initiate enhancements to the solution and workflow while managing team capacity,” Congyuan concludes.
Congyuan is optimistic about the future of AI technology in healthcare and is excited to see what it can develop to better serve people. Thanks to the dedicated efforts of his team who share similar values, he believes that this industry will continue to thrive and grow in the years ahead.
