Q+A CENTER

Point Grey Research (Vancouver, BC, Canada)
University of Dundee (Scotland)
University of Toronto (Canada)

The Imaging Challenge

Dementia is the progressive decline in cognitive functioning due to damage or disease in the brain. Dementia is increasing dramatically as the population ages, and severe problems are expected for health services in the next decade. Assistive devices are needed to alleviate caregiver burden and to maintain user independence. Such devices must be non-invasive, or invisible to the user, making cameras and image processing the ideal solutions in this domain.

The Solution

To help people with dementia remain independent and enjoy higher quality of life, researchers at the University of Dundee (Scotland) and the University of Toronto (Canada) developed a real-time, non-invasive system that assists such people in routine tasks like hand washing by giving audio-visual prompts. The system uses only video inputs and combines Bayesian sequential estimation for tracking hands and towel, with decision theory for computing policies of action—specifically a partially observable Markov decision process. A key element of the system is the ability to estimate and adapt to user internal states, such as awareness and responsiveness. These user states are automatically learned from the video of the user’s behaviors, and can be used to modify the system online, or to inform caregivers of user well-being. In operation, video is grabbed by an overhead Point Grey Research Dragonfly2 IEEE-1394 camera. The tracker processes the video and reports positions of hands and towel to a belief monitor that estimates the user’s current state: what they have done so far and what their internal mental state is. The belief about the user’s state is then mapped into an audio-visual prompt by the policy in such a way that long-term expected benefits are achieved.

The Tools Used

• Point Grey Research Dragonfly2 IEEE-1934 camera (for user trials in summer 2007)
• Point Grey Bumblebee2 IEEE-1394 stereo vision camera (current system)
• Dell Latitude d820 laptop with 2GB memory
• Intel T2500 (2GHz), running Ubuntu Linux v 6.10 and using a UDP-sockets-based IPC with a central broker for logging all communication
• Totem v2.16.3 prompting system
• ViewSonic VA912b LCD display
• The codebase for image processing is written in-house, entirely in C++, and uses a variety of libraries
• Panosonic PV-GS39 to capture video of each event for subsequent analysis
• Video post-processing captured raw DV from the camcorder with Windows Movie Maker then used Virtual Dub with Lagarith codec for video compression

The Difference it Made

The device underwent a trial in 2007 involving users with moderate to severe dementia. The system worked reliably and effectively. In the case of one user, a reduction of 100 percent in the need for human assistance resulted (a caregiver was not required for task completion with the device). This is significant for adults with cognitive disabilities: it increases their independence and promotes their well-being. It is also significant for health providers: it reduces caregiver burden. This pioneering solution is currently being applied to other activities, and is expected to have a considerable impact on future health care services for older adults.