Design Decisions and Strategies

Playing to Strengths

The immediacy of images and the emotive and physical presence of sound both play to processes of recognition in the brain that persist even when working memory and the ability to encode or compress new memories are severely impaired. I made the decision therefore to restrict the use of text. While words are themselves images and can be recognized, the processing of sentences to make meaning is considerably more difficult and lacks the emotive and physical qualities of speech. In Lauren Wedam's paper "Understanding Verbal Auditory Agnosia," she reports that "lexical processes during unusual input (real-word lists and sentences), but not during normal speech, were correlated with an increase in activation of the right inferior frontal cortex as well as the left inferior temporal cortex [] suggest[ing] a relation to areas responsible for the processing of word meaning. In subsequent trials, similar activation of the right cortex was quite insubstantial during normal speech."

As discussed above, concrete categories persist longer than abstract categories in verbal reasoning. By focusing on personal memories without general references, I hope to minimize distractions and capitalize on the episodic and affective qualities of familiar stories told in familiar voices. The downloadable prototype is populated with photos and stories from a fictional family history. The several voices of people shown in photos could be included both as feedback to interactions and in the primary narrative.

Unburdening Working Memory

DAT patients are able to do crossword puzzles, play solitaire, and remain focused on tasks with clear progression from beginning to end. The screen layout of memorable can accommodate any grid. My initial idea was that the columns and rows of a crossword puzzle provided clear structure, showing connections, gaps, and number of steps to completion. It is my unproven hunch that a greater number of elements can be displayed on screen if collectively they can be recognized in the periphery as a single shape. The crossword puzzle layout is intended to help subjects focus particular details of individual photographs while the edges "read" as a unified shape.

Graphics in the prototype are not intended as final, but it is worth mentioning that metallic or glass-like surfaces would not be acceptable. Simple, color-contrasted screen elements that lend directionality to the screen are to be strongly preferred.

same screenshot showing photos on a grid

The Effectiveness of Errorless Learning

Maida Shivik directed me to fascinating research on "Errorless Learning." As a further measure to unburden working memory, I included "I Don't Know" as a valid response to every interaction. As Shivik notes: "This works to keep intrusive (incorrect) information to a minimum which leads to a decrease in confusion and frustration." Whether it is better to accept the response as correct and proceed, or to encourage another response by cueing the subject to some specific detail or concrete association needs to be studied.

Gestural Technologies

Shivik also refers to motor cues as a strategy for helping DAT patients to "encode what he or she is trying to learn with some kind of motor cue or act. Patients who use this kind of encoding, generally do better on retrieving the information than with other types of memory aids, particularly when a retrieval clue is provided at the recall point. (Backman, 1994)." As much as we might try to unburden elements on the screen, it is elements off the screen that may pose the greatest barriers to use. Keyboards, mice, power indicators, and cords are not ideal. Polishing the ivory railing here in my tower, I have employed the language "Touch the photograph" throughout the prototype. As all audio and text prompts are created by caregivers, this is not a hardwired decision. I am thinking here of a wide range of input devices that software for DAT patients might support. Touch screen technologies are becoming commonplace, but flexible displays and pressure sensitive wearable interfaces are also seeking applications. Other gestural technologies include Natural Point which tracks a reflective dot worn on the face, or a ring worn on the hand. The potential of brain actuated technologies like Cyberlink, could also prove interesting since DAT patients may benefit from response and reinforcement for thoughts that remain un-communicated, hunches, etc. Strange as it may sound, Cyberlink is a functional and commercially available product.

next