Moving from a device that's purely "fun" to work with, to something that has real value and applicability, I've been considering applying the sound-activated cursor mechanism for use in existing assistive technology.
Severe sufferers of cerebral palsy, without voluntary limb, head or tongue movement, have a tough time communicating. Often, they are only able to control their speech using a series of grunt-like noises. The
Sing-a-Sketch mechanism could feasibly be adapted for use by a sufferer of cerebral palsy, who only has the ability to control the duration of vocal noises.
By attaching a bluetooth-enabled headset to their ear, and synchronizing their noises with a cursor moving on screen, a user can navigate through a "switching" communication device driven by software, which is then able to "speak" the sentence to carers.
While the
existing software for assistive technology is very comprehensive, users are still required to have significant control over particular parts of their body, with input devices designed for
finger movement only, tongue and head movement being particularly common.
Even
eye-movement readers are inappropriate for particular cases of cerebral palsy, as the sufferer often has limited control over the position of their head.
The portability of the Bluetooth protocol would allow for light-weight software / hardware pairing with the headset, allowing something as small as a PDA to be able to interpret vocal input into spoken or written messages by both the user and the carer. A carer could conceivably be sent SMS-like messages from the user who transmits textual translations to a web-enabled PDA.
The ever-moving cursor in
Sing-A-Sketch then, would have to be adapted to continually be at the centre of the screen, with only the translation icons and commonly-used buttons scrolling underneath in the direction dictated by the user.
Once certain 'dead zones' (blue in the diagram) in the screen are overcome (allowing for both a sound and interaction threshold), the screen is able to "switch" to the next-lowest branch in the heirarchical tree of iconography, while keeping the cursor at the centre of the screen.
Once the sentence or phrase is complete, the user can navigate to the ever present "speak" or "cancel" buttons to complete the interaction. "Yes" and "No" buttons are omnipresent, and navigating up and down the tree uses a consistent interface.
The screen would "switch" between a series of four options, while also keeping the most-recently selected option in view, allowing a user to navigate both up and down the heirarchy.
Selecting
1, for example, switches to the screen surrounded by green, where the next available options are
5, 6, 7 and, to navigate back,
1.
