Haptic devices allow touch-based information transfer between humans and intelligent systems -- enabling communication in a salient but private manner that frees other sensory channels. For such devices to become ubiquitous, they must be intuitive, unobtrusive, and wearable. However, the amount of information that can be transmitted through touch is limited in large part by the location and distribution of human mechanoreceptors. Our goal was to create wearable soft devices with distributed multi-modal haptic actuation that is mounted on the body and can also be actively touched by the fingertips. This approach could provide low-resolution haptic feedback appropriate for human perceptual abilities on the body (passive feeling), and also takes advantage of fingertip perception by allowing high- resolution information transfer when needed by the user (active touching). This project aimed to create new forms of communication and lead to increased capabilities of wearable devices that could impact human health and quality of life.

In this project, we developed several open source tools to design, fabricate, and control wearable devices that provide haptic feedback to users. Snaptics (snaptics.org) is a low-cost platform designed for rapid prototyping of fully wearable multi-sensory haptic devices. Snaptics uses modular framework allowing designers to quickly test and replace snaptic modules as desired. All modules are designed to be assembled using 3D printed components and inexpensive off-the-shelf actuators to minimize costs to the designer. Our platform exists to increase community engagement with and accessibility of wearable haptic devices by lowering the technical barrier to entry and cost of creating a wearable haptic device. Syntacts (syntacts.org) is a haptic rendering framework for vibrotactile feedback. It eliminates the need for expensive haptic controllers or custom electronics by leveraging commercial-off-the-shelf audio interfaces. As a complete package, Syntacts provides the software and hardware needed to interface audio devices with low latency, synthesize complex waveforms, and amplify signals to appropriate levels.

We conducted psychophysical experiments with human subjects to better understand the perception of multi-sensory haptic cues that combine vibration, skin stretch, and squeeze. We demonstrated the saliency and usability of Snaptics devices with their equivalent research grade counterparts in two experiments. Results indicate that users perform equally well in perceiving cue sets and in completing tasks when using Snaptics devices as compared to when using their research grade counterparts. Using a custom-designed test bed, we showed that cue amplitude and separate distances affect accurate perception of multi-sensory cues. We also demonstrated that users can detect discrete cues that are provided simultaneously with continuous cues. We designed a custom vibrotactile sleeve (the VT Sleeve) and investigated how cue durations of 100, 200, and 400ms affect the overall localizability of tactile cues presented along the forearm by measuring the response means to six tactile cues, as well as the response variance to each cue (to determine error in response). Tactile location had a significant effect on overall localizability but the durations considered had no effect. We are currently investigating tactile durations outside the 100-400ms range. 

We designed hardware and conducted experiments to better understand the relationship between haptic cue perception and the contact mechanics occurring between the haptic device end effector and the skin. We used our novel grounded test bed to complete formal psychophysical tests under position and force control, to measure the maximum comfort threshold, minimum felt threshold, and the perceptual resolution. Correlations between the psychophysical and contact mechanics measurements show strong correlations between the elastic strain energy measured from the force data in the normal and shear experiments and the comfort threshold and discrimination thresholds in the normal and shear directions. The resulting data inform device design processes and provides a framework for achievable performance on an individualized basis, leading to more salient and effective haptic feedback devices.

Finally, we developed wearable haptic devices and explored through pilot experiments how such devices can be used to elicit predictable emotional responses in users, with the goal of incorporating these devices as tools for emotion regulation, an approach used to treat mental disorders. We also conducted pilot experiments to better understand the neural correlates of haptic perception. We conducted an exploratory study to evaluate Representational Similarity Analysis (RSA) of electroencephalography (EEG) signals as a general method and specifically used our EEG-RSA framework to evaluate changes in the neural representation of haptic cues after association training, where the subject learned to map discrete phonemes to multi-sensory haptic cues. Our results suggest that training leads to a sharpening of the sensory response to haptic cues such that after training the neural representation of haptic cues starts to reflect the features of the cues themselves.

Throughout the project effort, we developed educational materials to teach concepts with haptics, we provided our tools and results through open access (software and publications), and we disseminated our findings through publications and presentations.

Last Modified: 10/07/2022
Modified by: Marcia K O'malley