A person perceives the world around them through multiple senses, with concurrent impulses of signals in different sensory modalities, such as a flash and a beep, readily deciphered and merged into a single event. However, real-world dynamic events often generate signals with different onsets and durations across the brain and the senses, raising the possibility of 4-D perception. It is hoped mapping these areas in the brain will enable the development of 4-D or multisensory electronic systems. Now, a study led by researchers at UC San Diego develops a pair of 4-D glasses allowing wearers to be physically ‘touched’ by a movie when they see a looming object on the screen, such as an approaching spacecraft. The team states their device was developed in conjunction with a study to map brain areas integrating the sight and touch of a looming object to aid in their understanding of the perceptual and neural mechanisms involved in multisensory integration. The opensource study is published in the journal Human Brain Mapping.
Previous studies indicate real-world objects approaching or passing by an observer often generate visual, auditory, and tactile signals with different onsets and durations. Prompt detection and avoidance of an impending threat depend on the precise binding of looming signals across the brain and senses. The current study uses multisensory apparatus integrating a direct-view wide-field screen with flexible air hoses to deliver spatially aligned looming visual and tactile stimuli near the face with a varying temporal offset.
The current study assesses the synchrony between a looming ball simulated in virtual reality and an air puff delivered to the same side of the face. Results show the air puff was perceived as completely out of sync with the looming ball when the onset of ball movement and the onset of an air puff were nearly simultaneous. Data findings show the two stimuli were perceived as one when there was a delay between 800 to 1,000 milliseconds between them as if an object had passed near the face generating a little wind.
In experiments using functional Magnetic Resonance Imaging (fMRI), the group delivered tactile-only, visual-only, tactile-visual out-of-sync, and tactile-visual in-sync stimuli to either side of the subject’s face in randomized events. Results show more than a dozen brain areas were found to respond more strongly to lateralized multisensory stimuli than to lateralized unisensory stimuli, and the response was further enhanced when the multisensory stimuli are in perceptual sync.
The team surmises their study demonstrates the perceptual and neural mechanisms of multisensory integration near the face, with potential applications for the development of 4-D multisensory entertainment systems and media. For the future, the researchers state their studies will help to further understand the perceptual and neural mechanisms of multisensory integration, and provide a solid scientific foundation for developing 4-D film.
Source: UC San Diego
Get Healthinnovations delivered to your inbox:
Michelle Petersen is the founder of Healthinnovations, having worked in the health and science industry for over 21 years, which includes tenure within the NHS and Oxford University. Healthinnovations is a publication that has reported on, influenced, and researched current and future innovations in health for the past decade.
Michelle has been picked up as an expert writer for Informa publisher’s Clinical Trials community, as well as being listed as a blog source by the world’s leading medical journals, including the acclaimed Nature-Springer journal series.
Healthinnovations is currently indexed by the trusted Altmetric and PlumX metrics systems, respectively, as a blog source for published research globally. Healthinnovations is also featured in the world-renowned BioPortfolio, BioPortfolio.com, the life science, pharmaceutical and healthcare portal.
Most recently the Texas A&M University covered The Top 10 Healthinnovations series on their site with distinguished Professor Stephen Maren calling the inclusion of himself and his team on the list a reflection of “the hard work and dedication of my students and trainees”.
Michelle Petersen’s copy was used in the highly successful marketing campaign for the mega-hit film ‘Jumanji: The Next Level, starring Jack Black, Karen Gilian, Kevin Hart and Dwayne ‘The Rock’ Johnson. Michelle Petersen’s copywriting was part of the film’s coverage by the Republic TV network. Republic TV is the most-watched English language TV channel in India since its inception in 2017.
An avid campaigner in the fight against child sex abuse and trafficking, Michelle is a passionate humanist striving for a better quality of life for all humans by helping to provide traction for new technologies and techniques within healthcare.