Multisensory integration is the study of how information from the different senses may be integrated by the nervous system. We are taught that there are five senses, sight, sound, touch, taste and smell but neurologists identify many more, some argue that there could be 21.
Proprioception
This can be demonstrated by proprioception - the awareness of our body and its parts. Close your eyes and put out your arms either side of you, now take your index finger and touch your nose. You can do this easily and yet you can’t see your nose as you have eyes shut, however, you’re aware of where it is. Another example is if you take a finger and touch somewhere on your exposed skin. This is the sense of touch, but you’re actually processing more sensory modalities, touch, temperature, pressure and if you pressed too hard, pain. That’s why multisensory is also called multimodal integration, as it is the combination of different sensory modalities.
Multisensory is becoming increasingly explored in many innovative ways. Here are a several examples where it is incorporated into daily life:
4D Movie Theatres
4D film is a marketing term for an entertainment presentation system combining a 3D film with physical effects that occur in the theatre in synchronisation with the film. Effects simulated in a 4D film may include rain, wind, temperature changes, strobe lights and vibration. What 4D movies have done, is take an event which was becoming a regular part of life for some people and turned it into an experience we are willing to pay more for.
3D Printed Food
Biozoon Food Innovations is a German company working with Chef Herbert Thill, the originator of the Smoothfood movement that has taken advantage of the new technology to create accessible meals for seniors who struggle to process solid foods. The company uses fresh chicken, carrots and other ingredients to create a nutritionally balanced array of purées that make up its seneoPro product line. With the help of an edible adhesive, it prints the tasty puree into the shape of the base ingredient. The result is an ordinary meal that can easily be consumed by those who would otherwise struggle with chewing solid foods.
Dining in the Dark
Dining in pitch darkness, being hosted and served by a visually impaired waiter may change your point of view about many preconceived ideas of the world. Our enjoyment of food is heavily influenced by sight, yet many other actors are involved. Removing the visual element from eating allows diners to engage with their food in a completely new way and heightens the taste experience.
Musical Kitchen
Cooking can be a dull and tedious activity for many people, which is why the Cooreator was conceptualised. This 2014 Electrolux Design Lab finalist entry makes cooking an engaging multi-sensory experience. While you prepare your meal, this appliance will do its own mixing, translating your ingredient choices into unique audio tracks.
Passing food in front of the scanner of the device would allow it to identify them instantly and pair certain colours and flavours to particular sounds, rhythms and melodies. Yu Yi's concept appliance would continually generate different sounds to correspond to the particular point in your recipe and even its temperature. You would also have the opportunity to share the Cooreator's cuisine compositions over social media, turning an “Instagrammable meal” into a musical experience.
Auditory Vest
The phenomenon of sensory substitution is fairly-well known anecdotally. When you lose a sense, your other senses compensate. Daniel Kish was a blind man who taught himself echolocation and is now able to hike and ride a bike. This theory has been exploited by Dr David Eagleman who explains in a famous TED talk that the brain doesn’t care where the sensory signal comes from, it can still learn to interpret them. This is how cochlear and retinal implants work. David has developed a vest that detects sounds in the person’s environment with microphones and translates them into vibrating patterns that the wearer can learn to interpret.
AR Texturiser
Using augmented reality (AR) to modify the appearance of a ketchup while keeping the remaining view the same, allowed researchers to change the perceived texture of the sauce to the extent that participants thought it was a different sauce. This could have application in reduced sugar formulations where texture and mouthfeel are a challenge to reinstate.
Chewing Jockey
We have known that the noise that some foods make a when chewed is integral to our enjoyment, the snap when you bite a crisp for example. This device, developed by a team from Keio University in Japan, measures jaw movement and plays mastication sounds back to the consumer at a rate corresponding to the rate of chewing. This has potential in making food more pleasurable for elderly consumers who are unable to chew harder foods.
Cutlery Weight
Heavy cutlery has been shown to increase hedonic response towards the food that is consumed. I personally have favourite cutlery at home and some pieces I refuse to use because the food doesn’t taste as good. Can we use this effect to make healthier foods taste better?
Facial Recognition for Sensory
Using facial recognition software, consumer’s responses can be measured to determine the validity of their response, do they really like the product or are they being polite? This technique also streamlines the survey process, if a simple hedonic response is required, this feedback is given in seconds.
Multisensory Retail Space
As we increasingly shop online, retailers are having to deliver more exciting shopping experiences to draw consumers. With the advent of smart fridges, we may no longer need to shop for our basic groceries as they will be ordered automatically, making food shopping an occasion-driven activity. To make this more appealing we could see retailers developing immersive multisensory areas in their shops, targeted to the occasion you are shopping for. Imagine shopping for a picnic and feeling the warm sun on your skin, the smell of freshly cut grass in the air and the hum of bees and birds. Or if we can, replicate the sensation of being on holiday with immersive technologies. Perhaps this could be used to encourage consumers to be more adventurous in their purchasing and open to trying more sustainable options such as insects for proteins.
Treatt and Multisensory
At Treatt, we exploit the multisensory effect of olfactory referral in our health and wellness range. Using our 100% natural calorie-free sugar distillates and flavours, our customers can reformulate to achieve modest sugar reductions, or for greater reductions give an authentic sugar profile to natural and artificial high intensity sweeteners. To find out how we can reduce the sugar content in your products, while improving the mouthfeel and sweetness of your products, please email@enquiries@treatt.com.
Written by Charlotte Catignani, Research and Development Manager at Treatt