rattleye_overview
RattlEye demonstrated at SIGGRAPH 2022. Gaze-based thermal feedback enables vision-like thermal experience.


RattlEye is a new media experience where people feel thermal sensation according to their gaze. Just like you perceive visible lights according to your gaze in vision, you can perceive infrared rays (thermal sensation) according to your gaze in RattlEye. You can "see" the world through your skin. People often react like, "Wow! It works!"


CONCEPT

rattleye_description
When the person sees the sun, they feel warmth. When they see the ocean, they feel coolness.


Infrared ray, along with visible light, is emitted from the sun and thermal light sources and is reflected by the world to reach us. Therefore, in perceiving the world, we not only see visible light with our eyes, but also feel infrared (heat) with our skin. The importance of heat in perceiving the world is highlighted by the fact that rattle snakes see the world through both visible and infrared lights. However, thermal sensation has a lower spatial resolution than vision, and it also does not have the same immediate effect on our attention, consciousness, and perceptual world as vision does. I thought that the strong connection of vision to attention, consciousness, and the perceptual world is due to the fact that vision is perceived at high refresh rate in response to the gaze, something that is constantly interacting with attention, consciousness, and the perceptual world. I also considered that spatial scanning by gaze converts the fast refresh of the limited perceptual unit of central vision into spatial perception. (As an aside, Richard A. Bolt's Gaze-Orchestrated Dynamic Windows seems to have applied similar ideas to the interface.) Therefore, to transform thermal sensation into vision-like sensation, we developed the RattlEye, which provides high refresh rate thermal feedback according to eye gaze.


IMPLEMENTATION

rattleye_side
RattlEye consists of an eye tracker, a visual display (a monitor in this case), a shutter mechanism, and an infrared source. The amount of infrared (heat) provided to the person is calculated based on their gaze position on the visual display. The angle and the shutter mechanism is controlled so that the required amount of infrared is passing through it. The relationship between the gaze and heat is defined differently for different contents. For the demonstration at SIGGRAPH 2022, the amount of heat was predefined for each pixel / object of the images (e.g., the sun is the hottest, while the ocean is the coolest).


VIDEO


DEMOS

SIGGRAPH 2022 Emerging Technologies.

rattleye_installation1
Demo at SIGGRAPH 2022.



rattleye_group
Demo at SIGGRAPH 2022.

PUBLICATIONS

  1. Sosuke Ichihashi, Arata Horie, Masaharu Hirose, Zendai Kashino, Shigeo Yoshida, Sohei Wakisaka, and Masahiko Inami. ThermoBlinds: Non-Contact, Highly Responsive Thermal Feedback for Thermal Interaction. In Special Interest Group on Computer Graphics and Interactive Techniques Conference Emerging Technologies (SIGGRAPH ’22 Emerging Technologies ). 2022. doi

  2. Sosuke Ichihashi, Arata Horie, Masaharu Hirose, Zendai Kashino, Shigeo Yoshida, and Masahiko Inami. High-Speed Non-Contact Thermal Display Using Infrared Rays and Shutter Mechanism. In Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers (UbiComp-ISWC ’21 Adjunct). 2021. Best Paper Award. doi


AWARDS

MIMSVAI Best Paper Award. 2021. UbiComp-ISWS 2021 Adjunct.

CREDITS

Sosuke Ichihashi
Arata Horie
Masaharu Hirose
Zendai Kashino
Shigeo Yoshida
Sohei Wakisaka
Masahiko Inami
Tomoya Sasaki