Music-enhanced Data Tables: A New Way of Experiencing Data Presentations

Angel Bermudez, Shamima Yasmin, Department of Computer Science and Electrical Engineering, Eastern Washington University, 526 5th St, Cheney, WA 99004

A multisensory application incorporates multiple senses, i.e., audio, vision, and touch. Incorporating multiple senses in a unimodal visual presentation can make the overall environment immersive and engaging. In data visualization applications such as heatmap-based presentations, bars are color-coded; variables are represented in terms of colors. In a 3D heatmap, bars can be depth-enhanced according to the values of variables. Hence, in a 3D heatmap, bars are of different heights which can be compared with the keyboard of a piano or other musical instruments. Music is essential to humans; it makes us happy. This research aims to incorporate music in data presentations; as users’ hands navigate a heatmap via the touch-screen of a laptop, ipad, or similar device, the heatmap will be sonified. Depths of bars in a heatmap are extracted as users’ hands navigate it. This facilitates playing a particular song or music with different pitches the same way heatmaps are color coded or depth-enhanced in visual presentations. The tonal variations will be performed by mapping to values of variables, which will vary depending on the data range. A music-enhanced data table can make data accessible to the blind and visually impaired (BVI). In addition, users will be able to upload music or songs of their choice. The aim of this research is to investigate how a visual presentation can be successfully represented by popular music. In the basic application, sound waves will be mapped to data heights. While navigating  from one bar to the other, smooth transition of sound waves will be ensured which may necessitate representing bars with  waves of varying amplitudes instead of traditional rectangular bars. The overall approach will be further explored in mapping music to functionally defined 3D models. The same approach will be explored in relaxation therapy via incorporating music into different visual presentations.

Additional Abstract Information

Presenter: Angel Bermudez

Institution: Eastern Washington University

Type: Oral

Subject: Computer Science

Status: Approved

Time and Location

Session: Oral 8
Date/Time: Tue 5:00pm-6:00pm
Session Number: 815
List other presenters in this same room and session