Designing Speech, Acoustic and Multimodal Interactions

Abstract

Traditional interfaces are continuously being replaced by mobile, wearable, or pervasive interfaces. Yet when it comes to the input and output modalities enabling our interactions, we have yet to fully embrace some of the most natural forms of communication and information processing that humans possess: speech, language, gestures, thoughts. Very little HCI attention has been dedicated to designing and developing spoken language, acoustic-based, or multimodal interaction techniques, especially for mobile and wearable devices. In addition to the enormous, recent engineering progress in processing such modalities, there is now sufficient evidence that many real-life applications do not require 100% accuracy of processing multimodal input to be useful, particularly if such modalities complement each other. This multidisciplinary, one-day workshop will bring together interaction designers, usability researchers, and general HCI practitioners to analyze the opportunities and directions to take in designing more natural interactions especially with mobile and wearable devices, and to look at how we can leverage recent advances in speech, acoustic, and multimodal processing.

Publisher Link

https://doi.org/10.1145/3027063.3027086

Citation

Cosmin Munteanu, Pourang Irani, Sharon Oviatt, Matthew Aylett, Gerald Penn, Shimei Pan, Nikhil Sharma, Frank Rudzicz, Randy Gomez, Ben Cowan, and Keisuke Nakamura. 2017. Designing Speech, Acoustic and Multimodal Interactions. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '17). Association for Computing Machinery, New York, NY, USA, 601–608. DOI:https://doi.org/10.1145/3027063.3027086

Bibtext Entry

@inproceedings{10.1145/3027063.3027086,
author = {Munteanu, Cosmin and Irani, Pourang and Oviatt, Sharon and Aylett, Matthew and Penn, Gerald and Pan, Shimei and Sharma, Nikhil and Rudzicz, Frank and Gomez, Randy and Cowan, Ben and Nakamura, Keisuke},
title = {Designing Speech, Acoustic and Multimodal Interactions},
year = {2017},
isbn = {9781450346566},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3027063.3027086},
doi = {10.1145/3027063.3027086},
abstract = {Traditional interfaces are continuously being replaced by mobile, wearable, or pervasive interfaces. Yet when it comes to the input and output modalities enabling our interactions, we have yet to fully embrace some of the most natural forms of communication and information processing that humans possess: speech, language, gestures, thoughts. Very little HCI attention has been dedicated to designing and developing spoken language, acoustic-based, or multimodal interaction techniques, especially for mobile and wearable devices. In addition to the enormous, recent engineering progress in processing such modalities, there is now sufficient evidence that many real-life applications do not require 100% accuracy of processing multimodal input to be useful, particularly if such modalities complement each other. This multidisciplinary, one-day workshop will bring together interaction designers, usability researchers, and general HCI practitioners to analyze the opportunities and directions to take in designing more natural interactions especially with mobile and wearable devices, and to look at how we can leverage recent advances in speech, acoustic, and multimodal processing.},
booktitle = {Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems},
pages = {601–608},
numpages = {8},
keywords = {mobile devices, wearable interaction, language interaction, acoustic interaction, natural language processing, speech interaction, multimodal interaction, natural user interfaces, acoustic processing, speech recognition},
location = {Denver, Colorado, USA},
series = {CHI EA '17}
}