How access to an empathic healthcare provider be ensured 24/7, and healthcare professionals be trained to deal with unusual situations? The project Virtual Humans in the Brabant Economy (ViBE) addresses these questions via the development, testing and deployment of embodied conversational agents (ECAs) in healthcare. The ECAs will communicate with people through natural language and non-verbally (via body posture, eye contact, etc.), allowing them to support training of healthcare professionals, and provide emotional care for patients. More broadly, the agents developed by VIBE will be applicable to any simulation (including virtual, mixed and augmented reality), with the goal to allow for customization for various domains so that training and education programs be enhanced.
VIBE has received considerable attention in the national media, in newspapers, television and radio. The project totaling 7 million euros is funded by the European Union, OP Zuid, the Ministry of Economic Affairs and the municipality of Tilburg. It involves twelve partners; five knowledge institutions and four companies (who together bring thorough knowledge of algorithms and related VR technology), and two hospitals (who offer medical expertise and living lab environments for testing and validation of the developed agents). Tilburg University leads this project.
Societal impact
VIBE focuses on two main activities. First, Mind Labs, in the ‘Proeftuin Spoorzone Tilburg’, where we are working on the development of a basic technology platform for efficient development of ECAs. Secondly, the Living Lab Health, which has two application domains (obstetrics in Maxima Medical Center and bariatrics in Spaarne Hospital) where the developed agent technology will be applied. This project therefore manifests itself in a two-fold business case. First, two concrete business cases for the medical sphere will be explored. Secondly, the basic technology platform can be further applied in various sectors and social domains, including maintenance (virtual companion), aerospace (virtual instructors), and education (intelligent tutoring systems).
Scientific impact
VIBE strives for a radical change in the ECA development and human-agent interaction. Current ECAs do not excel in their intelligent behaviors. Existing ECAs have robotic appearance and respond by means of pre-scripted or spoken text, they also do not learn from interactions with ‘real’ humans, and fail to respond to changing situations. In addition, the interaction between humans and the current ECAs is typically kept at a very basic level, in that ECAs cannot respond to non-verbal signals of their interaction partners, nor carry on a natural conversation for a long period. By pushing forward the frontiers of Artificial Intelligence, Cognitive Science, Psychology, and Human-Technology Interaction, VIBE aims to overcome the above shortcomings in order to deliver a humanlike agent that is capable of natural communication and behaviour, and that can also be applied in many sectors.
Publications
Blomsma, P., Vaitonyte, J., Alimardani, M., & Louwerse, M.M. (2020). Spontaneous facial behavior revolves around neutral facial displays. Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents. Glasgow (p.1-8).
Blomsma, P., Linders, G., Vaitonyte, J., & Louwerse, M.M. (2020). Intrapersonal dependencies in multimodal behavior. Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents. Glasgow (p.1-8).
Linders, G. & Louwerse, M.M. (2020). Zipf’s Law in Human-Machine Dialog. Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents. Glasgow (p.1-8).
Louwerse, M.M., Nilsenova-Postma, M., Van Limpt-Broers, H.A.T., de Back, T.T., Tinga, A.M. & Horden, M. (2021). Beyond the frontiers of education: How immersive media change the way we learn. ITU Journal: ICT Discoveries.
Vaitonyte, J., Blomsma, P. A., Alimardani, M., & Louwerse, M. M. (2019, July). Generating Facial Expression Data: Computational and Experimental Evidence. In Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents (pp. 94-96). ACM.
Vaitonytė, J., Blomsma, P. A., Alimardani, M., & Louwerse, M. M. (2021). Realism of the face lies in skin and eyes: Evidence from virtual and human agents. Computers in Human Behavior Reports, 3, 100065.
Press coverage (selection)
https://www.europaomdehoek.nl/nieuws/virtuele-mens-gaat-zorg-en-onderwijs-ontlasten
https://www.ed.nl/veldhoven/robot-victoria-komt-tot-leven-in-veldhoven~a66d7f24b
https://www.tilburgers.nl/tag/max-louwerse/
https://www.npo3fm.nl/radio/fragmenten/291370-binnen-4-jaar-kunnen-we-vr-opereren
https://www.ed.nl/brabant/brabants-miljoenenproject-met-avatars-in-ziekenhuizen~a1fdad60/
https://www.okvisie.nl/nieuws/training-van-zorgmedewerkers-door-avatars-in-virtual-reality
https://www.skipr.nl/nieuws/amphia-mmc-en-spaarne-gasthuis-gaan-trainen-met-avatars/