Abstract
Virtual agents, as a promising technology for human-computer interaction, have become focus of research community in resent years. They serve as communicative fellows in a variety of applications. Employing virtual agents to realize human-computer communication on the web is a promising way to make the interaction attractive. In order to make use of intelligent interaction in the web by virtual agents, an important issue is that we should have a scripting language, which is easy to be used by authors. In this chapter, we discuss our research on the Multimodal Interaction Markup Language (MIML), which is a powerful and easy-to-use XML-based language. Different from the related languages in existence, MIML can script not only the presentations of virtual agents, but also their affective capability. We will describe the architecture of MIML, the facial expression recognition, speech emotion recognition, emotional speech synthesis ActiveX controllers and illustrate one scenario that instantiates the affective web-based human-agent interaction scripted by MIML. With the MIML we designed, web-based affective interaction can be described and generated easily.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Allen, J., Byron, D., Dzikovska, M., Ferguson, G., Galescu, L., Stent, A.: Towards Conversational Human-Computer Interaction. AI Magazine 22(4), 27–38 (2001)
Cassell, J., Sullivan, J., Prevost, S., Churchill, E.: Embodied Conversational Agents. The MIT Press, Cambridge (2000)
Picard, R.: Affective Computing. The MIT Press, Cambridge (2000)
Preece, J., Rogers, Y., Sharp, H.: Interaction Design, Beyond Human-Computer Interaction. John Wiley&Sons, Inc., Chichester (2002)
MIT Media Lab, http://affect.media.mit.edu/
Toda, M.: Basic Structure of the Urge Operations, in the urge theory of emotion and cognition. SCCS Technical report, Chuyko University, Nagoya (1994)
Minsky, M.: The Society of Mind. Simon and Schuster, New York (1986)
Prendinger, H., Descamps, S., Ishizuka, M.: MPML: A Markup Language for Controlling the Behavior of Life-like Characters. Journal of Visual Languages and Computing 15(2), 183–203 (2004)
Prendinger, H., Ishizuka, M.: Life-Like Characters-Tools, Affective Functions and Applications. Cognitive Technologies Series. Springer, Heidelberg (2004)
Marriott, A., Stallo, J.: VHML - Uncertainties and problems, A discussion. In: Proc. AAMAS 2002 Workshop on ECA-Let’s Specify and Evaluate Them, Bologna, Italy (2002)
VHML Home Page, http://www.vhml.org/
Cassell, J., Vilhjalmsson, H., Bickmore, T.: BEAT: The Behavior Expression Animation Toolkit. In: Proc. SIGGRAPH 2001, Los Angeles, USA, pp. 477–486 (2001)
DeCarolis, B., Caroglio, V., Bilvi, M., Pelachaud, C.: APML: a Mark-up Language for Believable Behavior Generation. In: Proc. AAMAS 2002 Workshop on ECA-Let’s Specify and Evaluate Them, Bologna, Italy (2002)
Kopp, S., Wachsmuth, I.: Synthesizing Multimodal Utterances for Conversational Agents. Computer Animation and Virtual Worlds 15(1), 39–52 (2004)
Kopp, S., Krenn, B., Marsella, S., Marshall, A., Pelachaud, C., Pirker, H.: Towards a Common Framework for Multimodal Generation: the Behavior Markup Language. In: Gratch, J., Young, M., Aylett, R.S., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 205–217. Springer, Heidelberg (2006)
Heylen, D., Kopp, S., Mareslla, S., Pelachaud, C., Vilhjalmsson, H.: The Next Step towards a Function Markup Language. In: Prendinger, H., Lester, J.C., Ishizuka, M. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 270–280. Springer, Heidelberg (2008)
Heylen, D., Maat, M.: A Linguistic View on Functional Markup Language. In: Proc. AAMAS 2008 Workshop on Functional Markup Language, Estoril (2008)
Kreen, B., Sieber, G.: Functional Markup for behavior Planning. Theory and Practice. In: Proc. AAMAS 2008, Estoril (2008)
Prendinger, H., Ishizuka, M.: Scream: Scripting Emotion-based Agent Minds. In: Proceeding of AAMAS 2002 Workshop on ECA-Let’s Specify and Evaluate Them, Italy (2002)
Prendinger, H., Descamps, S., Ishizuka, M.: Scripting Affective Communication with Virtual Characters in Web-based Interaction System. Applied Artificial Intelligence (2002)
Prendinger, H., Ishizuka, M.: Virtual Characters Tools, Affective Functions and Applications. Cognitive Technologies Series. Springer, Heidelberg (2004)
Prendinger, H., Ishizuka, M.: The Empathic Companion: a Character-based Interface that Addresses User’s Affective States. Journal of Applied Artificial Intelligence 19(3-4), 267–285 (2005)
MPML Home Page, http://www.miv.t.u-tokyo.ac.jp/MPML/mpml.html
Nischt, M., Prendinger, H., Ishizuka, M.: MPML3D: A Reactive Framework for the Multimodal Presentation Markup Language. In: Gratch, J., Young, M., Aylett, R.S., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 218–229. Springer, Heidelberg (2006)
Ullrich, S., Bruegmann, K., Prendinger, H., Ishizuka, M.: Extending MPML3D to Second Life. In: Prendinger, H., Lester, J.C., Ishizuka, M. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 281–288. Springer, Heidelberg (2008)
TVML Home Page, http://www.nhk.or.jp/strl/tvml/
Badler, N.: Parameterized Action Representation for Virtual Human Agents. The MIT Press, Cambridge (2000)
PAR Home Page, http://hms.upenn.edu/software/par/
Huang, Z., Eliebs, A.: STEP: a Scripting Language for Embodied Agent. In: Proc. of PRCAI 2002 Workshop on Virtual Animated Agent - Tools, Affective Functions and Applications, Tokyo (2002)
Microsoft Agent Home Page, http://www.microsoft.com/msagent
Microsoft, Developing for Microsoft Agent. The Microsoft Press (1998)
SMIL Home Page, http://www.w3.org/AudioVideo/
Huang, T., Chen, L., Tao, H.: Bimodal Emotion Recognition by Man and Machine. In: ATR Workshop on Virtual Communication Environments (1998)
DeSilva, L., Miyasato, T., Nakatsu, R.: Facial Emotion Recognition Using Multimodal Information. In: Han, Y., Quing, S. (eds.) ICICS 1997. LNCS, vol. 1334, pp. 397–401. Springer, Heidelberg (1997)
Mao, X., Xue, Y.L.: Beihang University Facial Expression Database and Multiple Facial Expression Recognition. In: Proc. of ICMLC 2006, pp. 369–372 (2006)
Viola, P.: Rapid Object Detection Using a Boosted Cascade of Simple Features. In: Proc. CVPR 2001, pp. 511–518 (2001)
Mao, X., Zhang, B., Luo, Y.: Speech Emotion Recognition Based on a Hybrid of HMM/ANN. In: Proc. WSEAS 2007, pp. 181–184 (2007)
Moulin, H.: Axioms of cooperative decision making. Cambridge University Press, Cambridge (1988)
Mao, X., Li, Z., Bao, H.Y.: Describing and Generating Web-based Affective Human-agent Interaction. In: Lovrek, I., Howlett, R.J., Jain, L.C. (eds.) KES 2008, Part I. LNCS (LNAI), vol. 5177, pp. 625–632. Springer, Heidelberg (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Mao, X., Li, Z. (2010). Web-Based Affective Human-Agent Interaction Generation. In: Hãkansson, A., Hartung, R., Nguyen, N.T. (eds) Agent and Multi-agent Technology for Internet and Enterprise Systems. Studies in Computational Intelligence, vol 289. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13526-2_15
Download citation
DOI: https://doi.org/10.1007/978-3-642-13526-2_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13525-5
Online ISBN: 978-3-642-13526-2
eBook Packages: EngineeringEngineering (R0)