Skip to main content
Springer Nature Link
Log in

Natural Language Generation as Planning under Uncertainty for Spoken Dialogue Systems

  • Chapter
Empirical Methods in Natural Language Generation (EACL 2009, ENLG 2009)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 5790))

  • 1406 Accesses

Abstract

We present and evaluate a new model for Natural Language Generation (NLG) in Spoken Dialogue Systems, based on statistical planning, given noisy feedback from the current generation context (e.g. a user and a surface realiser). The model is adaptive and incremental at the turn level, and optimises NLG actions with respect to a data-driven objective function. We study its use in a standard NLG problem: how to present information (in this case a set of search results) to users, given the complex trade-offs between utterance length, amount of information conveyed, and cognitive load. We set these trade-offs in an objective function by analysing existing match data. We then train a NLG policy using Reinforcement Learning (RL), which adapts its behaviour to noisy feedback from the current generation context. This policy is compared to several baselines derived from previous work in this area. The learned policy significantly outperforms all the prior approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Baddeley, A.: Working memory and language: an overview. Journal of Communication Disorder 36(3), 189–208 (2001)

    Article  Google Scholar 

  2. Boidin, C., Rieser, V., van der Plas, L., Lemon, O., Chevelu, J.: Predicting how it sounds: Re-ranking dialogue prompts based on TTS quality for adaptive spoken dialogue systems. In: Proceedings of the Interspeech Special Session: Machine Learning for Adaptivity in Spoken Dialogue (2009)

    Google Scholar 

  3. Branavan, S., Chen, H., Zettlemoyer, L., Barzilay, R.: Reinforcement learning for mapping instructions to actions. In: Proceedings of ACL, pp. 82–90 (2009)

    Google Scholar 

  4. van Deemter, K.: What game theory can do for NLG: the case of vague language (keynote paper). In: 12th European Workshop on Natural Language Generation (ENLG), pp. 154–161 (2009)

    Google Scholar 

  5. Demberg, V., Moore, J.D.: Information presentation in spoken dialogue systems. In: Proceedings of EACL, pp. 65–72 (2006)

    Google Scholar 

  6. Gasic, M., Keizer, S., Mairesse, F., Schatzmann, J., Thomson, B., Young, S.: Training and Evaluation of the HIS POMDP Dialogue System in Noise. In: Proceedings of SIGdial Workshop on Discourse and Dialogue, pp. 112–119 (2008)

    Google Scholar 

  7. Henderson, J., Lemon, O.: Mixture Model POMDPs for Efficient Handling of Uncertainty in Dialogue Management. In: Proceedings of ACL, pp. 73–76 (2008)

    Google Scholar 

  8. Henderson, J., Lemon, O., Georgila, K.: Hybrid reinforcement / supervised learning of dialogue policies from fixed datasets. Computational Linguistics 34(4), 487–512 (2008)

    Article  Google Scholar 

  9. Janarthanam, S., Lemon, O.: Learning Adaptive Referring Expression Generation Policies for Spoken Dialogue Systems. In: Krahmer, E., Theune, M. (eds.) Empirical Methods in NLG. LNCS (LNAI), vol. 5790, pp. 67–84. Springer, Heidelberg (2010)

    Google Scholar 

  10. Janarthanam, S., Lemon, O.: User simulations for online adaptation and knowledge-alignment in Troubleshooting dialogue systems. In: Proceedings of SEMdial, pp. 133–134 (2008)

    Google Scholar 

  11. Koller, A., Petrick, R.: Experiences with planning for natural language generation. In: ICAPS (2008)

    Google Scholar 

  12. Koller, A., Stone, M.: Sentence generation as planning. In: Proceedings of ACL, pp. 336–343 (2007)

    Google Scholar 

  13. Lemon, O.: Adaptive Natural Language Generation in Dialogue using Reinforcement Learning. In: Proceedings of SEMdial (2008)

    Google Scholar 

  14. Lemon, O.: Learning what to say and how to say it: joint optimization of spoken dialogue management and Natural Language Generation. Computer Speech and Language (to appear)

    Google Scholar 

  15. Liu, X., Rieser, V., Lemon, O.: A Wizard-of-Oz interface to study Information Presentation strategies for Spoken Dialogue Systems. In: Proceedings of the 1st International Workshop on Spoken Dialogue Systems Technology (2009)

    Google Scholar 

  16. Moore, J., Foster, M.E., Lemon, O., White, M.: Generating tailored, comparative descriptions in spoken dialogue. In: Proceedings of FLAIRS (2004)

    Google Scholar 

  17. Nakatsu, C., White, M.: Learning to say it well: Reranking realizations by predicted synthesis quality. In: Proceedings of ACL (2006)

    Google Scholar 

  18. Oh, A., Rudnicky, A.: Stochastic natural language generation for spoken dialog systems. Computer, Speech & Language 16(3/4), 387–407 (2002)

    Article  Google Scholar 

  19. Paek, T., Horvitz, E.: Conversation as action under uncertainty. In: Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, pp. 455–464 (2000)

    Google Scholar 

  20. Polifroni, J., Walker, M.: Intensional Summaries as Cooperative Responses in Dialogue Automation and Evaluation. In: Proceedings of ACL, pp. 479–487 (2008)

    Google Scholar 

  21. Rieser, V., Lemon, O.: Does this list contain what you were searching for? Learning adaptive dialogue strategies for Interactive Question Answering. J. Natural Language Engineering 15(1), 55–72 (2008)

    Article  Google Scholar 

  22. Rieser, V., Lemon, O.: Learning Effective Multimodal Dialogue Strategies from Wizard-of-Oz data: Bootstrapping and Evaluation. In: Proceedings of ACL, pp. 638–646 (2008)

    Google Scholar 

  23. Rieser, V., Lemon, O.: Learning and evaluation of dialogue strategies for new applications: Empirical methods for optimization from small data sets. Computational Linguistics (subm)

    Google Scholar 

  24. Singh, S., Litman, D., Kearns, M., Walker, M.: Optimizing dialogue management with Reinforcement Learning: Experiments with the NJFun system. Journal of Artificial Intelligence Research (JAIR) 16, 105–133 (2002)

    MATH  Google Scholar 

  25. Stent, A., Prasad, R., Walker, M.: Trainable sentence planning for complex information presentation in spoken dialog systems. In: Proceedings of ACL, pp. 79–86 (2004)

    Google Scholar 

  26. Stent, A., Walker, M., Whittaker, S., Maloor, P.: User-tailored generation for spoken dialogue: an experiment. In: Proceedings of ICSLP (2002)

    Google Scholar 

  27. Sutton, R., Barto, A.: Reinforcement Learning. MIT Press, Cambridge (1998)

    Google Scholar 

  28. Wahlster, W., Andre, E., Finkler, W., Profitlich, H.J., Rist, T.: Plan-based integration of natural language and graphics generation. Artificial Intelligence 16(63), 387–427 (1993)

    Article  Google Scholar 

  29. Walker, M., Stent, A., Mairesse, F., Prasad, R.: Individual and domain adaptation in sentence planning for dialogue. Journal of Artificial Intelligence Research (JAIR) 30, 413–456 (2007)

    MATH  Google Scholar 

  30. Walker, M., Whittaker, S., Stent, A., Maloor, P., Moore, J., Johnston, M., Vasireddy, G.: User tailored generation in the match multimodal dialogue system. Cognitive Science 28, 811–840 (2004)

    Article  Google Scholar 

  31. Walker, M.A., Kamm, C.A., Litman, D.J.: Towards developing general models of usability with PARADISE. Natural Language Engineering 6(3) (2000)

    Google Scholar 

  32. Whittaker, S., Walker, M., Maloor, P.: Should I Tell All? An Experiment on Conciseness in Spoken Dialogue. In: Proceedings of Eurospeech (2003)

    Google Scholar 

  33. Whittaker, S., Walker, M., Moore, J.: Fish or Fowl: A Wizard of Oz evaluation of dialogue strategies in the restaurant domain. In: Proceedings of LREC (2002)

    Google Scholar 

  34. Winterboer, A., Hu, J., Moore, J.D., Nass, C.: The influence of user tailoring and cognitive load on user performance in spoken dialogue systems. In: Proceedings of Interspeech/ICSLP (2007)

    Google Scholar 

  35. Young, S., Schatzmann, J., Weilhammer, K., Ye, H.: The Hidden Information State Approach to Dialog Management. In: ICASSP (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Informatics, University of Edinburgh, UK

    Verena Rieser

  2. School of Mathematical and Computer Sciences, Heriot-Watt University, UK

    Oliver Lemon

Authors
  1. Verena Rieser
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Oliver Lemon
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Faculty of Humanities, Department of Communication and Information Sciences (DCI), Tilburg University, P.O.Box 90153, 5000 LE, Tilburg, The Netherlands

    Emiel Krahmer

  2. Human Media Interaction (HMI), Department of Electrical Engineering, Mathematics and Computer Science (EEMCS), University of Twente, P.O. Box 217, 7500 AE, Enschede, The Netherlands

    Mariët Theune

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Rieser, V., Lemon, O. (2010). Natural Language Generation as Planning under Uncertainty for Spoken Dialogue Systems. In: Krahmer, E., Theune, M. (eds) Empirical Methods in Natural Language Generation. EACL ENLG 2009 2009. Lecture Notes in Computer Science(), vol 5790. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15573-4_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-15573-4_6

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-15572-7

  • Online ISBN: 978-3-642-15573-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics