Abstract
When writing an academic paper, researchers often spend considerable time reviewing and summarizing papers to extract relevant citations and data to compose the Introduction and Related Work sections. To address this problem, we propose QuOTeS, an interactive system designed to retrieve sentences related to a summary of the research from a collection of potential references and hence assist in the composition of new papers. QuOTeS integrates techniques from Query-Focused Extractive Summarization and High-Recall Information Retrieval to provide Interactive Query-Focused Summarization of scientific documents. To measure the performance of our system, we carried out a comprehensive user study where participants uploaded papers related to their research and evaluated the system in terms of its usability and the quality of the summaries it produces. The results show that QuOTeS provides a positive user experience and consistently provides query-focused summaries that are relevant, concise, and complete. We share the code of our system and the novel Query-Focused Summarization dataset collected during our experiments at https://github.com/jarobyte91/quotes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Uploaded at the following anonymous link: https://streamable.com/edhej9
References
Zhang, J., Zhao, Y., Saleh, M., Liu., P.J.: PEGASUS: pre-training with extracted gap-sentences for abstractive summarization. In: Proceedings of the 37th International Conference on Machine Learning, ICML’20. JMLR.org (2020)
Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA (2002). Association for Computational Linguistics
Lin, C.-Y.: ROUGE: a package for automatic evaluation of summaries. In: Text Summarization Branches Out, pp. 74–81, Barcelona, Spain (2004). Association for Computational Linguistics
Banerjee, S., Lavie, A.: METEOR: an automatic metric for MT evaluation with improved correlation with human judgments. In: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pp. 65–72, Ann Arbor, Michigan (2005). Association for Computational Linguistics
Rush, A.M., Chopra, S., Weston, J.: A neural attention model for abstractive sentence summarization. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 379–389, Lisbon, Portugal (2015). Association for Computational Linguistics
Dang, H.T.: Overview of DUC 2005. In: Proceedings of the Document Understanding Conference. vol. 2005, pp. 1–12 (2005)
Leuski, A., Lin, C.-Y., Hovy, E.: iNeATS: interactive multi-document summarization. In: The Companion Volume to the Proceedings of 41st Annual Meeting of the Association for Computational Linguistics, pp. 125–128, Sapporo, Japan (2003). Association for Computational Linguistics
Cormack, C.V., Grossman, M.R.: Evaluation of machine-learning protocols for technology-assisted review in electronic discovery. In: Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’14, pp. 153–162, New York, NY, USA (2014). Association for Computing Machinery
Brooke, J.: SUS - a quick and dirty usability scale. Usability Eval. Ind. 189(194), 4–7 (1996)
Dang, H.T.: Overview of DUC 2006. In: Proceedings of the Document Understanding Conference. vol. 2006, pp. 1–10 (2006)
Dang, H.T.: Overview of DUC 2007. In Proceedings of the Document Understanding Conference. vol. 2007, pp. 1–53 (2007)
Baumel, T., Cohen, R., Elhadad, M.: Topic concentration in query focused summarization datasets. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, vol. 30 (2016)
Shinoda, K., Aizawa, A.: Query-focused scientific paper summarization with localized sentence representation. In: BIRNDL@ SIGIR (2018)
Cormack, G.V., Grossman, M.R.: Multi-faceted recall of continuous active learning for technology-assisted review. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’15, pp. 763–766, New York, NY, USA (2015). Association for Computing Machinery
Cormack, G.V., Grossman, M.R.: Scalability of continuous active learning for reliable high-recall text classification. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, CIKM ’16, pp. 1039–1048, New York, NY, USA (2016). Association for Computing Machinery
Erera, S., et al.: A summarization system for scientific documents. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations, pp. 211–216, Hong Kong, China (2019). Association for Computational Linguistics
Zarinbal, M., et al.: A New Social Robot for Interactive Query-Based Summarization: Scientific Document Summarization. In: Ronzhin, A., Rigoll, G., Meshcheryakov, R. (eds.) ICR 2019. LNCS (LNAI), vol. 11659, pp. 330–340. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-26118-4_32
Bayatmakou, F., Mohebi, A., Ahmadi, A.: An interactive query-based approach for summarizing scientific documents. Inf. Discovery Delivery 50(2), 176–191 (2021)
Plotly. Dash. Python package, https://plotly.com/dash, 2013. Visited on August 30, 2022
Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using siamese bert-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, vol. 11 (2019)
Pallets Projects. Flask: web development, one drop at a time. Python package, https://flask.palletsprojects.com, 2010. Visited on August 30, 2022
Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22(1), 79–86 (1951)
Acknowledgements
We thank the Digital Research Alliance of Canada (https://alliancecan.ca/en), CIUNSa (Project B 2825), CONICET (PUE 22920160100056CO, PIBAA 2872021010 1236CO), MinCyT (PICT PRH-2017-0007), UNS (PGI 24/N051) and the Natural Sciences and Engineering Research Council of Canada (NSERC) for the resources provided to enable this research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ramirez-Orta, J., Xamena, E., Maguitman, A., Soto, A.J., Zanoto, F.P., Milios, E. (2023). QuOTeS: Query-Oriented Technical Summarization. In: Fink, G.A., Jain, R., Kise, K., Zanibbi, R. (eds) Document Analysis and Recognition - ICDAR 2023. ICDAR 2023. Lecture Notes in Computer Science, vol 14189. Springer, Cham. https://doi.org/10.1007/978-3-031-41682-8_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-41682-8_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-41681-1
Online ISBN: 978-3-031-41682-8
eBook Packages: Computer ScienceComputer Science (R0)
