Abstract
Usability of a consumer electronic product (CEP) is one of the most important factors that the users consider in purchasing a CEP as well as functionality, price, etc. This has led many companies to realize new shapes of user interfaces (UIs) and styles of interaction for CEPs, ranging from modern touchscreens to physical controls and displays of any kind. Even if the general feeling is that such increased interactivity may enhance the overall user experience, the side effect is that often a CEP’s UI provides too many functions that are difficult to learn and use without referring to the user manual, leading to many usability issues. In this paper, we leverage a case study in the CEPs sector to present a novel log-based evaluation technique in the field of Human-Computer Interaction (HCI). Our technique allows us not only to keep track of the user interactions with a CEP’s UI during its daily use, but also to understand what has gone wrong during a user interaction, detecting which user actions have caused usability issues and suggesting explanations for solving them, thus providing a crucial feedback to improve the design of the CEP’s UI next version.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The case study is just an exploration of some possible design issues of a real oven’s UI. Since the authors are not affiliated in any way with the CEP company that manufactured the oven, the company name and the oven’s model are not disclosed.
- 2.
The complete list of cooking functions provided by the microwave oven can be found in the user manual associated to the oven.
- 3.
Users participating to remote tests are often rewarded with discounts, coupons, etc.
- 4.
- 5.
References
Alan, D., Janet, F., Gregory, A., Russell, B.: Human-Computer Interaction. Pearson, London (2004)
Augusto, A., et al.: Automated discovery of process models from event logs: review and benchmark. IEEE Trans. Knowl. Data Eng. 31(4), 686–705 (2018)
Bastide, R., Palanque, P., Sy, O., Le, D.-H., Navarre, D.: Petri net based behavioural specification of CORBA systems. In: Donatelli, S., Kleijn, J. (eds.) ICATPN 1999. LNCS, vol. 1639, pp. 66–85. Springer, Heidelberg (1999). https://doi.org/10.1007/3-540-48745-X_5
Benyon, D.: Designing Interactive Systems: A Comprehensive Guide to HCI, UX and Interaction Design. Pearson, London (2014)
Campos, J.C., Sousa, M., Alves, M.C.B., Harrison, M.D.: Formal verification of a space system’s user interface with the IVY workbench. IEEE Trans. Hum.-Mach. Syst. 46(2), 303–316 (2016)
Carmona, J., et al.: Conformance Checking. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99414-7
Dignum, M.: A model for organizational interaction: based on agents, Founded in Logic. Ph.D. thesis, Utrecht University (2004)
Dumais, S., Jeffries, R., Russell, D.M., Tang, D., Teevan, J.: Understanding user behavior through log data and analysis. In: Olson, J.S., Kellogg, W.A. (eds.) Ways of Knowing in HCI, pp. 349–372. Springer, New York (2014). https://doi.org/10.1007/978-1-4939-0378-8_14
Feary, M.S.: A toolset for supporting iterative human automation: interaction in design. NASA Ames Research Center. Tech. rep. 20100012861 (2010)
Ferre, X., Juristo, N., Moreno, A.M.: Improving software engineering practice with HCI aspects. In: Ramamoorthy, C.V., Lee, R., Lee, K.W. (eds.) SERA 2003. LNCS, vol. 3026, pp. 349–363. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24675-6_27
Ferre, X., Villalba, E., Julio, H., Zhu, H.: Extending mobile app analytics for usability test logging. In: Bernhaupt, R., Dalvi, G., Joshi, A., K. Balkrishan, D., O’Neill, J., Winckler, M. (eds.) INTERACT 2017, Part III. LNCS, vol. 10515, pp. 114–131. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67687-6_9
Fu, B., Lin, J., Li, L., Faloutsos, C., Hong, J., Sadeh, N.: Why people hate your app: making sense of user feedback in a mobile app store. In: 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1276–1284 (2013)
Han, S.H., Yun, M.H., Kwahk, J., Hong, S.W.: Usability of consumer electronic products. Int. J. Ind. Ergon. 28(3–4), 143–151 (2001)
Harel, D.: Statecharts: a visual formalism for complex systems. Sci. Comput. Program. 8(3), 231–274 (1987)
Hartson, H.R., Andre, T.S., Williges, R.C.: Criteria for evaluating usability evaluation methods. Int. J. Hum.-Comput. Interact. 13(4), 373–410 (2001)
Humayoun, S.R., et al.: The WORKPAD user interface and methodology: developing smart and effective mobile applications for emergency operators. In: Stephanidis, C. (ed.) UAHCI 2009, Part III. LNCS, vol. 5616, pp. 343–352. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-02713-0_36
Jagadeesh Chandra Bose, R.P., van der Aalst, W.: Trace alignment in process mining: opportunities for process diagnostics. In: Hull, R., Mendling, J., Tai, S. (eds.) Business Process Management. BPM 2010. LNCS, vol. 6336, pp. 227–242. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15618-2_17
Janlert, L.E., Stolterman, E.: Things that Keep us Busy: The Elements of Interaction. MIT Press, Cambridge (2017)
John, B.E., Kieras, D.E.: The GOMS family of user interface analysis techniques: comparison and contrast. ACM Trans. Comput.-Hum. Interact. 3(4), 320–351 (1996)
Joung, J., Jung, K., Ko, S., Kim, K.: Customer complaints analysis using text mining and outcome-driven innovation method for market-oriented product development. Sustainability 11(1), 1–14 (2019)
Kao, C.K., Liebovitz, D.M.: Consumer mobile health apps: current state, barriers, and future directions. J. Am. Acad. Phys. Med. Rehabil. 9(5), 106–115 (2017)
Kim, J., Han, S.H.: A methodology for developing a usability index of consumer electronic products. Int. J. Ind. Ergon. 38(3–4), 333–345 (2008)
Kjeldskov, J., Skov, M.B.: Was it worth the hassle?: ten years of mobile HCI research discussions on lab and field evaluations. In: 16th International Conference on Human-Computer Interaction with Mobile Devices & Services (MobileHCI 2014), pp. 43–52. ACM (2014)
van Kuijk, J., Christiaans, H., Kanis, H., van Eijk, D.: Usability in the development of consumer electronics: issues and actors. In: 16th World Congress on Ergonomics (IEA 2006) (2006)
van Kuijk, J., Kanis, H., Christiaans, H., van Eijk, D.: Barriers to and enablers of usability in electronic consumer product development: a multiple case study. Hum.-Comput. Interact. 32(1), 1–71 (2017)
Kwahk, J., Han, S.H.: A methodology for evaluating the usability of audiovisual consumer electronic products. Appl. Ergon. 33(5), 419–431 (2002)
Lau, T., Horvitz, E.: Patterns of search: analyzing and modeling web query refinement. In: Kay, J. (ed.) UM99 User Modeling. CICMS, vol. 407, pp. 119–128. Springer, Vienna (1999). https://doi.org/10.1007/978-3-7091-2490-1_12
de Leoni, M., Marrella, A.: Aligning real process executions and prescriptive process models through automated planning. Expert Syst. Appl. 82, 162–183 (2017)
Lettner, F., Holzmann, C.: Automated and unsupervised user interaction logging as basis for usability evaluation of mobile applications. In: 10th International Conference on Advances in Mobile Computing & Multimedia (MoMM 2012), pp. 118–127. ACM (2012)
Marrella, A., Catarci, T.: Measuring the learnability of interactive systems using a petri net based approach. In: Designing Interactive Systems Conference, DIS 2018, pp. 1309–1319 (2018)
Marrella, A., Ferro, L.S., Catarci, T.: An approach to identifying what has gone wrong in a user interaction. In: Lamas, D., Loizides, F., Nacke, L., Petrie, H., Winckler, M., Zaphiris, P. (eds.) INTERACT 2019, Part III. LNCS, vol. 11748, pp. 361–370. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-29387-1_20
Marrella, A., Mecella, M., Russo, A.: Collaboration on-the-field: suggestions and beyond. In: 8th International Conference on Information Systems for Crisis Response and Management (2011)
Marrella, A., Mecella, M., Sardiña, S.: Supporting adaptiveness of cyber-physical processes through action-based formalisms. AI Commun. 31(1), 47–74 (2018)
Mori, G., Paternò, F., Santoro, C.: CTTE: support for developing and analyzing task models for interactive system design. IEEE Trans. Softw. Eng. 28(8), 797–813 (2002)
Murata, T.: Petri nets: properties, analysis and applications. Proc. IEEE 77(4), 541–580 (1989)
Nielsen, J.: Finding usability problems through heuristic evaluation. In: SIGCHI Conference on Human Factors in Computing Systems (CHI 1992), pp. 373–380 (1992)
Nielsen, J.: 10 Usability Heuristics for User Interface Design. Nielsen Norman Group (1994). https://www.nngroup.com/articles/ten-usability-heuristics/
Okada, H., Asahi, T.: Guitester: a log-based usability testing tool for graphical user interfaces. IEICE Trans. Inf. Syst. 82(6), 1030–1041 (1999)
Page, T.: Skeuomorphism or flat design: future directions in mobile device User Interface (UI) design education. Int. J. Mob. Learn. Organ. 8(2), 130–142 (2014)
Palanque, P.A., Bastide, R.: Petri net based design of user-driven interfaces using the interactive cooperative objects formalism. In: Paternó, F. (ed.) Interactive Systems: Design, Specification, and Verification. Focus on Computer Graphics (Tutorials and Perspectives in Computer Graphics). Springer, Heidelberg (1995). https://doi.org/10.1007/978-3-642-87115-3_23
Paterno, F.: Model-Based Design and Evaluation of Interactive Applications, 1st edn. Springer, Heidelberg (1999). https://doi.org/10.1007/978-1-4471-0445-2
Pnueli, A.: The temporal logic of programs. In: 18th Annual IEEE Symposium on Foundations of Computer Science. IEEE (1977)
Supply & Demand Chain Executive: What Are Late Product Launches Really Costing You? (2017). https://www.sdcexec.com/sourcing-procurement/article/20985884/
Sutcliffe, A.G., Wang, I.: Integrating human computer interaction with Jackson system development. Comput. J. 34(2), 132–142 (1991)
Thapliyal, H.: Internet of things-based consumer electronics: reviewing existing consumer electronic devices, systems, and platforms and exploring new research paradigms. IEEE Consum. Electron. Mag. 7(1), 66–67 (2017)
Tyler, S.K., Teevan, J.: Large scale query log analysis of re-finding. In: Third ACM International Conference on Web Search and Data Mining, pp. 191–200 (2010)
Valle, T., Prata, W., et al.: Automated usability tests for mobile devices through live emotions logging. In: 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI 2015), pp. 636–643. ACM (2015)
Van Den Bos, J., Plasmeijer, M.J., Hartel, P.H.: Input-output tools: a language facility for interactive and real-time systems. IEEE Trans. Softw. Eng. 9(3), 247–259 (1983)
Wasserman, A.I.: Extending state transition diagrams for the specification of human-computer interaction. IEEE Trans. Softw. Eng. 8, 699–713 (1985)
Zhang, X., Qiao, Z., Tang, L., Fan, W., Fox, E., Wang, G.: Identifying product defects from user complaints: a probabilistic defect model. In: Decision Support and Analytics (SIGDSA 2016) (2016)
Acknowledgments
This work has been supported by the “Dipartimento di Eccellenza” grant, the H2020 project DataCloud and the Sapienza grant BPbots.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 IFIP International Federation for Information Processing
About this paper
Cite this paper
Benvenuti, D., Buda, E., Fraioli, F., Marrella, A., Catarci, T. (2021). Detecting and Explaining Usability Issues of Consumer Electronic Products. In: Ardito, C., et al. Human-Computer Interaction – INTERACT 2021. INTERACT 2021. Lecture Notes in Computer Science(), vol 12935. Springer, Cham. https://doi.org/10.1007/978-3-030-85610-6_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-85610-6_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-85609-0
Online ISBN: 978-3-030-85610-6
eBook Packages: Computer ScienceComputer Science (R0)
