Skip to main content
Springer Nature Link
Log in

Aligning requirements and testing through metamodeling and patterns: design and evaluation

  • Original Article
  • Published:
Requirements Engineering Aims and scope Submit manuscript

Abstract

Poorly executed requirements engineering activities profoundly affect the deliverables’ quality and project’s budget and schedule. High-quality requirements reuse through requirement patterns has been widely discussed to mitigate these adverse outcomes. Requirement patterns aggregate similar applications’ behaviors and services into well-defined templates that can be reused in later specifications. The abstraction capabilities of metamodeling have shown promising results concerning the improvement of the requirement specifications’ quality and professionals’ productivity. However, there is a lack of research on requirement patterns beyond requirements engineering, even using metamodels as the underlying structure. Besides, most companies often struggle with the cost, rework, and delay effects resulting from a weak alignment between requirements and testing. In this paper, we present a novel metamodeling approach, called  Software Pattern MetaModel (SoPaMM), which aligns requirements and testing through requirement patterns and test patterns. Influenced by well-established agile practices, SoPaMM describes functional requirement patterns and acceptance test patterns as user stories integrated with executable behaviors. Another novelty is the evaluation of SoPaMM’s quality properties against a metamodel quality evaluation framework. We detail the evaluation planning, discuss evaluation results, and present our study’s threats to validity. Our experience with the design and evaluation of SoPaMM is summarized as lessons learned.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. Terminal models are metamodel instances as defined in the Meta-Object Facility (MOF) architecture [33].

  2. ID consists of an abbreviated alphabetic code with the initial letter in uppercase of the quality characteristic followed by two letters representing the sub-characteristic and an ordinal number of the sequential order within a quality sub-characteristic. For instance, the UAp-2 represents the second measure of appropriateness recognizability (Ap), which is sub-characteristic of Usability (U).

  3. Ecore is the core metamodel of the Eclipse Modeling Framework and describes models and runtime support for them. Available at https://wiki.eclipse.org/Ecore.

References

  1. Amorndettawin M, Senivongse T (2019) Non-functional requirement patterns for agile software development. In: Proceedings of the 2019 3rd International Conference on Software and E-Business, ICSEB 2019, pp 66–74. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3374549.3374561

  2. Taciana N. Kudo, Renato F. Bulcão-Neto, Auri M.R. Vincenzi (2019) A Conceptual Metamodel to Bridging Requirement Patterns to Test Patterns. In: Proceedings of the XXXIII Brazilian Symposium on Software Engineering, SBES 2019, pp. 155–160. ACM, Salvador, Brazil. https://doi.org/10.1145/3350768.3351300

  3. Taciana N. Kudo, Renato F. Bulcão-Neto, Auri M.R. Vincenzi (2020) Toward a Metamodel Quality Evaluation Framework: Requirements, Model, Measures, and Process. In: Proceedings of the XXXIV Brazilian Symposium on Software Engineering, SBES 2020, pp. 102–107. ACM, Natal, Brazil. https://doi.org/10.1145/3422392.3422461

  4. Taciana N. Kudo, Renato F. Bulcão-Neto, Auri M.R. Vincenzi (2020) Uma Ferramenta para Construção de Catálogos de Padrões de Requisitos com Comportamento. In: Workshop em Engenharia de Requisitos, WER 2020, pp. 1–14. Editora PUC-Rio, São José dos Campos, Brazil. http://wer.inf.pucrio.br/WERpapers/artigos/artigos_WER20/12_WER_2020_paper_16.pdf

  5. Taciana N. Kudo, Renato F. Bulcão-Neto, Auri M.R. Vincenzi (2020) Metamodel Quality Requirements and Evaluation (MQuaRE). CoRR abs/2008.09459. arxiv.org/abs/2008.09459

  6. Taciana N. Kudo, Renato F. Bulcão-Neto, Auri M.R. Vincenzi (2020) Requirement patterns: a tertiary study and a research agenda. IET Softw 14(1):18–26. https://doi.org/10.1049/iet-sen.2019.0016

  7. Taciana N. Kudo, Renato F. Bulcão-Neto, Auri M.R. Vincenzi, Alessandra A. Macedo (2019) A revisited systematic literature mapping on the support of requirement patterns for the software development life cycle. J. Softw. Eng. Res. Dev. 7:9. https://doi.org/10.5753/jserd.2019.458

  8. Baudry B, Nebut C, Traon YL (2007) Model-driven engineering for requirements analysis. In: 11th IEEE international enterprise distributed object computing conference (EDOC 2007), p 459

  9. Beckers K, Côté I, Goeke L (2014) A catalog of security requirements patterns for the domain of cloud computing systems. In: Proceedings of the ACM symposium on applied computing, pp 337–342. ACM, Gyeongju, Republic of Korea

  10. Bjarnason E, Borg M (2017) Aligning requirements and testing: working together toward the same goal. IEEE Softw 34(1):20–23. https://doi.org/10.1109/MS.2017.14

    Article  Google Scholar 

  11. Chelimsky D, Astels D, Helmkamp B, North D, Dennis Z, Hellesoy A (2010) The RSpec book: behaviour driven development with Rspec, cucumber, and friends, 1st edn. Pragmatic Bookshelf, Raleigh, NC

    Google Scholar 

  12. Cheng BHC, Atlee JM (2009) Current and future research directions in requirements engineering. In: Lyytinen K, Loucopoulos P, Mylopoulos J, Robinson B (eds) Design requirements engineering: a ten-year perspective. Springer, Berlin, pp 11–43

    Chapter  Google Scholar 

  13. Chernak Y (2012) Requirements reuse: the state of the practice. In: 2012 IEEE international conference on software science. Technology and Engineering, SWSTE 2012, Herzlia, Israel, June 12–13, 2012. IEEE Computer Society, Los Alamitos, CA, USA, pp 46–53

  14. Costal D, Franch X, López L, Palomares C, Quer C (2019) On the use of requirement patterns to analyse request for proposal documents. In: Laender AHF, Pernici B, Lim E, de Oliveira JPM (eds) conceptual modeling—38th international conference, ER 2019, Salvador, Brazil, November 4–7, 2019, Proceedings. Lecture Notes in Computer Science, vol. 11788, pp. 549–557. Springer . https://doi.org/10.1007/978-3-030-33223-5_45

  15. Ebert C, Ray R (2021) Test-driven requirements engineering. IEEE Softw 38(1):16–24. https://doi.org/10.1109/MS.2020.3029811

    Article  Google Scholar 

  16. Franch X (2015) Software requirements patterns: a state of the art and the practice. In: Proceedings of the 37th international conference on software engineering, vol 2, ICSE’15, pp 943–944. IEEE Press, Piscataway

  17. Franch X, Palomares C, Quer C (2020) Industrial practices on requirements reuse: an interview-based study. In: Madhavji NH, Pasquale L, Ferrari A, Gnesi S (eds) Requirements engineering: foundation for software quality—26th international working conference, REFSQ 2020, Pisa, Italy, March 24–27, 2020, Proceedings [REFSQ 2020 was postponed], Lecture Notes in Computer Science, vol 12045, pp 78–94. Springer. https://doi.org/10.1007/978-3-030-44429-7_6

  18. Franch X, Palomares C, Quer C, Renault S, De Lazzer F (2010) A metamodel for software requirement patterns. In: Wieringa R, Persson A (eds) Requirements engineering: foundation for software quality. Springer, Berlin, pp 85–90

  19. Fricker S, Grau R, Zwingli A (2015) Requirements engineering: best practice, pp 25–46. Springer, Berlin

  20. Gamma E, Helm R, Johnson R, Vlissides J (1995) Design patterns: elements of reusable object-oriented software. Addison-Wesley Longman, Boston

    MATH  Google Scholar 

  21. Haskins C (2003) Using patterns to share best results—a proposal to codify the Sebok. INCOSE Int Symp 13(1):15–23

    Article  Google Scholar 

  22. Irshad M, Petersen K, Poulding S (2018) A systematic literature review of software requirements reuse approaches. Inf Softw Technol 93(C):223–245

  23. ISO/IEC: ISO/IEC 9126-1:2001 Software engineering—product quality—Part 1: quality model. ISO/IEC 9126-1:2001 1:1–25 (2001)

  24. ISO/IEC: ISO/IEC 25000:2014 Systems and software engineering—systems and software Quality Requirements and Evaluation (SQuaRE)—Guide to SQuaRE. ISO/IEC 25000:2014 2:1–27 (2014)

  25. ISO/IEC: ISO/IEC 25023:2016 systems and software engineering—systems and software quality requirements and evaluation (SQuaRE)—measurement of system and software product quality. ISO/IEC 25023:2016 1:1–45 (2016)

  26. Leotta M, Clerissi D, Ricca F, Spadaro C (2013) Improving test suites maintainability with the page object pattern: an industrial case study. In: ICST workshops, pp 108–113. IEEE Computer Society, Washington, DC

  27. Ma H, Shao W, Zhang L, Ma Z, Jiang Y (2004) Applying OO metrics to assess UML meta-models. In: Baar T, Strohmeier A, Moreira A, Mellor SJ (eds) UML 2004—the unified modeling language. Modeling languages and applications, pp 12–26. Springer, Berlin

  28. Ma Z, He X, Liu C (2013) Assessing the quality of metamodels. Front Comput Sci 7(4):558

    Article  MathSciNet  Google Scholar 

  29. Macasaet RJ, Noguera M, Rodríguez ML, Garrido JL, Supakkul S, Chung L (2019) Micro-business requirements patterns in practice: remote communities in developing nations. J Univ Comput Sci 25(7):764–787. http://www.jucs.org/jucs_25_7/micro_business_requirements_patterns

  30. Meszaros G (2006) XUnit test patterns: refactoring test code. Prentice Hall, Upper Saddle River

    Google Scholar 

  31. Moreira RMLM, Paiva ACR (2014) A GUI modeling DSL for pattern-based GUI testing—PARADIGM. In: ENASE 2014—Proceedings of the 9th international conference on evaluation of novel approaches to software engineering, Lisbon, Portugal, 28–30 April, 2014, pp 126–135. IEEE, Lisbon, Portugal

  32. Oliveira G, Marczak S, Moralles C (2019) How to evaluate BDD scenarios’ quality? In: do Carmo Machado I, Souza R, Maciel RSP, Sant’Anna C (eds) Proceedings of the XXXIII Brazilian symposium on software engineering, SBES 2019, Salvador, Brazil, September 23–27, pp 481–490. ACM. https://doi.org/10.1145/3350768.3351301

  33. OMG: Meta object facility (mof) specification, version 1.4. Object Management Group, Inc. (2002)

  34. OMG: Structured patterns metamodel standard. OMG—Object Management Group (2017)

  35. Palomares C, Quer C, Franch X (2011) Pabre-man: management of a requirement patterns catalogue. In: RE 2011, 19th IEEE international requirements engineering conference, Trento, Italy, August 29 2011–September 2, 2011, pp 341–342. IEEE Computer Society. https://doi.org/10.1109/RE.2011.6051666

  36. Palomares C, Quer C, Franch X, Renault S, Guerlain C (2013) A catalogue of functional software requirement patterns for the domain of content management systems. In: Proceedings of the 28th annual ACM symposium on applied computing, SAC ’13, pp 1260–1265. ACM, New York

  37. Rising L (1999) Patterns: a way to reuse expertise. IEEE Commun Mag 37(4):34–36

    Article  Google Scholar 

  38. Rocco J, Di Ruscio D, Iovino L, Pierantonio A (2014) Mining metrics for understanding metamodel characteristics. In: Proceedings of the 6th international workshop on modeling in software engineering (MiSE 2014), pp 55–60. ACM, New York

  39. Rook P (1986) Controlling software projects. Softw Eng J 1:7

    Article  Google Scholar 

  40. Smart JF (2014) BDD in action: behavior-driven development for the whole software lifecycle, 1st edn. Manning Publications

  41. de Souza Cunha H, do Prado Leite JCS, Duboc L, Werneck V (2013) The challenges of representing transparency as patterns. In: Third IEEE International Workshop on Requirements Patterns, RePa 2013, Rio de Janeiro, Brazil, July 16, 2013, pp 25–30. IEEE Computer Society . https://doi.org/10.1109/RePa.2013.6602668

  42. Stocco A, Leotta M, Ricca F, Tonella P (2015) Why creating web page objects manually if it can be done automatically? In: Proceedings of the 10th international workshop on automation of software test, AST ’15, pp 70–74. IEEE Press, Piscataway

  43. Strahonja V (2007) The evaluation criteria of workflow metamodels. In: 29th international conference on information technology interfaces, pp 553–558. IEEE, New York

  44. Tockey S (2015) Insanity, hiring, and the software industry. Computer 48:96–101

    Article  Google Scholar 

  45. Videira C, da Silva AR (2005) Patterns and metamodel for a natural-language-based requirements specification language. In: Belo O, Eder J, Cunha JF, Pastor O (eds) The 17th conference on advanced information systems engineering (CAiSE ’05), Porto, Portugal, 13–17 June, 2005, CAiSE Forum, Short Paper Proceedings, CEUR Workshop Proceedings, vol 161. CEUR-WS.org. http://ceur-ws.org/Vol-161/FORUM_31.pdf

  46. Withall S (2007) Software requirement patterns. Best practices. Microsoft Press, Redmond

    Google Scholar 

  47. Wohlin C, Runeson P, Hst M, Ohlsson MC, Regnell B, Wessln A (2012) Experimentation in software engineering. Springer, Berlin

    Book  Google Scholar 

  48. Xuan X, Wang Y, Li S (2014) Privacy requirements patterns for mobile operating systems. In: Zhao L, do Prado Leite JCS, Supakkul S, Chung L, Wang Y (eds) 4th IEEE international workshop on requirements patterns, RePa 2014, Karlskrona, Sweden, August 26, 2014, pp. 39–42. IEEE Computer Society. https://doi.org/10.1109/RePa.2014.6894842

  49. Ya’u B, Nordin A, Salleh N (2016) Software requirements patterns and meta model: a strategy for enhancing requirements reuse (rr). In: 2016 6th international conference on information and communication technology for the muslim world, pp 188–193. ICT4M, Jakarta, Indonesia

Download references

Acknowledgements

The work described in this manuscript has not been published before. It is not also under consideration for publication anywhere else. Besides, its publication has been approved by all co-authors.

Funding

Partial financial support was received from the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior−Brazil (CAPES)−Finance Code 001.

Author information

Authors and Affiliations

  1. Departamento de Computação, Universidade Federal de São Carlos (UFSCar), SP, São Carlos, Brazil

    Taciana Novo Kudo & Auri Marcelo Rizzo Vincenzi

  2. Instituto de Informática, Universidade Federal de Goiás (UFG), GO, Goiânia, Brazil

    Taciana Novo Kudo, Renato de Freitas Bulcão-Neto & Valdemar Vicente Graciano Neto

Authors
  1. Taciana Novo Kudo
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Renato de Freitas Bulcão-Neto
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Valdemar Vicente Graciano Neto
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Auri Marcelo Rizzo Vincenzi
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Taciana Novo Kudo, Renato de Freitas Bulcão-Neto were involved in conceptualization and writing—original draft preparation. Taciana Novo Kudo, Renato de Freitas Bulcão-Neto, Valdemar Vicente Graciano Neto contributed to methodology. All authors were involved in results analysis and threats to validity. Renato de Freitas Bulcão-Neto, Auri Marcelo Rizzo Vincenzi contributed to writing—review and editing. Taciana Novo Kudo was involved in funding acquisition. Auri Marcelo Rizzo Vincenzi contributed to supervision.

Corresponding author

Correspondence to Taciana Novo Kudo.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kudo, T.N., Bulcão-Neto, R.d.F., Neto, V.V.G. et al. Aligning requirements and testing through metamodeling and patterns: design and evaluation. Requirements Eng 28, 97–115 (2023). https://doi.org/10.1007/s00766-022-00377-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00766-022-00377-5

Keywords