Skip to main content
Springer Nature Link
Log in

eMOP: A Maven Plugin for Evolution-Aware Runtime Verification

  • Conference paper
  • First Online:
Runtime Verification (RV 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14245))

Included in the following conference series:

Abstract

We present eMOP, a tool for incremental runtime verification (RV) of test executions during software evolution. We previously used RV to find hundreds of bugs in open-source projects by monitoring passing tests against formal specifications of Java APIs. We also proposed evolution-aware techniques to reduce RV ’s runtime overhead and human time to inspect specification violations. eMOP brings these benefits to developers in a tool that seamlessly integrates with the Maven build system. We describe eMOP ’s design, implementation, and usage. We evaluate eMOP on 676 versions of 21 projects, including those from our earlier prototypes’ evaluation. eMOP is up to 8.4\(\times \) faster and shows up to 31.3\(\times \) fewer violations, compared to running RV from scratch after each code change. eMOP also does not miss new violations in our evaluation, and it is open-sourced at https://github.com/SoftEngResearch/emop.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. ajc. https://www.eclipse.org/aspectj/doc/next/devguide/ajc-ref.html

  2. Arcuri, A., Campos, J., Fraser, J.: Unit test generation during software development: Evosuite plugins for Maven, IntelliJ, and Jenkins. In: ICST, pp. 401–408 (2016)

    Google Scholar 

  3. Bartocci, E., Falcone, Y., Francalanza, A., Reger, G.: Introduction to runtime verification. In: Bartocci, E., Falcone, Y. (eds.) Lectures on Runtime Verification. LNCS, vol. 10457, pp. 1–33. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75632-5_1

    Chapter  Google Scholar 

  4. Basin, D., Harvan, M., Klaedtke, F., Zălinescu, E.: MONPOLY: monitoring usage-control policies. In: Khurshid, S., Sen, K. (eds.) RV 2011. LNCS, vol. 7186, pp. 360–364. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29860-8_27

    Chapter  Google Scholar 

  5. Bell, J., Legunsen, O., Hilton, M., Eloussi, L., Yung, T., Marinov, D.: DeFlaker: automatically detecting flaky tests. In: ICSE, pp. 433–444 (2018)

    Google Scholar 

  6. Biswas, S., Mall, R., Satpathy, M., Sukumaran, S.: Regression test selection techniques: a survey. Informatica 35(3), 289–321 (2011)

    Google Scholar 

  7. Collections_SynchronizedCollection Specification. https://github.com/owolabileg/property-db/blob/master/annotated-java-api/java/util/Collections_SynchronizedCollection.mop

  8. SuiteHTMLReporter does not synchronize iteration on a synchronized list. https://github.com/testng-team/testng/pull/931

  9. JUnitXMLReporter does not synchronize the two synchronized collections when iterating. https://github.com/testng-team/testng/pull/830

  10. Ellul, J., Pace, G.J.: Runtime verification of ethereum smart contracts. In: EDCC, pp. 158–163 (2018)

    Google Scholar 

  11. eMOP Artifacts. https://github.com/SoftEngResearch/emop-artifacts

  12. eMOP GitHub Page. https://github.com/SoftEngResearch/emop

  13. Falcone, Y., Havelund, K., Reger, G.: A tutorial on runtime verification. In: EDSS, pp. 141–175 (2013)

    Google Scholar 

  14. Falcone, Y., Krstić, S., Reger, G., Traytel, D.: A taxonomy for classifying runtime verification tools. In: Colombo, C., Leucker, M. (eds.) RV 2018. LNCS, vol. 11237, pp. 241–262. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03769-7_14

    Chapter  Google Scholar 

  15. Fraser, G., Arcuri, A.: Evosuite: automatic test suite generation for object-oriented software. In: FSE, pp. 416–419 (2011)

    Google Scholar 

  16. Fraser, G., Arcuri, A.: A large-scale evaluation of automated unit test generation using Evosuite. TOSEM 24(2), 1–42 (2014)

    Article  Google Scholar 

  17. Gligoric, M., Eloussi, L., Marinov, D.: Ekstazi: lightweight test selection. In: ICSE Demo, pp. 713–716 (2015)

    Google Scholar 

  18. Gligoric, M., Eloussi, L., Marinov, D.: Practical regression test selection with dynamic file dependencies. In: ISSTA, pp. 211–222 (2015)

    Google Scholar 

  19. Graves, T.L., Harrold, M.J., Kim, J.-M., Porter, A., Rothermel, G.: An empirical study of regression test selection techniques. TOSEM 10(2), 184–208 (2001)

    Article  MATH  Google Scholar 

  20. Gyori, A., Lahiri, S.K., Partush, N.: Refining interprocedural change-impact analysis using equivalence relations. In: ISSTA, pp. 318–328 (2017)

    Google Scholar 

  21. Gyori, A., Lambeth, B., Shi, A., Legunsen, O., Marinov, D.: NonDex: a tool for detecting and debugging wrong assumptions on Java API specifications. In: FSE Demo, pp. 993–997 (2016)

    Google Scholar 

  22. Hallé, S., Khoury, R.: Event stream processing with BeepBeep 3. In: RV-CuBES, pp. 81–88 (2017)

    Google Scholar 

  23. Havelund, K.: Rule-based runtime verification revisited. STTT 17, 143–170 (2015)

    Article  Google Scholar 

  24. Havelund, K., Peled, D.: Efficient runtime verification of first-order temporal properties. In: Gallardo, M.M., Merino, P. (eds.) SPIN 2018. LNCS, vol. 10869, pp. 26–47. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-94111-0_2

    Chapter  Google Scholar 

  25. Havelund, K., Peled, D., Ulus, D.: First-order temporal logic monitoring with BDDs. FMSD 56(1–3), 1–21 (2020)

    MATH  Google Scholar 

  26. Havelund, K., Roşu, G.: Monitoring programs using rewriting. In: ASE, pp. 135–143 (2001)

    Google Scholar 

  27. java.util.Collections. https://docs.oracle.com/javase/8/docs/api/java/util/Collections.html

  28. JGit. http://www.eclipse.org/jgit

  29. Jin, D., Meredith, P.O., Lee, C., Roşu, G.: JavaMOP: efficient parametric runtime monitoring framework. In: ICSE Demo, pp. 1427–1430 (2012)

    Google Scholar 

  30. Kim, M., Kannan, S., Lee, I., Sokolsky, O., Viswanathan, M.: Java-MaC: a run-time assurance tool for Java programs. In: RV, pp. 218–235 (2001)

    Google Scholar 

  31. Lee, C., Jin, D., Meredith, P.O., Roşu, G.: Towards categorizing and formalizing the JDK API. Computer Science Dept., UIUC, Technical report (2012)

    Google Scholar 

  32. Legunsen, O., Al Awar, N., Xu, X., Hassan, W.U., Roşu, G., Marinov, D.: How effective are existing Java API specifications for finding bugs during runtime verification? ASEJ 26(4), 795–837 (2019)

    Google Scholar 

  33. Legunsen, O., Hariri, F., Shi, A., Lu, Y., Zhang, L., Marinov, D.: An extensive study of static regression test selection in modern software evolution. In: FSE, pp. 583–594 (2016)

    Google Scholar 

  34. Legunsen, O., Hassan, W.U., Xu, X., Roşu, G., Marinov, D.: How good are the specs? A study of the bug-finding effectiveness of existing Java API specifications. In: ASE, pp. 602–613 (2016)

    Google Scholar 

  35. Legunsen, O., Marinov, D., Rosu, G.: Evolution-aware monitoring-oriented programming. In: ICSE NIER, pp. 615–618 (2015)

    Google Scholar 

  36. Legunsen, O., Shi, A., Marinov, D.: STARTS: STAtic regression test selection. In: ASE Demo, pp. 949–954 (2017)

    Google Scholar 

  37. Legunsen, O., Zhang, Y., Hadzi-Tanovic, M., Rosu, G., Marinov, D.: Techniques for evolution-aware runtime verification. In: ICST, pp. 300–311 (2019)

    Google Scholar 

  38. Leucker, M., Schallhart, C.: A brief account of runtime verification. In: Formal Languages and Analysis of Contract-Oriented Software, pp. 293–303 (2007)

    Google Scholar 

  39. Liu, Y., Zhang, J., Nie, P., Gligoric, M., Legunsen, O.: More precise regression test selection via reasoning about semantics-modifying changes. In: ISSTA, pp. 664–676 (2023)

    Google Scholar 

  40. Miranda, B., Lima, I., Legunsen, O., d’Amorim, M.: Prioritizing runtime verification violations. In: ICST, pp. 297–308 (2020)

    Google Scholar 

  41. O’Hearn, P.W.: Continuous reasoning: scaling the impact of formal methods. In: LICS, pp. 13–25 (2018)

    Google Scholar 

  42. Palomba, F., Zaidman, A.: Does refactoring of test smells induce fixing flaky tests? In: ICSME, pp. 1–12 (2017)

    Google Scholar 

  43. Reger, G., Cruz, H.C., Rydeheard, D.: MarQ: monitoring at runtime with QEA. In: Baier, C., Tinelli, C. (eds.) TACAS 2015. LNCS, vol. 9035, pp. 596–610. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-662-46681-0_55

    Chapter  Google Scholar 

  44. Schneider, F.B.: Enforceable security policies. TISSEC 3(1), 30–50 (2000)

    Article  Google Scholar 

  45. Shi, A., Gyori, A., Legunsen, O., Marinov, D.: Detecting assumptions on deterministic implementations of non-deterministic specifications. In: ICST, pp. 80–90 (2016)

    Google Scholar 

  46. Shi, A., Hadzi-Tanovic, M., Zhang, L., Marinov, D., Legunsen, O.: Reflection-aware static regression test selection. PACML, 3(OOPSLA), 1–29 (2019)

    Google Scholar 

  47. Shi, A., Lam, W., Oei, R., Xie, T., Marinov, D.: iFixFlakies: a framework for automatically fixing order-dependent flaky tests. In: FSE, pp. 545–555 (2019)

    Google Scholar 

  48. About surefire. https://maven.apache.org/surefire

Download references

Acknowledgements

We thank the anonymous reviewers for their comments on an earlier draft of this paper. This work was partially supported by funds from the Google Cyber NYC Institutional Research Program and the US National Science Foundation under Grant Nos. 2019277 and 2045596.

Author information

Authors and Affiliations

  1. Cornell University, Ithaca, USA

    Ayaka Yorihiro, Pengyue Jiang, Valeria Marqués, Benjamin Carleton & Owolabi Legunsen

Authors
  1. Ayaka Yorihiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pengyue Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Valeria Marqués
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Benjamin Carleton
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Owolabi Legunsen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pengyue Jiang .

Editor information

Editors and Affiliations

  1. Dept. of Informatics, Aristotle Univ. of Thessaloniki, Thessaloniki, Greece

    Panagiotis Katsaros

  2. University of Trieste, Trieste, Italy

    Laura Nenzi

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yorihiro, A., Jiang, P., Marqués, V., Carleton, B., Legunsen, O. (2023). eMOP: A Maven Plugin for Evolution-Aware Runtime Verification. In: Katsaros, P., Nenzi, L. (eds) Runtime Verification. RV 2023. Lecture Notes in Computer Science, vol 14245. Springer, Cham. https://doi.org/10.1007/978-3-031-44267-4_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-44267-4_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-44266-7

  • Online ISBN: 978-3-031-44267-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics