Skip to main content
Springer Nature Link
Log in

Occlusion handling in outdoors augmented reality games

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The use of augmented reality (AR) becomes increasingly common in mobile game development as a means of enhancing the players’ view of the physical world through computer-generated graphical information. A situation often encountered in AR applications is the -partial or full- occlusion of virtual objects by physical artifacts; if not appropriately handled, the visualization of occluded objects often misleads users’ perception. This paper introduces three alternative Geolocative Raycasting techniques aiming at assisting developers of outdoors AR games in generating a realistic field of view (FoV) for the players by integrating real time building recognition, so as to address the occlusion problem. Our geolocative raycasting methods have been applied in the location-based, AR game Order Elimination, which utilizes publicly and freely available building information to calculate the players FoV in real-time. The proposed algorithms are applicable to a variety of sensor-based AR applications and portable to any real setting, provided that sufficient topographical data exist. The three FoV determination methods have been tested with respect to several performance parameters demonstrating that real-time FoV rendering is feasible by modest mobile devices, even under stress conditions. A user evaluation study revealed that the consideration of buildings for determining the FoV in AR pervasive games can increase the quality of experience of players when compared with standard FoV generation methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Notes

  1. The term ‘first-person shooter’ refers to a video game genre centered on gun and projectile weapon-based combat through a first-person perspective; that is, the player experiences the action through the eyes of the protagonist.

  2. The ManHunt Android application file as well as user evaluation statistics are available from: http://www.barbarossarpg.com/.

  3. A commercial version of Order Elimination is available in Google Play (https://play.google.com/store/apps/details?id=bl.on.mi.en)

  4. http://overpass-turbo.eu/

  5. The OSM Overpass API provides textual description of buildings in a certain area, essentially a list of polygons, each comprising a series of latitude/longitude points.

  6. http://www.beyondar.com/

  7. Bearing refers to the angle of a moving object’s direction from the North.

  8. http://developer.android.com/guide/topics/sensors/sensors_overview.html

  9. https://github.com/googlemaps/android-maps-utils

  10. As reported in the TimeWarp evaluation results, the virtual artifacts in AR games should be preferably hidden when the player has no LoS with them. Even though we could effortlessly hide the AR content when eye contact with the Zombie was infeasible, we chose not to for game design purposes.

  11. Vuzix M100 Smart Glasses (https://www.vuzix.com/Products/M100-Smart-Glasses) supports a FoV angle of 15°, while Epson Moverio BT-200 (http://global.epson.com/newsroom/2014/news_20140107.html) supports 23°. Both products fully support smartphone integration.

  12. The Android Augment Reality Framework (https://github.com/phishman3579/android-augment-reality-framework) and Mixare (http://www.mixare.org/) support a FoV angle of 45°.

  13. The intuition for choosing the 5° angle has been, firstly, to create a sufficient FoV polygon for the Zombie to be included in and, secondly, to allow sufficiently wide FoV (than that of a single ray) so as to compensate for GPS location fix inaccuracies, which do not allow precise positioning calculations as in computer games.

  14. http://developer.android.com/training/displaying-bitmaps/load-bitmap.html

  15. http://developer.android.com/guide/practices/screens_support.html

  16. Android applications may obtain the GPS accuracy, defined as the radius of 68 % confidence. That is, there is a 68 % probability that the true location is inside a circle centered at the estimated location, where the circle’s radius is equal to the accuracy value. For example, if the accuracy value is 10, then there’s a 68 % chance the true location of the device is within 10 m of the reported coordinates.

  17. The evaluators have been recruited through an open invitation advertised in the University of the Aegean, Mytilene, Greece. Ten (10) of the evaluators have been male and two (2) female. Six (6) evaluators have been in the age group of 19–23, four (4) in the age group of 24–33 and two (2) in the age group 34–39.

  18. The application distributed to evaluators implemented the RC algorithm. Since this method performs worse than RPI, it serves the purpose of letting evaluators to test the application under the worst possible performance conditions. Note that the IRC method has not been an option as we have chosen not to confine the evaluation within a pre-registered area or prevent users from zooming-in/out the map view.

  19. A video of the application can be found at https://www.youtube.com/watch?v=D--A3fEghbA. The Android Application File (APK) can be downloaded at http://zarcrash.x10.mx/OrderEliminationc.apk.

  20. The questionnaire can be found at http://zarcrash.x10.mx/OrderEliminationGoogleForm.pdf

  21. https://www.2kgames.com/bioshock/

  22. http://www.sniperelite3.com/

  23. http://eu.battle.net/wow/en/

  24. http://assassinscreed.ubi.com/en-gb/home/

  25. The application distributed to evaluators has been configured with the ray length set to 100 m, the FoV angle set to 28° and a game space of 250,000 m2 (square with side length of 500 m) while the mobile devices used by players varied in technical specifications (Sony Xperia S, Samsung (Galaxy S4 Mini, Beam, Note 4, S4), Nexus (4 & 6), F&U Tablet and Motorola Moto G 2nd Generation).

References

  1. Behzadan AH, Kamat VR (2010) Scalable algorithm for resolving incorrect occlusion in dynamic augmented reality engineering environments. Comput-Aided Civil Infrastruct Eng 25:3–19

    Article  Google Scholar 

  2. Benford S, Crabtree A, Flintham M, Drozd A, Anastasi R, Paxton M (2006) Can You See Me Now? ACM Trans Comput-Human Inter 13:100–133

    Article  Google Scholar 

  3. Benford S, Magerkurth C, Ljungstrand P (2005) Bridging the Physical and Digital in Pervasive Gaming. Commun ACM 48:54–57

    Article  Google Scholar 

  4. Billinghurst M, Clark A, Lee G (2014) A Survey of Augmented Reality. Foundations Trends Human–Comput Inter 8:73–272

    Article  Google Scholar 

  5. Broll W, Ohlenburg J, Lindt I, Herbst I, Braun AK (2006) Meeting technology challenges of pervasive augmented reality games, Proc 5th ACM SIGCOMM Workshop Network System Support Games, 28–39

  6. Capra M, Radenkovic M, Benford S, Oppermann L, Drozd A, Flintham M (2005) The multimedia challenges raised by pervasive games, Proc 13th Annual ACM Int Conf Mult, 89–95

  7. Cheok AD, Sreekumar A, Lei C, Thang LM (2006) Capture the Flag: Mixed-Reality Social Gaming with Smart Phones. IEEE Pervasive Comput 5:62–63

    Article  Google Scholar 

  8. Ciepłuch B, Jacob R, Mooney P, Winstanley A (2010) Comparison of the accuracy of OpenStreetMap for Ireland with Google maps and Bing maps, Proc 9th Int Symp Spatial Accuracy Assessment Natural Res Environ Sci, 337

  9. Clark A, Piumsomboon T (2011) A realistic augmented reality racing game using a depth-sensing camera, Proc 10th Int Conf Virtual Reality Continuum Appl Industry, 499–502

  10. Dey A, Sandor C (2014) Lessons learned: Evaluating visualizations for occluded objects in handheld augmented reality. Int J Human-Comput Studies 72:704–716

    Article  Google Scholar 

  11. Fischer J, Huhle B, Schilling A (2007) Using time-of-flight range data for occlusion handling in augmented reality, IPT/EGVE, 109–116

  12. Fischer J, Lindt I, Stenros J (2006) Final crossmedia report (part II) – Epidemic menace II evaluation report, Integrated Project Pervasive Gaming

  13. Haklay M, Weber P (2008) Openstreetmap: User-generated street maps. Pervasive Comput 7:12–18

    Article  Google Scholar 

  14. Hayashi K, Kato H, Nishida S (2005) Occlusion detection of real objects using contour based stereo matching, Proc 2005 Int Conf Augmented Tele-existence, 180–186

  15. Herbst I, Braun A-K, McCall R, Broll W (2008) TimeWarp: interactive time travel with a mobile mixed reality game, Proc 10th Int Conf Human Computer Interaction Mobile Devices Services, 235–244

  16. Hodson H (2012) Google’s Ingress game is a gold mine for augmented reality, New Scientist, 19

  17. Kasapakis V, Gavalas D (2014) Blending history and fiction in a pervasive game prototype, Proc 13th Int Conf Mobile Ubiquitous Mult, 116–122

  18. Kasapakis V, Gavalas D (2015) Pervasive gaming: Status, Trends and design principles. J Netw Comput Appl 55:213–236

    Article  Google Scholar 

  19. Kasapakis V, Gavalas D (2015) Geolocative Raycasting for real-time buildings detection in pervasive games, Proc Int Workshop Network Syst Support Games, 1–3

  20. Kasapakis V, Gavalas D, Bubaris N (2015) Pervasive Games Field Trials: Recruitment of Eligible Participants through Preliminary Game Phases. Pers Ubiquit Comput 19:523–536

    Article  Google Scholar 

  21. Kim H, Yang S-j, Sohn K (2003) 3D reconstruction of stereo images for interaction between real and virtual worlds, IEEE ACM Int Symp Mixed Augmented Reality, 169–176

  22. Lepetit V, Berger M-O (2000) Handling occlusion in augmented reality systems: a semi-automatic method, Proc IEEE ACM Int Symp Augmented Reality, 137–146

  23. Livingston MA, Ai Z, Karsch K, Gibson GO (2011) User interface design for military AR applications. Virtual Reality 15:175–184

    Article  Google Scholar 

  24. Ohta Y, Sugaya Y, Igarashi H, Ohtsuki T, Taguchi K (2002) Client/server depth sensing for see-through head-mounted displays. Presence: Teleoperators Virtual Environ 11:176–188

    Article  Google Scholar 

  25. Schroeder J (2013) AndEngine for Android game development cookbook, Packt Publishing Ltd

  26. Shah MM, Arshad H, Sulaiman R (2012) Occlusion in augmented reality, Proc 8th Int Conf Inf Sci Digital Content Technol, 372–378

  27. Thomas BH (2012) A survey of visual, mixed, and augmented reality gaming. ACM Comput Entertain 10:1–33

    Article  Google Scholar 

  28. Tian Y, Guan T, Wang C (2010) Real-time occlusion handling in augmented reality based on an object tracking approach. Sensors 10:2885–2900

    Article  Google Scholar 

  29. Tian Y, Long Y, Xia D, Yao H, Zhang J (2015) Handling occlusions in augmented reality based on 3D reconstruction method. Neurocomputing 156:96–104

    Article  Google Scholar 

  30. Wetzel W, Blum L, McCall R, Oppermann L, Broeke TS, Szalavári Z (2009) Final prototype of TimeWarp application, IPCity

  31. Yang T, Pan Q, Li J, Li SZ (2005) Real-time multiple objects tracking with occlusion handling in dynamic scenes, IEEE Comput Soc Conf Comput Vision Pattern Recognition, 970–975

  32. Zhu J, Pan Z, Sun C, Chen W (2010) Handling occlusions in video‐based augmented reality using depth information. Comput Animation Virtual Worlds 21:509–521

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Cultural Technology and Communication, University of the Aegean, Mytilene, Greece

    Vlasios Kasapakis & Damianos Gavalas

Authors
  1. Vlasios Kasapakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Damianos Gavalas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vlasios Kasapakis.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kasapakis, V., Gavalas, D. Occlusion handling in outdoors augmented reality games. Multimed Tools Appl 76, 9829–9854 (2017). https://doi.org/10.1007/s11042-016-3581-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3581-1

Keywords