Skip to main content
Springer Nature Link
Log in

Flow data processing paradigm and its application in smart city using a cluster analysis approach

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

In the digital revolution era the work style of the people changed into real and live communicative systems. Smart cities are evolved from the smart ecosystem with various domains like healthcare, information exchange, transportation, buildings etc., the presence of multiple information system makes these smart cities into a heterogeneous which includes multiple data transfer based on the large no of interconnected sub systems. Since the systems provide data from various terminals to the host the amount of information generates is same as the chances of increasing the challenges and issues in the smart city. These flow data from various sources are handled using cluster computing and this proposed research provides the data flow in the smart city using cluster computing. Apache spark is the tool used to handle the issues in the smart city which is much better than the Hadoop. The results discuss the comparison between the applications in same environment on both the tools.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Ranjbar, M., Amiri, M.: On the role of astrocyte analog circuit in neural frequency adaptation. Neural Comput. Appl. 28(5), 1109–1121 (2017)

    Article  Google Scholar 

  2. Chen, Q., Zhang, G., Yang, X., Li, S., Li, Y., Wang, H.H.: Single image shadow detection and removal based on feature fusion and multiple dictionary learning. Multimed. Tools Appl. 77, 18601–18624 (2018)

    Article  Google Scholar 

  3. Xiong, W., Shi, Y., Cao, J.: Stability analysis of two-dimensional neutral-type Cohen–Grossberg BAM neural networks. Neural Comput. Appl. 28(4), 703–716 (2017)

    Article  Google Scholar 

  4. Zhang, S., Wang, H., Huang, W., You, Z.: Plant diseased leaf segmentation and recognition by fusion of superpixel, K-means and PHOG. Optik Int. J. Light Electron Opt. 157, 866–872 (2018)

    Article  Google Scholar 

  5. Mansouri, I., Gholampour, A., Kisi, O., Ozbakkaloglu, T.: Evaluation of peak and residual conditions of actively confined concrete using neuro-fuzzy and neural computing techniques. Neural Comput. Appl. 29(3), 873–888 (2018)

    Article  Google Scholar 

  6. Zhang, Y., Ren, J., Liu, J., Xu, C., Guo, H., Liu, Y.: A survey on emerging computing paradigms for big data. Chin. J. Electron. 26(1), 1–12 (2017)

    Article  Google Scholar 

  7. Duan, M., Li, K., Tang, Z., Xiao, G., Li, K.: Selection and replacement algorithms for memory performance improvement in spark. Concurr. Comput. Pract. Exp. 28(8), 2473–2486 (2016)

    Article  Google Scholar 

  8. Zhang, Y., Liu, M., Ma, B., Zhen, Y.: The performance evaluation of diagonal recurrent neural network with different chaos neurons. Neural Comput. Appl. 28(7), 1611–1618 (2017)

    Article  Google Scholar 

  9. Koçer, S., Tümer, A.E.: Classifying neuromuscular diseases using artificial neural networks with applied Autoregressive and Cepstral analysis. Neural Comput. Appl. 28(1), 945–952 (2017)

    Article  Google Scholar 

  10. Huang, W., Meng, L., Zhang, D., Zhang, W.: In-memory parallel processing of massive remotely sensed data using an apache spark on Hadoop YARN model. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 10(1), 3–19 (2017)

    Article  Google Scholar 

  11. Chen, J., Li, K., Tang, Z., Bilal, K., Yu, S., Weng, C., Li, K.: A parallel random forest algorithm for big data in a spark cloud computing environment. IEEE Trans. Parallel Distrib. Syst. 28(4), 919–933 (2017)

    Article  Google Scholar 

  12. Krim, H., Gentimis, T., Chintakunta, H.: Discovering the whole by the coarse: a topological paradigm for data analysis. IEEE Signal Process. Mag. 33(2), 95–104 (2016)

    Article  Google Scholar 

  13. Liu, H., Ning, H., Xiong, Q., Yang, L.T.: Shared authority based privacy-preserving authentication protocol in cloud computing. IEEE Trans. Parallel Distrib. Syst. 26(1), 241–251 (2015)

    Article  Google Scholar 

  14. Cao, D., Liu, P., Cui, W., Zhong, Y., An, B.: Cluster as a service: a resource sharing approach for private cloud. Tsinghua Sci. Technol. 21(6), 610–619 (2016)

    Article  MATH  Google Scholar 

  15. Munir, A., Kansakar, P., Khan, S.U.: IFCIoT: integrated Fog Cloud IoT: a novel architectural paradigm for the future Internet of Things. IEEE Consum. Electron. Mag. 6(3), 74–82 (2017)

    Article  Google Scholar 

  16. Cai, H., Xu, B., Jiang, L., Vasilakos, A.V.: IoT-based big data storage systems in cloud computing: perspectives and challenges. IEEE Internet Things J. 4(1), 75–87 (2017)

    Google Scholar 

  17. Khalid, O., Khan, M.U.S., Huang, Y., Khan, S.U., Zomaya, A.: Evacsys: a cloud-based service for emergency evacuation. IEEE Cloud Comput. 3(1), 60–68 (2016)

    Article  Google Scholar 

  18. Yassine, A., Singh, S., Alamri, A.: Mining human activity patterns from smart home big data for health care applications. IEEE Access 5, 13131–13141 (2017)

    Article  Google Scholar 

  19. Ma, Y., et al.: Remote sensing big data computing: challenges and opportunities. Future Gener. Comput. Syst. 51, 47–60 (2015)

    Article  Google Scholar 

  20. Alves Filho, S.E., Burlamaqui, A.M.F., Aroca, R.V., Gonçalves, L.M.G.: NPi-cluster: a low power energy-proportional computing cluster architecture. IEEE Access 5, 16297–16313 (2017)

    Article  Google Scholar 

  21. Giachetta, R.: A framework for processing large scale geospatial and remote sensing data in map reduce environment. Comput. Graph 49, 37–46 (2015)

    Article  Google Scholar 

  22. Brisimi, T.S., Cassandras, C.G., Osgood, C., Paschalidis, I.C.H., Zhang, Y.: Sensing and classifying roadway obstacles in smart cities: the street bump system. IEEE Access 4, 1301–1312 (2016)

    Article  Google Scholar 

  23. Lyu, Y., et al.: High-performance scheduling model for multisensor gateway of cloud sensor system-based smart-living. Inf. Fusion 21, 42–56 (2015)

    Article  Google Scholar 

  24. Rehman, M.H., Liew, C.S., Wah, T.Y., Khan, M.K.: Towards next-generation heterogeneous mobile data stream mining applications: opportunities challenges and future research directions. J. Netw. Comput. Appl. 79, 1–24 (2017)

    Article  Google Scholar 

  25. Cickovski, T., Flor, T., Irving-Sachs, G., Novikov, P., Parda, J., Narasimhan, G.: GPUDePiCt: a parallel implementation of a clustering algorithm for computing degenerate primers on graphics processing units. IEEE/ACM Trans. Comput. Biol. Bioinform. 12(2), 445–454 (2015)

    Article  Google Scholar 

Download references

Acknowledgements

The paper is supported by the National Natural Science Foundation Project “Research on key technology of malicious node identification in wireless sensor network”. 61772101, 2018.1-2021.12 and Liaoning Provincial Education Department Project, the Innovation Ream of Liaoning University, “ubiquitous network security and computing technology” ZX2015KJ018.

Author information

Authors and Affiliations

  1. Software Center, Northeastern University, Shenyang, People’s Republic of China

    Xiang Zou & Tao Wen

  2. Dalian Neusoft Institute of Information, Dalian, People’s Republic of China

    Xiang Zou, Jinghua Cao, Wei Sun, Quan Guo & Tao Wen

  3. Dalian Maritime University, Dalian, People’s Republic of China

    Jinghua Cao

Authors
  1. Xiang Zou
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jinghua Cao
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Wei Sun
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Quan Guo
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Tao Wen
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Tao Wen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zou, X., Cao, J., Sun, W. et al. Flow data processing paradigm and its application in smart city using a cluster analysis approach. Cluster Comput 22, 435–444 (2019). https://doi.org/10.1007/s10586-018-2839-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-018-2839-y

Keywords