Skip to main content
Springer Nature Link
Log in

ELM-NET, a closer to practice approach for classifying the big data using multiple independent ELMs

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

In this paper, a new ELM based classification method is presented to deal with the large volume of data in an efficient way. By inspiration from both parallel and sequential ELMs, this method consists of some independent ELMs which are trained using data batches in parallel. The main goal of this method is preventing exponential training time by running some ELMs in parallel which similar to the sequential methods, are trained using different data chunks. Moreover, a new aggregation method is used here to outperform this structure which can relatively achieve stable results for the different number of the ELMs. The stable results can persuade us to use such classification method on regular platforms to decrease the cost of the big data analyzing. Our method is tested on different platforms to indicate that it can be used for reducing the costs of big data analyzing. Experimental results on MNIST, KDDCup99, KDDCup99_2, Susy, and Higgs datasets shows the better performance of our method than the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Suykens, J.A., Vandewalle, J.: Least squares support vector machine classifiers. Neural Process. Lett. 9(3), 293–300 (1999)

    Google Scholar 

  2. Scholkopf, B., Smola, A.J.: Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT Press, Cambridge (2001)

    Google Scholar 

  3. Choi, J.K., Jeon, K.H., Won, Y., Kim, J.J.: Application of big data analysis with decision tree for the foot disorder. Clust. Comput. 18(4), 1399–1404 (2015)

    Google Scholar 

  4. Adamo, J.M.: Fuzzy decision trees. Fuzzy Sets Syst. 4(3), 207–219 (1980)

    MathSciNet  MATH  Google Scholar 

  5. Quinlan, J.R.: Decision trees and decision-making. IEEE Trans. Sys. Man Cybern. 20(2), 339–346 (1990)

    Google Scholar 

  6. Quinlan, J.R.: Induction of decision trees. Mach. Learn. 1(1), 81–106 (1986)

    Google Scholar 

  7. Huang, G.B., Wang, D.H., Lan, Y.: Extreme learning machines: a survey. Int. J. Mach. Learn. Cybern. 2(2), 107–122 (2011)

    Google Scholar 

  8. Huang, G.B., Zhou, H., Ding, X., Zhang, R.: Extreme learning machine for regression and multiclass classification. IEEE Trans. Syst. Man Cybern. Part B 42(2), 513–529 (2012)

    Google Scholar 

  9. Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: a new learning scheme of feedforward neural networks. Neural Netw. 2, 985–990 (2004)

    Google Scholar 

  10. Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Neurocomputing 70(1), 489–501 (2006)

    Google Scholar 

  11. Kirk, D.B., Wen-Mei, W.H.: Programming massively parallel processors: a hands-on approach. Elsevier, New York (2016)

    Google Scholar 

  12. Leighton, F.T.: Introduction to parallel algorithms and architectures: Arrays trees hypercubes. Elsevier, New York (2014)

    MATH  Google Scholar 

  13. Chen, C., Li, K., Ouyang, A., Tang, Z., Li, K.: Gpu-accelerated parallel hierarchical extreme learning machine on flink for big data. IEEE Trans. Syst. Man Cybern. 47(10), 2740–2753 (2017)

    Google Scholar 

  14. He, Q., Shang, T., Zhuang, F., Shi, Z.: Parallel extreme learning machine for regression based on MapReduce. Neurocomputing 102, 52–58 (2013)

    Google Scholar 

  15. He, Y., Geng, Z., Zhu, Q.: Positive and negative correlation input attributes oriented subnets based double parallel extreme learning machine (PNIAOS-DPELM) and its application to monitoring chemical processes in steady state. Neurocomputing 165, 171–181 (2015)

    Google Scholar 

  16. Wang, B., Huang, S., Qiu, J., Liu, Y., Wang, G.: Parallel online sequential extreme learning machine based on MapReduce. Neurocomputing 149, 224–232 (2015)

    Google Scholar 

  17. Wang, Y., Dou, Y., Liu, X., Lei, Y.: PR-ELM: parallel regularized extreme learning machine based on cluster. Neurocomputing 173, 1073–1081 (2016)

    Google Scholar 

  18. Roul, R.K., Nanda, A., Patel, V., Sahay, S.K.: Extreme learning machines in the field of text classification. In: 16th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, pp. 1–7 (2015)

  19. Zheng, W., Qian, Y., Lu, H.: Text categorization based on regularization extreme learning machine. Neural Comput. Appl. 22(3), 447–456 (2013)

    Google Scholar 

  20. He, B., Xu, D., Nian, R., van Heeswijk, M., Yu, Q., Miche, Y., Lendasse, A.: Fast face recognition via sparse coding and extreme learning machine. Cognit. Comput. 6(2), 264–277 (2014)

    Google Scholar 

  21. Marques, I., Graña, M.: Face recognition with lattice independent component analysis and extreme learning machines. Soft. Comput. 16(9), 1525–1537 (2012)

    Google Scholar 

  22. Cecotti, H., Boumedine, C., Callaghan, M.: Hand-drawn symbol recognition in immersive virtual reality using deep extreme learning machines. In: International Conference on Recent Trends in Image Processing and Pattern Recognition, pp. 80–92 (2016)

  23. Krawczyk, B.: GPU-accelerated extreme learning machines for imbalanced data streams with concept drift. Proc. Comput. Sci 80, 1692–1701 (2016)

    Google Scholar 

  24. Xu, S., Wang, J.: Dynamic extreme learning machine for data stream classification. Neurocomputing 238, 433–449 (2017)

    Google Scholar 

  25. Lu, S., Wang, X., Zhang, G., Zhou, X.: Effective algorithms of the Moore–Penrose inverse matrices for extreme learning machine. Intell. Data Anal. 19(4), 743–760 (2015)

    Google Scholar 

  26. Pei, H., Wang, K., Lin, Q., Zhong, P.: Robust semi-supervised extreme learning machine. Knowl. Based Syst. 159, 203–220 (2018)

    Google Scholar 

  27. Scardapane, S., Comminiello, D., Scarpiniti, M., Uncini, A.: Online sequential extreme learning machine with kernels. IEEE Trans. Neural Netw. Learn. Syst. 26(9), 2214–2220 (2015)

    MathSciNet  Google Scholar 

  28. Vong, C.M., Tai, K.I., Pun, C.M., Wong, P.K.: Fast and accurate face detection by sparse Bayesian extreme learning machine. Neural Comput. Appl. 26(5), 1149–1156 (2015)

    Google Scholar 

  29. Zhai, J.H., Xu, H.Y., Wang, X.Z.: Dynamic ensemble extreme learning machine based on sample entropy. Soft. Comput. 16(9), 1493–1502 (2012)

    Google Scholar 

  30. Liao, S., Feng, C.: Meta-ELM: ELM with ELM hidden nodes. Neurocomputing 128, 81–87 (2014)

    Google Scholar 

  31. Huang, G., Huang, G.B., Song, S., You, K.: Trends in extreme learning machines: a review. Neural Netw. 61, 32–48 (2015)

    MATH  Google Scholar 

  32. Duan, M., Li, K., Liao, X., Li, K.: A parallel multiclassification algorithm for big data using an extreme learning machine. IEEE Trans. Neural Netw. Learn. Syst. 29(6), 2337–2351 (2018)

    MathSciNet  Google Scholar 

  33. Xin, J., Wang, Z., Qu, L., Yu, G., Kang, Y.: A-ELM: adaptive distributed extreme learning machine with MapReduce. Neurocomputing 174, 368–374 (2016)

    Google Scholar 

  34. Lee, K.H., Lee, Y.J., Choi, H., Chung, Y.D., Moon, B.: Parallel data processing with MapReduce: a survey. AcM sIGMoD Record 40(4), 11–20 (2012)

    Google Scholar 

  35. Liu, T., Fang, Z., Zhao, C., Zhou, Y.: Parallelization of a series of extreme learning machine algorithms based on spark. In: 15th International Conference on Computer and Information Science, pp. 1–5 (2016)

  36. Han, M., Liu, B.: Ensemble of extreme learning machine for remote sensing image classification. Neurocomputing 149, 65–70 (2015)

    Google Scholar 

  37. Jin, Y., Cao, J., Wang, Y., Zhi, R.: Ensemble based extreme learning machine for cross-modality face matching. Multimed. Tools Appl. 75(19), 11831–11846 (2016)

    Google Scholar 

  38. Huang, S., Wang, B., Qiu, J., Yao, J., Wang, G., Yu, G.: Parallel ensemble of online sequential extreme learning machine based on MapReduce. Neurocomputing 174, 352–367 (2016)

    Google Scholar 

  39. Tang, X., Chen, L.: Artificial bee colony optimization-based weighted extreme learning machine for imbalanced data learning. Clust. Comput., 1–16 (2018)

  40. Luo, J., Vong, C.M., Wong, P.K.: Sparse Bayesian extreme learning machine for multi-classification. IEEE Trans. Neural Netw. Learn. Syst. 25(4), 836–843 (2014)

    Google Scholar 

  41. Lu, H.J., An, C.L., Zheng, E.H., Lu, Y.: Dissimilarity based ensemble of extreme learning machine for gene expression data classification. Neurocomputing 128, 22–30 (2014)

    Google Scholar 

  42. He, Q., Zhuang, F., Li, J., Shi, Z.: Parallel implementation of classification algorithms based on MapReduce. International Conference on Rough Sets and Knowledge Technology, pp. 655–662. Springer, Berlin (2010)

    Google Scholar 

  43. Serre, D.: Matrices. Graduate Texts in Mathematics, vol. 216. Springer, Berlin (2002)

    Google Scholar 

  44. Rao, C.R.: Generalized inverse of matrices and its applications. Elsevier, New York (1971)

    MATH  Google Scholar 

  45. Huang, G.B., Bai, Z., Kasun, L.L.C., Vong, C.M.: Local receptive fields based extreme learning machine. IEEE Comput. Intell. Mag. 10(2), 18–29 (2015)

    Google Scholar 

  46. Zhang, C., Li, F., Jestes, J.: Efficient parallel kNN joins for large data in MapReduce. In: Proceedings of the 15th International Conference on Extending Database Technology, pp. 38–49 (2012)

  47. You, Z.H., Yu, J.Z., Zhu, L., Li, S., Wen, Z.K.A.: MapReduce based parallel SVM for large-scale predicting protein–protein interactions. Neurocomputing 145, 37–43 (2014)

    Google Scholar 

  48. Panda, B., Herbach, J.S., Basu, S., Bayardo, R.J.: Planet: massively parallel learning of tree ensembles with mapreduce. Proc. VLDB Endow. 2(2), 1426–1437 (2009)

    Google Scholar 

  49. Xin, J., Wang, Z., Chen, C., Ding, L., Wang, G., Zhao, Y.: ELM∗: distributed extreme learning machine with MapReduce. World Wide Web 17(5), 1189–1204 (2014)

    Google Scholar 

  50. Xin, J., Wang, Z., Qu, L., Wang, G.: Elastic extreme learning machine for big data classification. Neurocomputing 149, 464–471 (2015)

    Google Scholar 

  51. Inaba, F.K., Salles, E.O.T., Perron, S., Caporossi, G.: DGR-ELM-distributed generalized regularized ELM for classification. Neurocomputing 275, 1522–1530 (2018)

    Google Scholar 

  52. Chen, J., Chen, H., Wan, X., Zheng, G.: MR-ELM: a MapReduce-based framework for large-scale ELM training in big data era. Neural Comput. Appl. 27(1), 101–110 (2016)

    Google Scholar 

  53. Chen, C., Li, K., Ouyang, A., Li, K.: FlinkCL: an OpenCL-based in-memory computing architecture on heterogeneous CPU-GPU clusters for big data. IEEE Trans. Comput. 67(2), 1765–1779 (2018)

    MathSciNet  MATH  Google Scholar 

  54. Van Heeswijk, M., Miche, Y., Oja, E., Lendasse, A.: GPU-accelerated and parallelized ELM ensembles for large-scale regression. Neurocomputing 74(16), 2430–2437 (2011)

    Google Scholar 

  55. Liang, N.Y., Huang, G.B., Saratchandran, P., Sundararajan, N.: A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans. Neural Networks 17(6), 1411–1423 (2006)

    Google Scholar 

  56. Huang, S., Wang, B., Chen, Y., Wang, G., Yu, G.: Efficient batch parallel online sequential extreme learning machine algorithm based on MapReduce. In: Proceedings of ELM 2015, pp. 13–25 (2016)

  57. Huang, S., Wang, B., Chen, Y., Wang, G., Yu, G.: An efficient parallel method for batched OS-ELM training using MapReduce. Memet. Comput. 9(3), 183–197 (2017)

    Google Scholar 

  58. Segatori, A., Bechini, A., Ducange, P., Marcelloni, F.: A distributed fuzzy associative classifier for big data. IEEE Trans. Cybern. 48(9), 2656–2669 (2018)

    Google Scholar 

  59. Bechini, A., Marcelloni, F., Segatori, A.: A MapReduce solution for associative classification of big data. Inf. Sci. 332, 33–55 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Faculty of Engineering, University of Kurdistan, Sanandaj, Iran

    Amin Shokrzade & Fardin Akhlaghian Tab

  2. Department of Computer Science, University of Kurdistan, Sanandaj, Iran

    Mohsen Ramezani

Authors
  1. Amin Shokrzade
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Fardin Akhlaghian Tab
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Mohsen Ramezani
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Fardin Akhlaghian Tab.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shokrzade, A., Tab, F.A. & Ramezani, M. ELM-NET, a closer to practice approach for classifying the big data using multiple independent ELMs. Cluster Comput 23, 735–757 (2020). https://doi.org/10.1007/s10586-019-02957-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-019-02957-7

Keywords