Skip to main content
Springer Nature Link
Log in

Multi-step greedy Kaczmarz algorithms with simple random sampling for solving large linear systems

  • Published:
Computational and Applied Mathematics Aims and scope Submit manuscript

Abstract

By exploiting the simple random sampling and the greedy technique for capturing large residual entries, we put forth two multi-step greedy Kaczmarz algorithms for solving large linear systems. Both algorithms employ simple random sampling ab initio yet differ in the rules for choosing the working row. The new algorithms are proved to be convergent when the linear system is consistent. Some numerical experiments are carried out to verify the effectiveness of the proposed algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. In the actual computation, the MATLAB built-in function ceil is used such that ceil(\(\eta m\)) is an integer. In what follows, ceil(\(\eta m\)) is denoted by \(\eta m\) if there is no ambiguity.

  2. In the original work (Jiang et al. 2020), the authors set \(Z\ge q\) as the rejection region. However, this is in fact a two-sided hypothesis test. Therefore, it makes sense to choose the reject regions as \(|Z|\ge q\).

  3. In GRK (Bai and Wu 2018a), the computational cost involving the residual can be cheaper if the product \(AA^*\) is handy. In practice, however, it may happen that only some rows of A are accessible at a time for large-scale problems. In light of this, we only consider the general situation when \(AA^*\) is not available.

  4. In Table 5, crew1\(^{*}\) and df2177\(^{*}\) denote the conjugate transpose of crew1 and df2177, respectively.

References

  • Ansorge R (1984) Connections between the Cimmino-method and the Kaczmarz-method for the solution of singular and regular systems of equations. Computing 33:367–375

    Article  MathSciNet  Google Scholar 

  • Bai Z-Z, Pan J-Y (2021) Matrix analysis and computations. SIAM, Philadelphia

    Book  Google Scholar 

  • Bai Z-Z, Wang L, Muratova GV (2022) On relaxed greedy randomized augmented Kaczmarz methods for solving large sparse inconsistent linear systems. East Asian J Appl Math 12:323–332

    Article  MathSciNet  Google Scholar 

  • Bai Z-Z, Wu W-T (2018) On greedy randomized Kaczmarz method for solving large sparse linear systems. SIAM J Sci Comput 40:A592–A606

    Article  MathSciNet  Google Scholar 

  • Bai Z-Z, Wu W-T (2018) On relaxed greedy randomized Kaczmarz methods for solving large sparse linear systems. Appl Math Lett 83:21–26

    Article  MathSciNet  Google Scholar 

  • Bai Z-Z, Wu W-T (2018) On convergence rate of the randomized Kaczmarz method. Linear Algebra Appl 553:252–269

    Article  MathSciNet  Google Scholar 

  • Bai Z-Z, Wu W-T (2021) On greedy randomized augmented Kaczmarz method for solving large sparse inconsistent linear systems. SIAM J Sci Comput 43:A3892–A3911

    Article  MathSciNet  Google Scholar 

  • Baron M (2019) Probability and statistics for computer scientists. CRC Press, New York

    Book  Google Scholar 

  • Censor Y (1981) Row-action methods for huge and sparse systems and their applications. SIAM Rev 23:444–466

    Article  MathSciNet  Google Scholar 

  • Davis TA, Hu Y (2011) The university of Florida sparse matrix collection. ACM Trans Math Softw 38:1–25

    MathSciNet  MATH  Google Scholar 

  • De Loera JA, Haddock J, Needell D (2017) A sampling Kaczmarz–Motzkin algorithm for linear feasibility. SIAM J Sci Comput 39:S66–S87

    Article  MathSciNet  Google Scholar 

  • Du Y-S, Hayami K, Zheng N, Morikuni K, Yin J-F (2021) Kaczmarz-type inner-iteration preconditioned flexible GMRES methods for consistent linear systems. SIAM J Sci Comput 43:S345–S366

    Article  MathSciNet  Google Scholar 

  • Du K, Si W-T, Sun X-H (2020) Randomized extended average block Kaczmarz for solving least squares. SIAM J Sci Comput 42:A3541–A3559

    Article  MathSciNet  Google Scholar 

  • Du K, Sun X-H (2021) Randomized double and triple Kaczmarz for solving extended normal equations. Calcolo 58:1–13

    Article  MathSciNet  Google Scholar 

  • Eldar YC, Needell D (2011) Acceleration of randomized Kaczmarz method via the Johnson-Lindenstrauss lemma. Numer Algorithms 58:163–177

    Article  MathSciNet  Google Scholar 

  • Escalante R, Raydan M (2011) Alternating projection methods. SIAM, Philadelphia

    Book  Google Scholar 

  • Gordon R, Bender R, Herman GT (1970) Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and X-ray photography. J Theor Biol 29:471–481

    Article  Google Scholar 

  • Gower RM, Richtárik P (2015) Randomized iterative methods for linear systems. SIAM J Matrix Anal Appl 36:1660–1690

    Article  MathSciNet  Google Scholar 

  • Hansen PC, Jørgensen JS (2018) AIR Tools II: algebraic iterative reconstruction methods, improved implementation. Numer Algorithms 79:107–137

    Article  MathSciNet  Google Scholar 

  • Haddock J, Ma A (2021) Greed works: an improved analysis of sampling Kaczmarz-Motzkin. SIAM J Math Data Sci 3:342–368

    Article  MathSciNet  Google Scholar 

  • Herman GT (2009) Fundamentals of computerized tomography: image reconstruction from projections. Springer, New York

    Book  Google Scholar 

  • Herman GT, Davidi R (2008) Image reconstruction from a small number of projections. Inverse Prob 24:045011

    Article  MathSciNet  Google Scholar 

  • Jiang X-L, Zhang K, Yin J-F (2022) Randomized block Kaczmarz methods with \(k\)-means clustering for solving large linear systems. J Comput Appl Math 403:113828

    Article  MathSciNet  Google Scholar 

  • Jiang Y, Wu G, Jiang L (2020) A Kaczmarz method with simple random sampling for solving large linear systems. arXiv:2011.14693

  • Kaczmarz S (1937) Angenäherte Auflösung von systemen linearer gleichungen. Bull Int Acad Pol Sci Lett A 35:355–357

    MATH  Google Scholar 

  • Kolodziej SP, Aznaveh M, Bullock M, David J, Davis TA, Henderson M, Hu Y, Sandstrom R (2019) The SuiteSparse matrix collection website interface. J Open Source Softw 4:1244–1248

    Article  Google Scholar 

  • Li H, Zhang Y (2020a) A novel greedy Kaczmarz method for solving consistent linear systems. arXiv:2004.02062

  • Li H, Zhang Y (2020b) Greedy block Gauss-Seidel methods for solving large linear least squares problem. arXiv:2004.02476

  • Liu Y, Gu C-Q (2019) Variant of greedy randomized Kaczmarz for ridge regression. Appl Numer Math 143:223–246

    Article  MathSciNet  Google Scholar 

  • Lohr SL (2019) Sampling: design and analysis. CRC Press, New York

    Book  Google Scholar 

  • Lorenz DA, Wenger S, Schopfer F, Magnor M (2014) A sparse Kaczmarz solver and a linearized Bregman method for online compressed sensing. In: IEEE International Conference on Image Processing (ICIP), Paris, France

  • Lv Z, Bao W, Li W, Wang F, Wu G (2022) On extended Kaczmarz methods with random sampling and maximum-distance for solving large inconsistent linear systems. Results Appl Math 13:100240

    Article  MathSciNet  Google Scholar 

  • Martinsson P-G, Tropp JA (2020) Randomized numerical linear algebra: foundations and algorithms. Acta Numer 29:403–572

    Article  MathSciNet  Google Scholar 

  • Needell D, Tropp JA (2014) Paved with good intentions: analysis of a randomized block Kaczmarz method. Linear Algebra Appl 441:199–221

    Article  MathSciNet  Google Scholar 

  • Saad Y (2003) Iterative methods for sparse linear systems. SIAM, Philadelphia

    Book  Google Scholar 

  • Strohmer T, Vershynin R (2009) A randomized Kaczmarz algorithm with exponential convergence. J Fourier Anal Appl 15:262–278

    Article  MathSciNet  Google Scholar 

  • Tanabe K (1971) Projection method for solving a singular system of linear equations and its applications. Numer Math 17:203–214

    Article  MathSciNet  Google Scholar 

  • Woodruff DP (2014) Sketching as a tool for numerical linear algebra. Theoret Comput Sci 10:1–157

    Article  MathSciNet  Google Scholar 

  • Zhang J-J (2019) A new greedy Kaczmarz algorithm for the solution of very large linear systems. Appl Math Lett 91:207–212

    Article  MathSciNet  Google Scholar 

  • Zhang Y, Li H (2021) A count sketch maximal weighted residual Kaczmarz method for solving highly overdetermined linear systems. App Math Comput 410:126486

    Article  MathSciNet  Google Scholar 

  • Zhang Y, Li H (2021) Block sampling Kaczmarz-Motzkin methods for consistent linear systems. Calcolo 58:39

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (Nos. 12271342 and 11601323) and National Social Science Foundation of China (No. 21BTJ038). The authors are grateful to the referee for the insightful comments and valuable suggestions which greatly improved the original version of this paper.

Author information

Authors and Affiliations

  1. Department of Mathematics, Shanghai Maritime University, Shanghai, 201306, People’s Republic of China

    Ke Zhang, Fu-Ting Li & Xiang-Long Jiang

Authors
  1. Ke Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Fu-Ting Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Xiang-Long Jiang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Ke Zhang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Communicated by Yimin Wei.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, K., Li, FT. & Jiang, XL. Multi-step greedy Kaczmarz algorithms with simple random sampling for solving large linear systems. Comp. Appl. Math. 41, 332 (2022). https://doi.org/10.1007/s40314-022-02044-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s40314-022-02044-5

Keywords

Mathematics Subject Classification