diff --git a/previews/PR911/.documenter-siteinfo.json b/previews/PR911/.documenter-siteinfo.json index ac79c4d4d..b3b72f0a7 100644 --- a/previews/PR911/.documenter-siteinfo.json +++ b/previews/PR911/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.11.1","generation_timestamp":"2024-10-27T04:15:46","documenter_version":"1.7.0"}} \ No newline at end of file +{"documenter":{"julia_version":"1.11.1","generation_timestamp":"2024-10-27T04:29:01","documenter_version":"1.7.0"}} \ No newline at end of file diff --git a/previews/PR911/api/index.html b/previews/PR911/api/index.html index 7b167991d..726b8562b 100644 --- a/previews/PR911/api/index.html +++ b/previews/PR911/api/index.html @@ -1,41 +1,41 @@ -API · Krylov.jl
GitHub

Stats Types

Krylov.SimpleStats — Type

Type for statistics returned by the majority of Krylov solvers, the attributes are:

  • niter
  • solved
  • inconsistent
  • residuals
  • Aresiduals
  • Acond
  • timer
  • status
source
Krylov.LanczosStats — Type

Type for statistics returned by CG-LANCZOS, the attributes are:

  • niter
  • solved
  • residuals
  • indefinite
  • Anorm
  • Acond
  • timer
  • status
source
Krylov.LanczosShiftStats — Type

Type for statistics returned by CG-LANCZOS with shifts, the attributes are:

  • niter
  • solved
  • residuals
  • indefinite
  • Anorm
  • Acond
  • timer
  • status
source
Krylov.SymmlqStats — Type

Type for statistics returned by SYMMLQ, the attributes are:

  • niter
  • solved
  • residuals
  • residualscg
  • errors
  • errorscg
  • Anorm
  • Acond
  • timer
  • status
source
Krylov.AdjointStats — Type

Type for statistics returned by adjoint systems solvers BiLQR and TriLQR, the attributes are:

  • niter
  • solved_primal
  • solved_dual
  • residuals_primal
  • residuals_dual
  • timer
  • status
source
Krylov.LNLQStats — Type

Type for statistics returned by the LNLQ method, the attributes are:

  • niter
  • solved
  • residuals
  • errorwithbnd
  • errorbndx
  • errorbndy
  • timer
  • status
source
Krylov.LSLQStats — Type

Type for statistics returned by the LSLQ method, the attributes are:

  • niter
  • solved
  • inconsistent
  • residuals
  • Aresiduals
  • err_lbnds
  • errorwithbnd
  • errubndslq
  • errubndscg
  • timer
  • status
source
Krylov.LsmrStats — Type

Type for statistics returned by LSMR. The attributes are:

  • niter
  • solved
  • inconsistent
  • residuals
  • Aresiduals
  • Acond
  • Anorm
  • xNorm
  • timer
  • status
source

Solver Types

Krylov.MinresSolver — Type

Type for storing the vectors required by the in-place version of MINRES.

The outer constructors

solver = MinresSolver(m, n, S; window :: Int=5)
-solver = MinresSolver(A, b; window :: Int=5)

may be used in order to create these vectors.

source
Krylov.MinaresSolver — Type

Type for storing the vectors required by the in-place version of MINARES.

The outer constructors

solver = MinaresSolver(m, n, S)
-solver = MinaresSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CgSolver — Type

Type for storing the vectors required by the in-place version of CG.

The outer constructors

solver = CgSolver(m, n, S)
-solver = CgSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CrSolver — Type

Type for storing the vectors required by the in-place version of CR.

The outer constructors

solver = CrSolver(m, n, S)
-solver = CrSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CarSolver — Type

Type for storing the vectors required by the in-place version of CAR.

The outer constructors

solver = CarSolver(m, n, S)
-solver = CarSolver(A, b)

may be used in order to create these vectors.

source
Krylov.SymmlqSolver — Type

Type for storing the vectors required by the in-place version of SYMMLQ.

The outer constructors

solver = SymmlqSolver(m, n, S)
-solver = SymmlqSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CgLanczosSolver — Type

Type for storing the vectors required by the in-place version of CG-LANCZOS.

The outer constructors

solver = CgLanczosSolver(m, n, S)
-solver = CgLanczosSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CgLanczosShiftSolver — Type

Type for storing the vectors required by the in-place version of CG-LANCZOS-SHIFT.

The outer constructors

solver = CgLanczosShiftSolver(m, n, nshifts, S)
-solver = CgLanczosShiftSolver(A, b, nshifts)

may be used in order to create these vectors.

source
Krylov.MinresQlpSolver — Type

Type for storing the vectors required by the in-place version of MINRES-QLP.

The outer constructors

solver = MinresQlpSolver(m, n, S)
-solver = MinresQlpSolver(A, b)

may be used in order to create these vectors.

source
Krylov.DiomSolver — Type

Type for storing the vectors required by the in-place version of DIOM.

The outer constructors

solver = DiomSolver(m, n, memory, S)
-solver = DiomSolver(A, b, memory = 20)

may be used in order to create these vectors. memory is set to n if the value given is larger than n. memory is an optional argument in the second constructor.

source
Krylov.FomSolver — Type

Type for storing the vectors required by the in-place version of FOM.

The outer constructors

solver = FomSolver(m, n, memory, S)
-solver = FomSolver(A, b, memory = 20)

may be used in order to create these vectors. memory is set to n if the value given is larger than n. memory is an optional argument in the second constructor.

source
Krylov.DqgmresSolver — Type

Type for storing the vectors required by the in-place version of DQGMRES.

The outer constructors

solver = DqgmresSolver(m, n, memory, S)
-solver = DqgmresSolver(A, b, memory = 20)

may be used in order to create these vectors. memory is set to n if the value given is larger than n. memory is an optional argument in the second constructor.

source
Krylov.GmresSolver — Type

Type for storing the vectors required by the in-place version of GMRES.

The outer constructors

solver = GmresSolver(m, n, memory, S)
-solver = GmresSolver(A, b, memory = 20)

may be used in order to create these vectors. memory is set to n if the value given is larger than n. memory is an optional argument in the second constructor.

source
Krylov.UsymlqSolver — Type

Type for storing the vectors required by the in-place version of USYMLQ.

The outer constructors

solver = UsymlqSolver(m, n, S)
-solver = UsymlqSolver(A, b)

may be used in order to create these vectors.

source
Krylov.UsymqrSolver — Type

Type for storing the vectors required by the in-place version of USYMQR.

The outer constructors

solver = UsymqrSolver(m, n, S)
-solver = UsymqrSolver(A, b)

may be used in order to create these vectors.

source
Krylov.TricgSolver — Type

Type for storing the vectors required by the in-place version of TRICG.

The outer constructors

solver = TricgSolver(m, n, S)
-solver = TricgSolver(A, b)

may be used in order to create these vectors.

source
Krylov.TrimrSolver — Type

Type for storing the vectors required by the in-place version of TRIMR.

The outer constructors

solver = TrimrSolver(m, n, S)
-solver = TrimrSolver(A, b)

may be used in order to create these vectors.

source
Krylov.TrilqrSolver — Type

Type for storing the vectors required by the in-place version of TRILQR.

The outer constructors

solver = TrilqrSolver(m, n, S)
-solver = TrilqrSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CgsSolver — Type

Type for storing the vectors required by the in-place version of CGS.

The outer constructorss

solver = CgsSolver(m, n, S)
-solver = CgsSolver(A, b)

may be used in order to create these vectors.

source
Krylov.BicgstabSolver — Type

Type for storing the vectors required by the in-place version of BICGSTAB.

The outer constructors

solver = BicgstabSolver(m, n, S)
-solver = BicgstabSolver(A, b)

may be used in order to create these vectors.

source
Krylov.BilqSolver — Type

Type for storing the vectors required by the in-place version of BILQ.

The outer constructors

solver = BilqSolver(m, n, S)
-solver = BilqSolver(A, b)

may be used in order to create these vectors.

source
Krylov.QmrSolver — Type

Type for storing the vectors required by the in-place version of QMR.

The outer constructors

solver = QmrSolver(m, n, S)
-solver = QmrSolver(A, b)

may be used in order to create these vectors.

source
Krylov.BilqrSolver — Type

Type for storing the vectors required by the in-place version of BILQR.

The outer constructors

solver = BilqrSolver(m, n, S)
-solver = BilqrSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CglsSolver — Type

Type for storing the vectors required by the in-place version of CGLS.

The outer constructors

solver = CglsSolver(m, n, S)
-solver = CglsSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CglsLanczosShiftSolver — Type

Workspace for the in-place version of CGLS-LANCZOS-SHIFT.

The outer constructors:

solver = CglsLanczosShiftSolver(m, n, nshifts, S)
-solver = CglsLanczosShiftSolver(A, b, nshifts)

can be used to initialize this workspace.

source
Krylov.CrlsSolver — Type

Type for storing the vectors required by the in-place version of CRLS.

The outer constructors

solver = CrlsSolver(m, n, S)
-solver = CrlsSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CgneSolver — Type

Type for storing the vectors required by the in-place version of CGNE.

The outer constructors

solver = CgneSolver(m, n, S)
-solver = CgneSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CrmrSolver — Type

Type for storing the vectors required by the in-place version of CRMR.

The outer constructors

solver = CrmrSolver(m, n, S)
-solver = CrmrSolver(A, b)

may be used in order to create these vectors.

source
Krylov.LslqSolver — Type

Type for storing the vectors required by the in-place version of LSLQ.

The outer constructors

solver = LslqSolver(m, n, S)
-solver = LslqSolver(A, b)

may be used in order to create these vectors.

source
Krylov.LsqrSolver — Type

Type for storing the vectors required by the in-place version of LSQR.

The outer constructors

solver = LsqrSolver(m, n, S)
-solver = LsqrSolver(A, b)

may be used in order to create these vectors.

source
Krylov.LsmrSolver — Type

Type for storing the vectors required by the in-place version of LSMR.

The outer constructors

solver = LsmrSolver(m, n, S)
-solver = LsmrSolver(A, b)

may be used in order to create these vectors.

source
Krylov.LnlqSolver — Type

Type for storing the vectors required by the in-place version of LNLQ.

The outer constructors

solver = LnlqSolver(m, n, S)
-solver = LnlqSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CraigSolver — Type

Type for storing the vectors required by the in-place version of CRAIG.

The outer constructors

solver = CraigSolver(m, n, S)
-solver = CraigSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CraigmrSolver — Type

Type for storing the vectors required by the in-place version of CRAIGMR.

The outer constructors

solver = CraigmrSolver(m, n, S)
-solver = CraigmrSolver(A, b)

may be used in order to create these vectors.

source
Krylov.GpmrSolver — Type

Type for storing the vectors required by the in-place version of GPMR.

The outer constructors

solver = GpmrSolver(m, n, memory, S)
-solver = GpmrSolver(A, b, memory = 20)

may be used in order to create these vectors. memory is set to n + m if the value given is larger than n + m. memory is an optional argument in the second constructor.

source
Krylov.FgmresSolver — Type

Type for storing the vectors required by the in-place version of FGMRES.

The outer constructors

solver = FgmresSolver(m, n, memory, S)
-solver = FgmresSolver(A, b, memory = 20)

may be used in order to create these vectors. memory is set to n if the value given is larger than n. memory is an optional argument in the second constructor.

source
Krylov.BlockGmresSolver — Type

Type for storing the vectors required by the in-place version of BLOCK-GMRES.

The outer constructors

solver = BlockGmresSolver(m, n, p, memory, SV, SM)
-solver = BlockGmresSolver(A, B, memory = 5)

may be used in order to create these vectors. memory is set to div(n,p) if the value given is larger than div(n,p). memory is an optional argument in the second constructor.

source

Utilities

Krylov.roots_quadratic — Function
roots = roots_quadratic(q₂, q₁, q₀; nitref)

Find the real roots of the quadratic

q(x) = q₂ x² + q₁ x + q₀,

where q₂, q₁ and q₀ are real. Care is taken to avoid numerical cancellation. Optionally, nitref steps of iterative refinement may be performed to improve accuracy. By default, nitref=1.

source
Krylov.sym_givens — Function
(c, s, ρ) = sym_givens(a, b)

Numerically stable symmetric Givens reflection. Given a and b reals, return (c, s, ρ) such that

[ c  s ] [ a ] = [ ρ ]
-[ s -c ] [ b ] = [ 0 ].
source

Numerically stable symmetric Givens reflection. Given a and b complexes, return (c, s, ρ) with c real and (s, ρ) complexes such that

[ c   s ] [ a ] = [ ρ ]
-[ s̅  -c ] [ b ] = [ 0 ].
source
Krylov.to_boundary — Function
roots = to_boundary(n, x, d, radius; flip, xNorm2, dNorm2)

Given a trust-region radius radius, a vector x lying inside the trust-region and a direction d, return σ1 and σ2 such that

‖x + σi d‖ = radius, i = 1, 2

in the Euclidean norm. n is the length of vectors x and d. If known, ‖x‖² and ‖d‖² may be supplied with xNorm2 and dNorm2.

If flip is set to true, σ1 and σ2 are computed such that

‖x - σi d‖ = radius, i = 1, 2.
source
Krylov.vec2str — Function
s = vec2str(x; ndisp)

Display an array in the form

[ -3.0e-01 -5.1e-01  1.9e-01 ... -2.3e-01 -4.4e-01  2.4e-01 ]

with (ndisp - 1)/2 elements on each side.

source
Krylov.ktypeof — Function
S = ktypeof(v)

Return the most relevant storage type S based on the type of v.

source
Krylov.kzeros — Function
v = kzeros(S, n)

Create a vector of storage type S of length n only composed of zero.

source
Krylov.kones — Function
v = kones(S, n)

Create a vector of storage type S of length n only composed of one.

source
Krylov.vector_to_matrix — Function
M = vector_to_matrix(S)

Return the dense matrix storage type M related to the dense vector storage type S.

source
Krylov.matrix_to_vector — Function
S = matrix_to_vector(M)

Return the dense vector storage type S related to the dense matrix storage type M.

source
+API · Krylov.jl
GitHub

Stats Types

Krylov.SimpleStats — Type

Type for statistics returned by the majority of Krylov solvers, the attributes are:

  • niter
  • solved
  • inconsistent
  • residuals
  • Aresiduals
  • Acond
  • timer
  • status
source
Krylov.LanczosStats — Type

Type for statistics returned by CG-LANCZOS, the attributes are:

  • niter
  • solved
  • residuals
  • indefinite
  • Anorm
  • Acond
  • timer
  • status
source
Krylov.LanczosShiftStats — Type

Type for statistics returned by CG-LANCZOS with shifts, the attributes are:

  • niter
  • solved
  • residuals
  • indefinite
  • Anorm
  • Acond
  • timer
  • status
source
Krylov.SymmlqStats — Type

Type for statistics returned by SYMMLQ, the attributes are:

  • niter
  • solved
  • residuals
  • residualscg
  • errors
  • errorscg
  • Anorm
  • Acond
  • timer
  • status
source
Krylov.AdjointStats — Type

Type for statistics returned by adjoint systems solvers BiLQR and TriLQR, the attributes are:

  • niter
  • solved_primal
  • solved_dual
  • residuals_primal
  • residuals_dual
  • timer
  • status
source
Krylov.LNLQStats — Type

Type for statistics returned by the LNLQ method, the attributes are:

  • niter
  • solved
  • residuals
  • errorwithbnd
  • errorbndx
  • errorbndy
  • timer
  • status
source
Krylov.LSLQStats — Type

Type for statistics returned by the LSLQ method, the attributes are:

  • niter
  • solved
  • inconsistent
  • residuals
  • Aresiduals
  • err_lbnds
  • errorwithbnd
  • errubndslq
  • errubndscg
  • timer
  • status
source
Krylov.LsmrStats — Type

Type for statistics returned by LSMR. The attributes are:

  • niter
  • solved
  • inconsistent
  • residuals
  • Aresiduals
  • Acond
  • Anorm
  • xNorm
  • timer
  • status
source

Solver Types

Krylov.MinresSolver — Type

Type for storing the vectors required by the in-place version of MINRES.

The outer constructors

solver = MinresSolver(m, n, S; window :: Int=5)
+solver = MinresSolver(A, b; window :: Int=5)

may be used in order to create these vectors.

source
Krylov.MinaresSolver — Type

Type for storing the vectors required by the in-place version of MINARES.

The outer constructors

solver = MinaresSolver(m, n, S)
+solver = MinaresSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CgSolver — Type

Type for storing the vectors required by the in-place version of CG.

The outer constructors

solver = CgSolver(m, n, S)
+solver = CgSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CrSolver — Type

Type for storing the vectors required by the in-place version of CR.

The outer constructors

solver = CrSolver(m, n, S)
+solver = CrSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CarSolver — Type

Type for storing the vectors required by the in-place version of CAR.

The outer constructors

solver = CarSolver(m, n, S)
+solver = CarSolver(A, b)

may be used in order to create these vectors.

source
Krylov.SymmlqSolver — Type

Type for storing the vectors required by the in-place version of SYMMLQ.

The outer constructors

solver = SymmlqSolver(m, n, S)
+solver = SymmlqSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CgLanczosSolver — Type

Type for storing the vectors required by the in-place version of CG-LANCZOS.

The outer constructors

solver = CgLanczosSolver(m, n, S)
+solver = CgLanczosSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CgLanczosShiftSolver — Type

Type for storing the vectors required by the in-place version of CG-LANCZOS-SHIFT.

The outer constructors

solver = CgLanczosShiftSolver(m, n, nshifts, S)
+solver = CgLanczosShiftSolver(A, b, nshifts)

may be used in order to create these vectors.

source
Krylov.MinresQlpSolver — Type

Type for storing the vectors required by the in-place version of MINRES-QLP.

The outer constructors

solver = MinresQlpSolver(m, n, S)
+solver = MinresQlpSolver(A, b)

may be used in order to create these vectors.

source
Krylov.DiomSolver — Type

Type for storing the vectors required by the in-place version of DIOM.

The outer constructors

solver = DiomSolver(m, n, memory, S)
+solver = DiomSolver(A, b, memory = 20)

may be used in order to create these vectors. memory is set to n if the value given is larger than n. memory is an optional argument in the second constructor.

source
Krylov.FomSolver — Type

Type for storing the vectors required by the in-place version of FOM.

The outer constructors

solver = FomSolver(m, n, memory, S)
+solver = FomSolver(A, b, memory = 20)

may be used in order to create these vectors. memory is set to n if the value given is larger than n. memory is an optional argument in the second constructor.

source
Krylov.DqgmresSolver — Type

Type for storing the vectors required by the in-place version of DQGMRES.

The outer constructors

solver = DqgmresSolver(m, n, memory, S)
+solver = DqgmresSolver(A, b, memory = 20)

may be used in order to create these vectors. memory is set to n if the value given is larger than n. memory is an optional argument in the second constructor.

source
Krylov.GmresSolver — Type

Type for storing the vectors required by the in-place version of GMRES.

The outer constructors

solver = GmresSolver(m, n, memory, S)
+solver = GmresSolver(A, b, memory = 20)

may be used in order to create these vectors. memory is set to n if the value given is larger than n. memory is an optional argument in the second constructor.

source
Krylov.UsymlqSolver — Type

Type for storing the vectors required by the in-place version of USYMLQ.

The outer constructors

solver = UsymlqSolver(m, n, S)
+solver = UsymlqSolver(A, b)

may be used in order to create these vectors.

source
Krylov.UsymqrSolver — Type

Type for storing the vectors required by the in-place version of USYMQR.

The outer constructors

solver = UsymqrSolver(m, n, S)
+solver = UsymqrSolver(A, b)

may be used in order to create these vectors.

source
Krylov.TricgSolver — Type

Type for storing the vectors required by the in-place version of TRICG.

The outer constructors

solver = TricgSolver(m, n, S)
+solver = TricgSolver(A, b)

may be used in order to create these vectors.

source
Krylov.TrimrSolver — Type

Type for storing the vectors required by the in-place version of TRIMR.

The outer constructors

solver = TrimrSolver(m, n, S)
+solver = TrimrSolver(A, b)

may be used in order to create these vectors.

source
Krylov.TrilqrSolver — Type

Type for storing the vectors required by the in-place version of TRILQR.

The outer constructors

solver = TrilqrSolver(m, n, S)
+solver = TrilqrSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CgsSolver — Type

Type for storing the vectors required by the in-place version of CGS.

The outer constructorss

solver = CgsSolver(m, n, S)
+solver = CgsSolver(A, b)

may be used in order to create these vectors.

source
Krylov.BicgstabSolver — Type

Type for storing the vectors required by the in-place version of BICGSTAB.

The outer constructors

solver = BicgstabSolver(m, n, S)
+solver = BicgstabSolver(A, b)

may be used in order to create these vectors.

source
Krylov.BilqSolver — Type

Type for storing the vectors required by the in-place version of BILQ.

The outer constructors

solver = BilqSolver(m, n, S)
+solver = BilqSolver(A, b)

may be used in order to create these vectors.

source
Krylov.QmrSolver — Type

Type for storing the vectors required by the in-place version of QMR.

The outer constructors

solver = QmrSolver(m, n, S)
+solver = QmrSolver(A, b)

may be used in order to create these vectors.

source
Krylov.BilqrSolver — Type

Type for storing the vectors required by the in-place version of BILQR.

The outer constructors

solver = BilqrSolver(m, n, S)
+solver = BilqrSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CglsSolver — Type

Type for storing the vectors required by the in-place version of CGLS.

The outer constructors

solver = CglsSolver(m, n, S)
+solver = CglsSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CglsLanczosShiftSolver — Type

Workspace for the in-place version of CGLS-LANCZOS-SHIFT.

The outer constructors:

solver = CglsLanczosShiftSolver(m, n, nshifts, S)
+solver = CglsLanczosShiftSolver(A, b, nshifts)

can be used to initialize this workspace.

source
Krylov.CrlsSolver — Type

Type for storing the vectors required by the in-place version of CRLS.

The outer constructors

solver = CrlsSolver(m, n, S)
+solver = CrlsSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CgneSolver — Type

Type for storing the vectors required by the in-place version of CGNE.

The outer constructors

solver = CgneSolver(m, n, S)
+solver = CgneSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CrmrSolver — Type

Type for storing the vectors required by the in-place version of CRMR.

The outer constructors

solver = CrmrSolver(m, n, S)
+solver = CrmrSolver(A, b)

may be used in order to create these vectors.

source
Krylov.LslqSolver — Type

Type for storing the vectors required by the in-place version of LSLQ.

The outer constructors

solver = LslqSolver(m, n, S)
+solver = LslqSolver(A, b)

may be used in order to create these vectors.

source
Krylov.LsqrSolver — Type

Type for storing the vectors required by the in-place version of LSQR.

The outer constructors

solver = LsqrSolver(m, n, S)
+solver = LsqrSolver(A, b)

may be used in order to create these vectors.

source
Krylov.LsmrSolver — Type

Type for storing the vectors required by the in-place version of LSMR.

The outer constructors

solver = LsmrSolver(m, n, S)
+solver = LsmrSolver(A, b)

may be used in order to create these vectors.

source
Krylov.LnlqSolver — Type

Type for storing the vectors required by the in-place version of LNLQ.

The outer constructors

solver = LnlqSolver(m, n, S)
+solver = LnlqSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CraigSolver — Type

Type for storing the vectors required by the in-place version of CRAIG.

The outer constructors

solver = CraigSolver(m, n, S)
+solver = CraigSolver(A, b)

may be used in order to create these vectors.

source
Krylov.CraigmrSolver — Type

Type for storing the vectors required by the in-place version of CRAIGMR.

The outer constructors

solver = CraigmrSolver(m, n, S)
+solver = CraigmrSolver(A, b)

may be used in order to create these vectors.

source
Krylov.GpmrSolver — Type

Type for storing the vectors required by the in-place version of GPMR.

The outer constructors

solver = GpmrSolver(m, n, memory, S)
+solver = GpmrSolver(A, b, memory = 20)

may be used in order to create these vectors. memory is set to n + m if the value given is larger than n + m. memory is an optional argument in the second constructor.

source
Krylov.FgmresSolver — Type

Type for storing the vectors required by the in-place version of FGMRES.

The outer constructors

solver = FgmresSolver(m, n, memory, S)
+solver = FgmresSolver(A, b, memory = 20)

may be used in order to create these vectors. memory is set to n if the value given is larger than n. memory is an optional argument in the second constructor.

source
Krylov.BlockGmresSolver — Type

Type for storing the vectors required by the in-place version of BLOCK-GMRES.

The outer constructors

solver = BlockGmresSolver(m, n, p, memory, SV, SM)
+solver = BlockGmresSolver(A, B, memory = 5)

may be used in order to create these vectors. memory is set to div(n,p) if the value given is larger than div(n,p). memory is an optional argument in the second constructor.

source

Utilities

Krylov.roots_quadratic — Function
roots = roots_quadratic(q₂, q₁, q₀; nitref)

Find the real roots of the quadratic

q(x) = q₂ x² + q₁ x + q₀,

where q₂, q₁ and q₀ are real. Care is taken to avoid numerical cancellation. Optionally, nitref steps of iterative refinement may be performed to improve accuracy. By default, nitref=1.

source
Krylov.sym_givens — Function
(c, s, ρ) = sym_givens(a, b)

Numerically stable symmetric Givens reflection. Given a and b reals, return (c, s, ρ) such that

[ c  s ] [ a ] = [ ρ ]
+[ s -c ] [ b ] = [ 0 ].
source

Numerically stable symmetric Givens reflection. Given a and b complexes, return (c, s, ρ) with c real and (s, ρ) complexes such that

[ c   s ] [ a ] = [ ρ ]
+[ s̅  -c ] [ b ] = [ 0 ].
source
Krylov.to_boundary — Function
roots = to_boundary(n, x, d, radius; flip, xNorm2, dNorm2)

Given a trust-region radius radius, a vector x lying inside the trust-region and a direction d, return σ1 and σ2 such that

‖x + σi d‖ = radius, i = 1, 2

in the Euclidean norm. n is the length of vectors x and d. If known, ‖x‖² and ‖d‖² may be supplied with xNorm2 and dNorm2.

If flip is set to true, σ1 and σ2 are computed such that

‖x - σi d‖ = radius, i = 1, 2.
source
Krylov.vec2str — Function
s = vec2str(x; ndisp)

Display an array in the form

[ -3.0e-01 -5.1e-01  1.9e-01 ... -2.3e-01 -4.4e-01  2.4e-01 ]

with (ndisp - 1)/2 elements on each side.

source
Krylov.ktypeof — Function
S = ktypeof(v)

Return the most relevant storage type S based on the type of v.

source
Krylov.kzeros — Function
v = kzeros(S, n)

Create a vector of storage type S of length n only composed of zero.

source
Krylov.kones — Function
v = kones(S, n)

Create a vector of storage type S of length n only composed of one.

source
Krylov.vector_to_matrix — Function
M = vector_to_matrix(S)

Return the dense matrix storage type M related to the dense vector storage type S.

source
Krylov.matrix_to_vector — Function
S = matrix_to_vector(M)

Return the dense vector storage type S related to the dense matrix storage type M.

source
diff --git a/previews/PR911/block_krylov/index.html b/previews/PR911/block_krylov/index.html index 9d23ac120..fa089cd41 100644 --- a/previews/PR911/block_krylov/index.html +++ b/previews/PR911/block_krylov/index.html @@ -18,5 +18,5 @@ restart::Bool=false, reorthogonalization::Bool=false, atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0, timemax::Float64=Inf, verbose::Int=0, history::Bool=false, - callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(X, stats) = block_gmres(A, B, X0::AbstractMatrix; kwargs...)

Block-GMRES can be warm-started from an initial guess X0 where kwargs are the same keyword arguments as above.

Solve the linear system AX = B of size n with p right-hand sides using block-GMRES.

Input arguments

Optional argument

Keyword arguments

Output arguments

source
Krylov.block_gmres! — Function
solver = block_gmres!(solver::BlockGmresSolver, B; kwargs...)
-solver = block_gmres!(solver::BlockGmresSolver, B, X0; kwargs...)

where kwargs are keyword arguments of block_gmres.

See BlockGmresSolver for more details about the solver.

source
+ callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(X, stats) = block_gmres(A, B, X0::AbstractMatrix; kwargs...)

Block-GMRES can be warm-started from an initial guess X0 where kwargs are the same keyword arguments as above.

Solve the linear system AX = B of size n with p right-hand sides using block-GMRES.

Input arguments

Optional argument

Keyword arguments

Output arguments

source
Krylov.block_gmres! — Function
solver = block_gmres!(solver::BlockGmresSolver, B; kwargs...)
+solver = block_gmres!(solver::BlockGmresSolver, B, X0; kwargs...)

where kwargs are keyword arguments of block_gmres.

See BlockGmresSolver for more details about the solver.

source
diff --git a/previews/PR911/block_processes/index.html b/previews/PR911/block_processes/index.html index cfba376fa..a88e59b82 100644 --- a/previews/PR911/block_processes/index.html +++ b/previews/PR911/block_processes/index.html @@ -9,7 +9,7 @@ & \ddots & \ddots & \Psi_k^H \\ & & \Psi_k & \Omega_k \\ & & & \Psi_{k+1} -\end{bmatrix}.\]

The function hermitian_lanczos returns $V_{k+1}$, $\Psi_1$ and $T_{k+1,k}$.

Krylov.hermitian_lanczos — Method
V, Ψ, T = hermitian_lanczos(A, B, k; algo="householder")

Input arguments

  • A: a linear operator that models an Hermitian matrix of dimension n;
  • B: a matrix of size n × p;
  • k: the number of iterations of the block Hermitian Lanczos process.

Keyword arguments

  • algo: the algorithm to perform reduced QR factorizations ("gs", "mgs", "givens" or "householder").

Output arguments

  • V: a dense n × p(k+1) matrix;
  • Ψ: a dense p × p upper triangular matrix such that V₁Ψ = B;
  • T: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p.
source

Block Non-Hermitian Lanczos

If the vectors $b$ and $c$ in the non-Hermitian Lanczos process are replaced by matrices $B$ and $C$ with both $p$ columns, we can derive the block non-Hermitian Lanczos process.

block_nonhermitian_lanczos

After $k$ iterations of the block non-Hermitian Lanczos process, the situation may be summarized as

\[\begin{align*} +\end{bmatrix}.\]

The function hermitian_lanczos returns $V_{k+1}$, $\Psi_1$ and $T_{k+1,k}$.

Krylov.hermitian_lanczos — Method
V, Ψ, T = hermitian_lanczos(A, B, k; algo="householder")

Input arguments

  • A: a linear operator that models an Hermitian matrix of dimension n;
  • B: a matrix of size n × p;
  • k: the number of iterations of the block Hermitian Lanczos process.

Keyword arguments

  • algo: the algorithm to perform reduced QR factorizations ("gs", "mgs", "givens" or "householder").

Output arguments

  • V: a dense n × p(k+1) matrix;
  • Ψ: a dense p × p upper triangular matrix such that V₁Ψ = B;
  • T: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p.
source

Block Non-Hermitian Lanczos

If the vectors $b$ and $c$ in the non-Hermitian Lanczos process are replaced by matrices $B$ and $C$ with both $p$ columns, we can derive the block non-Hermitian Lanczos process.

block_nonhermitian_lanczos

After $k$ iterations of the block non-Hermitian Lanczos process, the situation may be summarized as

\[\begin{align*} A V_k &= V_{k+1} T_{k+1,k}, \\ A^H U_k &= U_{k+1} T_{k,k+1}^H, \\ V_k^H U_k &= U_k^H V_k = I_{pk}, @@ -29,7 +29,7 @@ & \ddots & \ddots & \Psi_k^H \\ & & \Phi_k^H & \Omega_k^H \\ & & & \Phi_{k+1}^H -\end{bmatrix}.\]

The function nonhermitian_lanczos returns $V_{k+1}$, $\Psi_1$, $T_{k+1,k}$, $U_{k+1}$ $\Phi_1^H$ and $T_{k,k+1}^H$.

Krylov.nonhermitian_lanczos — Method
V, Ψ, T, U, Φᴴ, Tᴴ = nonhermitian_lanczos(A, B, C, k)

Input arguments

  • A: a linear operator that models a square matrix of dimension n;
  • B: a matrix of size n × p;
  • C: a matrix of size n × p;
  • k: the number of iterations of the block non-Hermitian Lanczos process.

Output arguments

  • V: a dense n × p(k+1) matrix;
  • Ψ: a dense p × p upper triangular matrix such that V₁Ψ = B;
  • T: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p;
  • U: a dense n × p(k+1) matrix;
  • Φᴴ: a dense p × p upper triangular matrix such that U₁Φᴴ = C;
  • Tᴴ: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p.
source

Block Arnoldi

If the vector $b$ in the Arnoldi process is replaced by a matrix $B$ with $p$ columns, we can derive the block Arnoldi process.

block_arnoldi

After $k$ iterations of the block Arnoldi process, the situation may be summarized as

\[\begin{align*} +\end{bmatrix}.\]

The function nonhermitian_lanczos returns $V_{k+1}$, $\Psi_1$, $T_{k+1,k}$, $U_{k+1}$ $\Phi_1^H$ and $T_{k,k+1}^H$.

Krylov.nonhermitian_lanczos — Method
V, Ψ, T, U, Φᴴ, Tᴴ = nonhermitian_lanczos(A, B, C, k)

Input arguments

  • A: a linear operator that models a square matrix of dimension n;
  • B: a matrix of size n × p;
  • C: a matrix of size n × p;
  • k: the number of iterations of the block non-Hermitian Lanczos process.

Output arguments

  • V: a dense n × p(k+1) matrix;
  • Ψ: a dense p × p upper triangular matrix such that V₁Ψ = B;
  • T: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p;
  • U: a dense n × p(k+1) matrix;
  • Φᴴ: a dense p × p upper triangular matrix such that U₁Φᴴ = C;
  • Tᴴ: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p.
source

Block Arnoldi

If the vector $b$ in the Arnoldi process is replaced by a matrix $B$ with $p$ columns, we can derive the block Arnoldi process.

block_arnoldi

After $k$ iterations of the block Arnoldi process, the situation may be summarized as

\[\begin{align*} A V_k &= V_{k+1} H_{k+1,k}, \\ V_k^H V_k &= I_{pk}, \end{align*}\]

where $V_k$ is an orthonormal basis of the block Krylov subspace $\mathcal{K}_k^{\square}(A,B)$,

\[H_{k+1,k} = @@ -39,7 +39,7 @@ & \ddots~ & \ddots & \Psi_{k-1,k} \\ & & \Psi_{k,k-1} & \Psi_{k,k} \\ & & & \Psi_{k+1,k} -\end{bmatrix}.\]

The function arnoldi returns $V_{k+1}$, $\Gamma$, and $H_{k+1,k}$.

Related method: BLOCK-GMRES.

Krylov.arnoldi — Method
V, Γ, H = arnoldi(A, B, k; algo="householder", reorthogonalization=false)

Input arguments

  • A: a linear operator that models a square matrix of dimension n;
  • B: a matrix of size n × p;
  • k: the number of iterations of the block Arnoldi process.

Keyword arguments

  • algo: the algorithm to perform reduced QR factorizations ("gs", "mgs", "givens" or "householder").
  • reorthogonalization: reorthogonalize the new matrices of the block Krylov basis against all previous matrices.

Output arguments

  • V: a dense n × p(k+1) matrix;
  • Γ: a dense p × p upper triangular matrix such that V₁Γ = B;
  • H: a dense p(k+1) × pk block upper Hessenberg matrix with a lower bandwidth p.
source

Block Golub-Kahan

If the vector $b$ in the Golub-Kahan process is replaced by a matrix $B$ with $p$ columns, we can derive the block Golub-Kahan process.

block_golub_kahan

After $k$ iterations of the block Golub-Kahan process, the situation may be summarized as

\[\begin{align*} +\end{bmatrix}.\]

The function arnoldi returns $V_{k+1}$, $\Gamma$, and $H_{k+1,k}$.

Related method: BLOCK-GMRES.

Krylov.arnoldi — Method
V, Γ, H = arnoldi(A, B, k; algo="householder", reorthogonalization=false)

Input arguments

  • A: a linear operator that models a square matrix of dimension n;
  • B: a matrix of size n × p;
  • k: the number of iterations of the block Arnoldi process.

Keyword arguments

  • algo: the algorithm to perform reduced QR factorizations ("gs", "mgs", "givens" or "householder").
  • reorthogonalization: reorthogonalize the new matrices of the block Krylov basis against all previous matrices.

Output arguments

  • V: a dense n × p(k+1) matrix;
  • Γ: a dense p × p upper triangular matrix such that V₁Γ = B;
  • H: a dense p(k+1) × pk block upper Hessenberg matrix with a lower bandwidth p.
source

Block Golub-Kahan

If the vector $b$ in the Golub-Kahan process is replaced by a matrix $B$ with $p$ columns, we can derive the block Golub-Kahan process.

block_golub_kahan

After $k$ iterations of the block Golub-Kahan process, the situation may be summarized as

\[\begin{align*} A V_k &= U_{k+1} B_k, \\ A^H U_{k+1} &= V_{k+1} L_{k+1}^H, \\ V_k^H V_k &= U_k^H U_k = I_{pk}, @@ -59,7 +59,7 @@ & & \ddots & \Psi_k^H & \\ & & & \Omega_k^H & \Psi_{k+1}^H \\ & & & & \Omega_{k+1}^H \\ -\end{bmatrix}.\]

The function golub_kahan returns $V_{k+1}$, $U_{k+1}$, $\Psi_1$ and $L_{k+1}$.

Krylov.golub_kahan — Method
V, U, Ψ, L = golub_kahan(A, B, k; algo="householder")

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • B: a matrix of size m × p;
  • k: the number of iterations of the block Golub-Kahan process.

Keyword argument

  • algo: the algorithm to perform reduced QR factorizations ("gs", "mgs", "givens" or "householder").

Output arguments

  • V: a dense n × p(k+1) matrix;
  • U: a dense m × p(k+1) matrix;
  • Ψ: a dense p × p upper triangular matrix such that U₁Ψ = B;
  • L: a sparse p(k+1) × p(k+1) block lower bidiagonal matrix with a lower bandwidth p.
source

Block Saunders-Simon-Yip

If the vectors $b$ and $c$ in the Saunders-Simon-Yip process are replaced by matrices $B$ and $C$ with both $p$ columns, we can derive the block Saunders-Simon-Yip process.

block_saunders_simon_yip

After $k$ iterations of the block Saunders-Simon-Yip process, the situation may be summarized as

\[\begin{align*} +\end{bmatrix}.\]

The function golub_kahan returns $V_{k+1}$, $U_{k+1}$, $\Psi_1$ and $L_{k+1}$.

Krylov.golub_kahan — Method
V, U, Ψ, L = golub_kahan(A, B, k; algo="householder")

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • B: a matrix of size m × p;
  • k: the number of iterations of the block Golub-Kahan process.

Keyword argument

  • algo: the algorithm to perform reduced QR factorizations ("gs", "mgs", "givens" or "householder").

Output arguments

  • V: a dense n × p(k+1) matrix;
  • U: a dense m × p(k+1) matrix;
  • Ψ: a dense p × p upper triangular matrix such that U₁Ψ = B;
  • L: a sparse p(k+1) × p(k+1) block lower bidiagonal matrix with a lower bandwidth p.
source

Block Saunders-Simon-Yip

If the vectors $b$ and $c$ in the Saunders-Simon-Yip process are replaced by matrices $B$ and $C$ with both $p$ columns, we can derive the block Saunders-Simon-Yip process.

block_saunders_simon_yip

After $k$ iterations of the block Saunders-Simon-Yip process, the situation may be summarized as

\[\begin{align*} A U_k &= V_{k+1} T_{k+1,k}, \\ A^H V_k &= U_{k+1} T_{k,k+1}^H, \\ V_k^H V_k &= U_k^H U_k = I_{pk}, @@ -79,7 +79,7 @@ & \ddots & \ddots & \Psi_k^H \\ & & \Phi_k^H & \Omega_k^H \\ & & & \Phi_{k+1}^H -\end{bmatrix}.\]

The function saunders_simon_yip returns $V_{k+1}$, $\Psi_1$, $T_{k+1,k}$, $U_{k+1}$, $\Phi_1^H$ and $T_{k,k+1}^H$.

Krylov.saunders_simon_yip — Method
V, Ψ, T, U, Φᴴ, Tᴴ = saunders_simon_yip(A, B, C, k; algo="householder")

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • B: a matrix of size m × p;
  • C: a matrix of size n × p;
  • k: the number of iterations of the block Saunders-Simon-Yip process.

Keyword argument

  • algo: the algorithm to perform reduced QR factorizations ("gs", "mgs", "givens" or "householder").

Output arguments

  • V: a dense m × p(k+1) matrix;
  • Ψ: a dense p × p upper triangular matrix such that V₁Ψ = B;
  • T: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p;
  • U: a dense n × p(k+1) matrix;
  • Φᴴ: a dense p × p upper triangular matrix such that U₁Φᴴ = C;
  • Tᴴ: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p.
source

Block Montoison-Orban

If the vectors $b$ and $c$ in the Montoison-Orban process are replaced by matrices $D$ and $C$ with both $p$ columns, we can derive the block Montoison-Orban process.

block_montoison_orban

After $k$ iterations of the block Montoison-Orban process, the situation may be summarized as

\[\begin{align*} +\end{bmatrix}.\]

The function saunders_simon_yip returns $V_{k+1}$, $\Psi_1$, $T_{k+1,k}$, $U_{k+1}$, $\Phi_1^H$ and $T_{k,k+1}^H$.

Krylov.saunders_simon_yip — Method
V, Ψ, T, U, Φᴴ, Tᴴ = saunders_simon_yip(A, B, C, k; algo="householder")

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • B: a matrix of size m × p;
  • C: a matrix of size n × p;
  • k: the number of iterations of the block Saunders-Simon-Yip process.

Keyword argument

  • algo: the algorithm to perform reduced QR factorizations ("gs", "mgs", "givens" or "householder").

Output arguments

  • V: a dense m × p(k+1) matrix;
  • Ψ: a dense p × p upper triangular matrix such that V₁Ψ = B;
  • T: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p;
  • U: a dense n × p(k+1) matrix;
  • Φᴴ: a dense p × p upper triangular matrix such that U₁Φᴴ = C;
  • Tᴴ: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p.
source

Block Montoison-Orban

If the vectors $b$ and $c$ in the Montoison-Orban process are replaced by matrices $D$ and $C$ with both $p$ columns, we can derive the block Montoison-Orban process.

block_montoison_orban

After $k$ iterations of the block Montoison-Orban process, the situation may be summarized as

\[\begin{align*} A U_k &= V_{k+1} H_{k+1,k}, \\ B V_k &= U_{k+1} F_{k+1,k}, \\ V_k^H V_k &= U_k^H U_k = I_{pk}, @@ -99,4 +99,4 @@ & \ddots~ & \ddots & \Phi_{k-1,k} \\ & & \Phi_{k,k-1} & \Phi_{k,k} \\ & & & \Phi_{k+1,k} -\end{bmatrix}.\]

The function montoison_orban returns $V_{k+1}$, $\Gamma$, $H_{k+1,k}$, $U_{k+1}$, $\Lambda$, and $F_{k+1,k}$.

Krylov.montoison_orban — Method
V, Γ, H, U, Λ, F = montoison_orban(A, B, D, C, k; algo="householder", reorthogonalization=false)

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • B: a linear operator that models a matrix of dimension n × m;
  • D: a matrix of size m × p;
  • C: a matrix of size n × p;
  • k: the number of iterations of the block Montoison-Orban process.

Keyword arguments

  • algo: the algorithm to perform reduced QR factorizations ("gs", "mgs", "givens" or "householder").
  • reorthogonalization: reorthogonalize the new matrices of the block Krylov basis against all previous matrices.

Output arguments

  • V: a dense m × p(k+1) matrix;
  • Γ: a dense p × p upper triangular matrix such that V₁Γ = D;
  • H: a dense p(k+1) × pk block upper Hessenberg matrix with a lower bandwidth p;
  • U: a dense n × p(k+1) matrix;
  • Λ: a dense p × p upper triangular matrix such that U₁Λ = C;
  • F: a dense p(k+1) × pk block upper Hessenberg matrix with a lower bandwidth p.
source
+\end{bmatrix}.\]

The function montoison_orban returns $V_{k+1}$, $\Gamma$, $H_{k+1,k}$, $U_{k+1}$, $\Lambda$, and $F_{k+1,k}$.

Krylov.montoison_orban — Method
V, Γ, H, U, Λ, F = montoison_orban(A, B, D, C, k; algo="householder", reorthogonalization=false)

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • B: a linear operator that models a matrix of dimension n × m;
  • D: a matrix of size m × p;
  • C: a matrix of size n × p;
  • k: the number of iterations of the block Montoison-Orban process.

Keyword arguments

  • algo: the algorithm to perform reduced QR factorizations ("gs", "mgs", "givens" or "householder").
  • reorthogonalization: reorthogonalize the new matrices of the block Krylov basis against all previous matrices.

Output arguments

  • V: a dense m × p(k+1) matrix;
  • Γ: a dense p × p upper triangular matrix such that V₁Γ = D;
  • H: a dense p(k+1) × pk block upper Hessenberg matrix with a lower bandwidth p;
  • U: a dense n × p(k+1) matrix;
  • Λ: a dense p × p upper triangular matrix such that U₁Λ = C;
  • F: a dense p(k+1) × pk block upper Hessenberg matrix with a lower bandwidth p.
source
diff --git a/previews/PR911/callbacks/index.html b/previews/PR911/callbacks/index.html index ce0c5ae02..221970c85 100644 --- a/previews/PR911/callbacks/index.html +++ b/previews/PR911/callbacks/index.html @@ -50,4 +50,4 @@ return false # We don't want to add new stopping conditions end -(x, stats) = gmres(A, b, callback = gmres_callback) +(x, stats) = gmres(A, b, callback = gmres_callback) diff --git a/previews/PR911/examples/bicgstab/index.html b/previews/PR911/examples/bicgstab/index.html index e3319e995..fbe13c3ef 100644 --- a/previews/PR911/examples/bicgstab/index.html +++ b/previews/PR911/examples/bicgstab/index.html @@ -51,4 +51,4 @@ [Left preconditioning] Residual norm: 3.2e-07 [Left preconditioning] Number of iterations: 9 [Right preconditioning] Residual norm: 7.8e-06 -[Right preconditioning] Number of iterations: 8 +[Right preconditioning] Number of iterations: 8 diff --git a/previews/PR911/examples/block_gmres/index.html b/previews/PR911/examples/block_gmres/index.html index 728792d11..7d0c3ce38 100644 --- a/previews/PR911/examples/block_gmres/index.html +++ b/previews/PR911/examples/block_gmres/index.html @@ -23,6 +23,6 @@ residuals: [] Aresiduals: [] κ₂(A): [] - timer: 262.32ms + timer: 265.17ms status: solution good enough given atol and rtol -Relative residual: 6.8e-09 +Relative residual: 6.8e-09 diff --git a/previews/PR911/examples/car/index.html b/previews/PR911/examples/car/index.html index e735cb99c..3dd438ebb 100644 --- a/previews/PR911/examples/car/index.html +++ b/previews/PR911/examples/car/index.html @@ -23,6 +23,6 @@ residuals: [] Aresiduals: [] κ₂(A): [] - timer: 18.66ms + timer: 18.93ms status: solution good enough given atol and rtol -Relative residual: 1.5e-08 +Relative residual: 1.5e-08 diff --git a/previews/PR911/examples/cg/index.html b/previews/PR911/examples/cg/index.html index 9d41631af..4c1367657 100644 --- a/previews/PR911/examples/cg/index.html +++ b/previews/PR911/examples/cg/index.html @@ -23,6 +23,6 @@ residuals: [] Aresiduals: [] κ₂(A): [] - timer: 2.85ms + timer: 2.77ms status: solution good enough given atol and rtol -Relative residual: 1.2e-08 +Relative residual: 1.2e-08 diff --git a/previews/PR911/examples/cg_lanczos_shift/index.html b/previews/PR911/examples/cg_lanczos_shift/index.html index be1e4e132..0fbc9cd3c 100644 --- a/previews/PR911/examples/cg_lanczos_shift/index.html +++ b/previews/PR911/examples/cg_lanczos_shift/index.html @@ -33,7 +33,7 @@ indefinite: Bool[0, 1, 1, 1] ‖A‖F: NaN κ₂(A): NaN - timer: 6.19ms + timer: 6.16ms status: solution good enough given atol and rtol Relative residuals with shifts: - 1.3e-08 1.1e-08 1.3e-08 1.5e-08 + 1.3e-08 1.1e-08 1.3e-08 1.5e-08 diff --git a/previews/PR911/examples/cgls/index.html b/previews/PR911/examples/cgls/index.html index 62b3d82cd..301007d26 100644 --- a/previews/PR911/examples/cgls/index.html +++ b/previews/PR911/examples/cgls/index.html @@ -28,7 +28,7 @@ residuals: [] Aresiduals: [] κ₂(A): [] - timer: 1.01ms + timer: 993.69μs status: solution good enough given atol and rtol CGLS: Relative residual: 1.8e-08 -CGLS: ‖x‖: 8.4e+03 +CGLS: ‖x‖: 8.4e+03 diff --git a/previews/PR911/examples/cgls_lanczos_shift/index.html b/previews/PR911/examples/cgls_lanczos_shift/index.html index 31835992e..8e07c93b3 100644 --- a/previews/PR911/examples/cgls_lanczos_shift/index.html +++ b/previews/PR911/examples/cgls_lanczos_shift/index.html @@ -37,7 +37,7 @@ indefinite: Bool[0, 0, 0, 0] ‖A‖F: NaN κ₂(A): NaN - timer: 39.23ms + timer: 40.07ms status: solution good enough given atol and rtol CGLS: Relative residuals with shifts: - 1.3e-08 6.5e-09 1.0e-08 1.1e-08 + 1.3e-08 6.5e-09 1.0e-08 1.1e-08 diff --git a/previews/PR911/examples/cgne/index.html b/previews/PR911/examples/cgne/index.html index 90dc560a3..1dc8a5ff2 100644 --- a/previews/PR911/examples/cgne/index.html +++ b/previews/PR911/examples/cgne/index.html @@ -27,7 +27,7 @@ residuals: [] Aresiduals: [] κ₂(A): [] - timer: 991.03μs + timer: 986.31μs status: solution good enough given atol and rtol CGNE: Relative residual: 1.5e-08 -CGNE: ‖x - x*‖₂: 3.0e-07 +CGNE: ‖x - x*‖₂: 3.0e-07 diff --git a/previews/PR911/examples/craig/index.html b/previews/PR911/examples/craig/index.html index 59a0a037a..711b83f89 100644 --- a/previews/PR911/examples/craig/index.html +++ b/previews/PR911/examples/craig/index.html @@ -21,12 +21,12 @@ @printf("Error in y: %7.1e\n", norm(λ * y - xy_exact[n+1:n+m]) / norm(xy_exact[n+1:n+m])) end
CRAIG: system of 5 equations in 8 variables
     k       ‖r‖       ‖x‖       ‖A‖      κ(A)         α        β  timer
-    0  9.90e+00  0.00e+00  0.00e+00  0.00e+00   ✗ ✗ ✗ ✗  ✗ ✗ ✗ ✗  0.49s
-    1  3.27e-01  2.80e+00  3.53e+00  3.53e+00   3.5e+00  1.2e-01  0.92s
-    2  4.51e-02  2.82e+00  3.66e+00  5.18e+00   9.6e-01  1.3e-01  1.10s
-    3  6.92e-03  2.83e+00  3.68e+00  6.38e+00   3.4e-01  5.2e-02  1.10s
-    4  7.22e-03  2.83e+00  3.78e+00  7.58e+00   6.0e-01  6.3e-01  1.10s
-    5  1.50e-09  2.83e+00  3.87e+00  9.05e+00   8.2e-01  1.7e-07  1.10s
+    0  8.78e+00  0.00e+00  0.00e+00  0.00e+00   ✗ ✗ ✗ ✗  ✗ ✗ ✗ ✗  0.51s
+    1  2.49e-01  2.78e+00  3.16e+00  3.16e+00   3.2e+00  9.0e-02  0.95s
+    2  5.54e-02  2.79e+00  3.33e+00  4.70e+00   1.0e+00  2.3e-01  1.13s
+    3  5.82e-02  2.80e+00  3.35e+00  5.82e+00   2.7e-01  2.8e-01  1.13s
+    4  1.73e-02  2.80e+00  3.42e+00  7.24e+00   6.5e-01  1.9e-01  1.13s
+    5  1.29e-09  2.80e+00  3.47e+00  8.19e+00   5.9e-01  4.4e-08  1.13s
 
 SimpleStats
  niter: 5
@@ -35,9 +35,9 @@
  residuals: []
  Aresiduals: []
  κ₂(A): []
- timer: 1.10s
+ timer: 1.13s
  status: solution good enough for the tolerances given
 Primal feasibility: 1.5e-10
-Dual   feasibility: 2.0e-16
+Dual   feasibility: 2.4e-16
 Error in x: 1.5e-10
-Error in y: 1.2e-10
+Error in y: 9.4e-11 diff --git a/previews/PR911/examples/craigmr/index.html b/previews/PR911/examples/craigmr/index.html index e52fe5e78..3ec4cd6fd 100644 --- a/previews/PR911/examples/craigmr/index.html +++ b/previews/PR911/examples/craigmr/index.html @@ -28,8 +28,8 @@ residuals: [] Aresiduals: [] κ₂(A): [] - timer: 642.43μs + timer: 632.71μs status: found approximate minimum-norm solution CRAIGMR: Relative residual: 1.3e-08 CRAIGMR: ‖x - x*‖₂: 1.4e-05 -CRAIGMR: 0 iterations +CRAIGMR: 0 iterations diff --git a/previews/PR911/examples/crls/index.html b/previews/PR911/examples/crls/index.html index 4eae29ee3..366947c16 100644 --- a/previews/PR911/examples/crls/index.html +++ b/previews/PR911/examples/crls/index.html @@ -28,7 +28,7 @@ residuals: [] Aresiduals: [] κ₂(A): [] - timer: 5.21ms + timer: 4.14ms status: solution good enough given atol and rtol CRLS: Relative residual: 2.1e-08 -CRLS: ‖x‖: 1.0e+04 +CRLS: ‖x‖: 1.0e+04 diff --git a/previews/PR911/examples/crmr/index.html b/previews/PR911/examples/crmr/index.html index 81bb01892..4453af955 100644 --- a/previews/PR911/examples/crmr/index.html +++ b/previews/PR911/examples/crmr/index.html @@ -29,7 +29,7 @@ residuals: [] Aresiduals: [] κ₂(A): [] - timer: 937.29μs + timer: 1.27ms status: system probably inconsistent but least squares/norm solution found CRMR: Relative residual: 4.0e-06 -CRMR: ‖x - x*‖₂: 6.3e-03 +CRMR: ‖x - x*‖₂: 6.3e-03 diff --git a/previews/PR911/examples/dqgmres/index.html b/previews/PR911/examples/dqgmres/index.html index 723511a30..199141ef7 100644 --- a/previews/PR911/examples/dqgmres/index.html +++ b/previews/PR911/examples/dqgmres/index.html @@ -51,4 +51,4 @@ [Left preconditioning] Residual norm: 7.6e-08 [Left preconditioning] Number of iterations: 34 [Right preconditioning] Residual norm: 3.4e-07 -[Right preconditioning] Number of iterations: 33 +[Right preconditioning] Number of iterations: 33 diff --git a/previews/PR911/examples/lsmr/index.html b/previews/PR911/examples/lsmr/index.html index 83e27036d..c2da15238 100644 --- a/previews/PR911/examples/lsmr/index.html +++ b/previews/PR911/examples/lsmr/index.html @@ -32,7 +32,7 @@ κ₂(A): 65.3013701595396 ‖A‖F: 45.007607017379236 xNorm: 16060.87554733281 - timer: 19.19ms + timer: 19.27ms status: truncated forward error small enough LSMR: Relative residual: 2.4e-03 -LSMR: ‖x‖: 1.6e+04 +LSMR: ‖x‖: 1.6e+04 diff --git a/previews/PR911/examples/lsqr/index.html b/previews/PR911/examples/lsqr/index.html index 0735d7ab0..03fb99dce 100644 --- a/previews/PR911/examples/lsqr/index.html +++ b/previews/PR911/examples/lsqr/index.html @@ -28,7 +28,7 @@ residuals: [] Aresiduals: [] κ₂(A): [] - timer: 12.19ms + timer: 12.11ms status: maximum number of iterations exceeded LSQR: Relative residual: 1.4e-03 -LSQR: ‖x‖: 9.4e+03 +LSQR: ‖x‖: 9.4e+03 diff --git a/previews/PR911/examples/minares/index.html b/previews/PR911/examples/minares/index.html index 420a7835c..078ff4f06 100644 --- a/previews/PR911/examples/minares/index.html +++ b/previews/PR911/examples/minares/index.html @@ -23,6 +23,6 @@ residuals: [] Aresiduals: [] κ₂(A): [] - timer: 904.58μs + timer: 902.70μs status: solution good enough given atol, rtol and Artol -Relative A-residual: 7.8e-09 +Relative A-residual: 7.8e-09 diff --git a/previews/PR911/examples/minres_qlp/index.html b/previews/PR911/examples/minres_qlp/index.html index 85f9f6030..b26077fe5 100644 --- a/previews/PR911/examples/minres_qlp/index.html +++ b/previews/PR911/examples/minres_qlp/index.html @@ -17,4 +17,4 @@ @printf("Minimum-norm solution? %s\n", x ≈ [1.0; 1.0; 1.0; 0.0])
Residual r: [-4.9e-15  2.4e-15  4.4e-16  4.0e+00 ]
 Relative residual norm ‖r‖:  7.3e-01
 Solution x: [ 1.0e+00  1.0e+00  1.0e+00 -2.0e-15 ]
-Minimum-norm solution? true
+Minimum-norm solution? true diff --git a/previews/PR911/examples/symmlq/index.html b/previews/PR911/examples/symmlq/index.html index dc65b2ea1..d2f1913d5 100644 --- a/previews/PR911/examples/symmlq/index.html +++ b/previews/PR911/examples/symmlq/index.html @@ -17,4 +17,4 @@ @printf("Minimum-norm solution? %s\n", x ≈ [1.0; 1.0; 1.0; 0.0])
Residual r: [ 0.0e+00  4.4e-16  0.0e+00  0.0e+00 ]
 Relative residual norm ‖r‖:  1.2e-16
 Solution x: [ 1.0e+00  1.0e+00  1.0e+00  0.0e+00 ]
-Minimum-norm solution? true
+Minimum-norm solution? true diff --git a/previews/PR911/examples/tricg/index.html b/previews/PR911/examples/tricg/index.html index 5346ce51c..906726468 100644 --- a/previews/PR911/examples/tricg/index.html +++ b/previews/PR911/examples/tricg/index.html @@ -75,10 +75,10 @@ TriCG: system of 10 equations in 10 variables k ‖rₖ‖ βₖ₊₁ γₖ₊₁ timer 0 1.1e+01 6.7e+00 8.7e+00 0.30s - 1 5.8e+00 2.6e+02 3.0e+02 0.46s - 2 2.5e+00 2.9e+02 2.2e+02 0.46s - 3 1.2e+00 2.4e+01 5.5e+01 0.46s - 4 1.9e-01 3.4e-01 8.1e-01 0.46s - 5 1.2e-08 6.7e-08 7.7e-08 0.46s + 1 5.8e+00 2.6e+02 3.0e+02 0.45s + 2 2.5e+00 2.9e+02 2.2e+02 0.45s + 3 1.2e+00 2.4e+01 5.5e+01 0.45s + 4 1.9e-01 3.4e-01 8.1e-01 0.45s + 5 1.2e-08 6.7e-08 7.7e-08 0.45s -TriCG: Relative residual: 1.2e-08 +TriCG: Relative residual: 1.2e-08 diff --git a/previews/PR911/examples/trimr/index.html b/previews/PR911/examples/trimr/index.html index 3395c5f4b..f7e968704 100644 --- a/previews/PR911/examples/trimr/index.html +++ b/previews/PR911/examples/trimr/index.html @@ -58,11 +58,11 @@ TriMR: Relative residual: 2.7e-11 TriMR: system of 10 equations in 10 variables k ‖rₖ‖ βₖ₊₁ γₖ₊₁ timer - 0 1.1e+00 8.7e-01 6.8e-01 0.19s - 1 7.3e-01 4.0e+00 8.8e-01 0.19s - 2 4.4e-01 2.2e+00 2.0e+00 0.19s - 3 3.2e-01 9.3e-02 1.4e+00 0.19s - 4 2.2e-03 1.0e-02 7.0e-03 0.19s - 5 1.2e-13 3.5e-10 1.9e-11 0.19s + 0 1.1e+00 8.7e-01 6.8e-01 0.20s + 1 7.3e-01 4.0e+00 8.8e-01 0.20s + 2 4.4e-01 2.2e+00 2.0e+00 0.20s + 3 3.2e-01 9.3e-02 1.4e+00 0.20s + 4 2.2e-03 1.0e-02 7.0e-03 0.20s + 5 1.2e-13 3.5e-10 1.9e-11 0.20s -TriMR: Relative residual: 1.2e-13 +TriMR: Relative residual: 1.2e-13 diff --git a/previews/PR911/gpu/index.html b/previews/PR911/gpu/index.html index d5c898719..435720479 100644 --- a/previews/PR911/gpu/index.html +++ b/previews/PR911/gpu/index.html @@ -180,4 +180,4 @@ b_gpu = MtlVector(b_cpu) # Solve a dense least-norm problem on an Apple M1 GPU -x, stats = craig(A_gpu, b_gpu)
Warning

Metal.jl is under heavy development and is considered experimental for now.

+x, stats = craig(A_gpu, b_gpu)
Warning

Metal.jl is under heavy development and is considered experimental for now.

diff --git a/previews/PR911/index.html b/previews/PR911/index.html index a4a8febec..5bd46a15a 100644 --- a/previews/PR911/index.html +++ b/previews/PR911/index.html @@ -1,4 +1,4 @@ Home · Krylov.jl
GitHub

Krylov.jl documentation

This package provides implementations of certain of the most useful Krylov method for a variety of problems:

1 - Square or rectangular full-rank systems

\[ Ax = b\]

should be solved when b lies in the range space of A. This situation occurs when

  • A is square and nonsingular,
  • A is tall and has full column rank and b lies in the range of A.

2 - Linear least-squares problems

\[ \min \|b - Ax\|\]

should be solved when b is not in the range of A (inconsistent systems), regardless of the shape and rank of A. This situation mainly occurs when

  • A is square and singular,
  • A is tall and thin.

Underdetermined systems are less common but also occur.

If there are infinitely many such x (because A is column rank-deficient), one with minimum norm is identified

\[ \min \|x\| \quad \text{subject to} \quad x \in \argmin \|b - Ax\|.\]

3 - Linear least-norm problems

\[ \min \|x\| \quad \text{subject to} \quad Ax = b\]

should be solved when A is column rank-deficient but b is in the range of A (consistent systems), regardless of the shape of A. This situation mainly occurs when

  • A is square and singular,
  • A is short and wide.

Overdetermined systems are less common but also occur.

4 - Adjoint systems

\[ Ax = b \quad \text{and} \quad A^H y = c\]

where A can have any shape.

5 - Saddle-point and Hermitian quasi-definite systems

\[ \begin{bmatrix} M & \phantom{-}A \\ A^H & -N \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \left(\begin{bmatrix} b \\ 0 \end{bmatrix},\begin{bmatrix} 0 \\ c \end{bmatrix},\begin{bmatrix} b \\ c \end{bmatrix}\right)\]

where A can have any shape.

6 - Generalized saddle-point and non-Hermitian partitioned systems

\[ \begin{bmatrix} M & A \\ B & N \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} b \\ c \end{bmatrix}\]

where A can have any shape and B has the shape of Aᴴ. A, B, b and c must be all nonzero.

Krylov solvers are particularly appropriate in situations where such problems must be solved but a factorization is not possible, either because:

  • A is not available explicitly,
  • A would be dense or would consume an excessive amount of memory if it were materialized,
  • factors would consume an excessive amount of memory.

Iterative methods are recommended in either of the following situations:

  • the problem is sufficiently large that a factorization is not feasible or would be slow,
  • an effective preconditioner is known in cases where the problem has unfavorable spectral structure,
  • the operator can be represented efficiently as a sparse matrix,
  • the operator is fast, i.e., can be applied with better complexity than if it were materialized as a matrix. Certain fast operators would materialize as dense matrices.

Features

All solvers in Krylov.jl have in-place version, are compatible with GPU and work in any floating-point data type.

How to Install

Krylov can be installed and tested through the Julia package manager:

julia> ]
 pkg> add Krylov
-pkg> test Krylov

Bug reports and discussions

If you think you found a bug, feel free to open an issue. Focused suggestions and requests can also be opened as issues. Before opening a pull request, start an issue or a discussion on the topic, please.

If you want to ask a question not suited for a bug report, feel free to start a discussion here. This forum is for general discussion about this repository and the JuliaSmoothOptimizers organization, so questions about any of our packages are welcome.

+pkg> test Krylov

Bug reports and discussions

If you think you found a bug, feel free to open an issue. Focused suggestions and requests can also be opened as issues. Before opening a pull request, start an issue or a discussion on the topic, please.

If you want to ask a question not suited for a bug report, feel free to start a discussion here. This forum is for general discussion about this repository and the JuliaSmoothOptimizers organization, so questions about any of our packages are welcome.

diff --git a/previews/PR911/inplace/index.html b/previews/PR911/inplace/index.html index 6ad97aacf..d8f58817f 100644 --- a/previews/PR911/inplace/index.html +++ b/previews/PR911/inplace/index.html @@ -7,9 +7,9 @@ dqgmres!(dqgmres_solver, A2, b2) lsqr_solver = LsqrSolver(m, n, CuVector{Float32}) -lsqr!(lsqr_solver, A3, b3)

A generic function solve! is also available and dispatches to the appropriate Krylov method.

Krylov.solve! — Function
solve!(solver, args...; kwargs...)

Use the in-place Krylov method associated to solver.

source

In-place methods return an updated solver workspace. Solutions and statistics can be recovered via solver.x, solver.y and solver.stats. Functions solution, statistics and results can be also used.

Krylov.nsolution — Function
nsolution(solver)

Return the number of outputs of solution(solver).

source
Krylov.issolved — Function
issolved(solver)

Return a boolean that determines whether the Krylov method associated to solver succeeded.

source
Krylov.solution — Function
solution(solver)

Return the solution(s) stored in the solver. Optionally you can specify which solution you want to recover, solution(solver, 1) returns x and solution(solver, 2) returns y.

source
Krylov.statistics — Function
statistics(solver)

Return the statistics stored in the solver.

source
Krylov.results — Function
results(solver)

Return a tuple containing the solution(s) and the statistics associated with the solver. Allows retrieving the output arguments of an out-of-place method from the in-place method.

For example, instead of x, stats = cg(A, b), you can use:

    solver = CgSolver(A, b)
+lsqr!(lsqr_solver, A3, b3)

A generic function solve! is also available and dispatches to the appropriate Krylov method.

Krylov.solve! — Function
solve!(solver, args...; kwargs...)

Use the in-place Krylov method associated to solver.

source

In-place methods return an updated solver workspace. Solutions and statistics can be recovered via solver.x, solver.y and solver.stats. Functions solution, statistics and results can be also used.

Krylov.issolved — Function
issolved(solver)

Return a boolean that determines whether the Krylov method associated to solver succeeded.

source
Krylov.solution — Function
solution(solver)

Return the solution(s) stored in the solver. Optionally you can specify which solution you want to recover, solution(solver, 1) returns x and solution(solver, 2) returns y.

source
Krylov.results — Function
results(solver)

Return a tuple containing the solution(s) and the statistics associated with the solver. Allows retrieving the output arguments of an out-of-place method from the in-place method.

For example, instead of x, stats = cg(A, b), you can use:

    solver = CgSolver(A, b)
     cg!(solver, A, b)
-    x, stats = results(solver)
source

Examples

We illustrate the use of in-place Krylov solvers with two well-known optimization methods. The details of the optimization methods are described in the section about Factorization-free operators.

Example 1: Newton's method for convex optimization without linesearch

using Krylov
+    x, stats = results(solver)
source

Examples

We illustrate the use of in-place Krylov solvers with two well-known optimization methods. The details of the optimization methods are described in the section about Factorization-free operators.

Example 1: Newton's method for convex optimization without linesearch

using Krylov
 
 function newton(∇f, ∇²f, x₀; itmax = 200, tol = 1e-8)
 
@@ -67,4 +67,4 @@
         tired = iter ≥ itmax
     end
     return x
-end
+end diff --git a/previews/PR911/matrix_free/index.html b/previews/PR911/matrix_free/index.html index 0b352a5d0..d662c9fcb 100644 --- a/previews/PR911/matrix_free/index.html +++ b/previews/PR911/matrix_free/index.html @@ -1,5 +1,5 @@ -Matrix-free operators · Krylov.jl
GitHub
","category":"page"},{"location":"storage/#storage-requirements","page":"Storage requirements","title":"Storage requirements","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"This section provides the storage requirements of all Krylov methods available in Krylov.jl.","category":"page"},{"location":"storage/#Notation","page":"Storage requirements","title":"Notation","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"We denote by m and n the number of rows and columns of the linear problem. The memory parameter of DIOM, FOM, DQGMRES, GMRES, FGMRES and GPMR is k. The numbers of shifts of CG-LANCZOS-SHIFT and CGLS-LANCZOS-SHIFT is p.","category":"page"},{"location":"storage/#Theoretical-storage-requirements","page":"Storage requirements","title":"Theoretical storage requirements","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"The following tables provide the number of coefficients that must be allocated for each Krylov method. The coefficients have the same type as those that compose the linear problem we seek to solve. Each table summarizes the storage requirements of Krylov methods recommended to a specific linear problem.","category":"page"},{"location":"storage/#Hermitian-positive-definite-linear-systems","page":"Storage requirements","title":"Hermitian positive definite linear systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods CG CR CAR CG-LANCZOS CG-LANCZOS-SHIFT\nStorage 4n 5n 7n 5n 3n + 2np + 5p","category":"page"},{"location":"storage/#Hermitian-indefinite-linear-systems","page":"Storage requirements","title":"Hermitian indefinite linear systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods SYMMLQ MINRES MINRES-QLP MINARES\nStorage 5n 6n 6n 8n","category":"page"},{"location":"storage/#Non-Hermitian-square-linear-systems","page":"Storage requirements","title":"Non-Hermitian square linear systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods CGS BICGSTAB BiLQ QMR\nStorage 6n 6n 8n 9n","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods DIOM DQGMRES\nStorage n(2k+1) + 2k - 1 n(2k+2) + 3k + 1","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods FOM GMRES FGMRES\nStoragedfrac n(2+k) +2k + dfrack(k + 1)2 n(2+k) + 3k + dfrack(k + 1)2 n(2+2k) + 3k + dfrack(k + 1)2","category":"page"},{"location":"storage/#Least-norm-problems","page":"Storage requirements","title":"Least-norm problems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods USYMLQ CGNE CRMR LNLQ CRAIG CRAIGMR\nStorage 5n + 3m 3n + 2m 3n + 2m 3n + 4m 3n + 4m 4n + 5m","category":"page"},{"location":"storage/#Least-squares-problems","page":"Storage requirements","title":"Least-squares problems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods USYMQR CGLS CGLS-LANCZOS-SHIFT CRLS LSLQ LSQR LSMR\nStorage 6n + 3m 3n + 2m 3n + 2m + 5p + 2np 4n + 3m 4n + 2m 4n + 2m 5n + 2m","category":"page"},{"location":"storage/#Adjoint-systems","page":"Storage requirements","title":"Adjoint systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods BiLQR TriLQR\nStorage 11n 6m + 5n","category":"page"},{"location":"storage/#Saddle-point-and-Hermitian-quasi-definite-systems","page":"Storage requirements","title":"Saddle-point and Hermitian quasi-definite systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods TriCG TriMR\nStorage 6n + 6m 8n + 8m","category":"page"},{"location":"storage/#Generalized-saddle-point-and-non-Hermitian-partitioned-systems","page":"Storage requirements","title":"Generalized saddle-point and non-Hermitian partitioned systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Method GPMR\nStorage (2+k)(n+m) + 2k^2 + 11k","category":"page"},{"location":"storage/#Practical-storage-requirements","page":"Storage requirements","title":"Practical storage requirements","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Each method has its own KrylovSolver that contains all the storage needed by the method. In the REPL, the size in bytes of each attribute and the total amount of memory allocated by the solver are displayed when we show a KrylovSolver.","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"using Krylov\n\nm = 5000\nn = 12000\nA = rand(Float64, m, n)\nb = rand(Float64, m)\nsolver = LsmrSolver(A, b)\nshow(stdout, solver, show_stats=false)","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"If we want the total number of bytes used by the solver, we can call nbytes = sizeof(solver).","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"nbytes = sizeof(solver)","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Thereafter, we can use Base.format_bytes(nbytes) to recover what is displayed in the REPL.","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Base.format_bytes(nbytes)","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"To verify that we match the theoretical results, we just need to multiply the storage requirement of a method by the number of bytes associated to the precision of the linear problem. For instance, we need 4 bytes for the precision Float32, 8 bytes for precisions Float64 and ComplexF32, and 16 bytes for the precision ComplexF64.","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"FC = Float64 # precision of the least-squares problem\nncoefs_lsmr = 5*n + 2*m # number of coefficients\nnbytes_lsmr = sizeof(FC) * ncoefs_lsmr # number of bytes","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Therefore, you can check that you have enough memory in RAM to allocate a KrylovSolver.","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"free_nbytes = Sys.free_memory()\nBase.format_bytes(free_nbytes) # Total free memory in RAM in bytes.","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"note: Note\nBeyond having faster operations, using low precisions, such as simple precision, allows to store more coefficients in RAM and solve larger linear problems.\nIn the file test_allocations.jl, we use the macro @allocated to test that we match the expected storage requirement of each method with a tolerance of 2%.","category":"page"},{"location":"examples/craig/","page":"CRAIG","title":"CRAIG","text":"using Krylov\nusing LinearAlgebra, Printf\n\nm = 5\nn = 8\nλ = 1.0e-3\nA = rand(m, n)\nb = A * ones(n)\nxy_exact = [A λ*I] \\ b # In Julia, this is the min-norm solution!\n\n(x, y, stats) = craig(A, b, λ=λ, atol=0.0, rtol=1.0e-20, verbose=1)\nshow(stats)\n\n# Check that we have a minimum-norm solution.\n# When λ > 0 we solve min ‖(x,s)‖ s.t. Ax + λs = b, and we get s = λy.\n@printf(\"Primal feasibility: %7.1e\\n\", norm(b - A * x - λ^2 * y) / norm(b))\n@printf(\"Dual feasibility: %7.1e\\n\", norm(x - A' * y) / norm(x))\n@printf(\"Error in x: %7.1e\\n\", norm(x - xy_exact[1:n]) / norm(xy_exact[1:n]))\nif λ > 0.0\n @printf(\"Error in y: %7.1e\\n\", norm(λ * y - xy_exact[n+1:n+m]) / norm(xy_exact[n+1:n+m]))\nend","category":"page"},{"location":"examples/cgne/","page":"CGNE","title":"CGNE","text":"using Krylov, HarwellRutherfordBoeing, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"wm2\")\npath = fetch_ssmc(matrix, format=\"RB\")\n\nA = RutherfordBoeingData(joinpath(path[1], \"$(matrix.name[1]).rb\")).data\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\nx_exact = A' * ones(m)\nx_exact_norm = norm(x_exact)\nx_exact /= x_exact_norm\nb = A * x_exact\n(x, stats) = cgne(A, b)\nshow(stats)\nresid = norm(A * x - b) / norm(b)\n@printf(\"CGNE: Relative residual: %7.1e\\n\", resid)\n@printf(\"CGNE: ‖x - x*‖₂: %7.1e\\n\", norm(x - x_exact))","category":"page"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"# Least-squares problems","category":"page"},{"location":"solvers/ls/#CGLS","page":"Least-squares problems","title":"CGLS","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"cgls\ncgls!","category":"page"},{"location":"solvers/ls/#Krylov.cgls","page":"Least-squares problems","title":"Krylov.cgls","text":"(x, stats) = cgls(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, radius::T=zero(T),\n λ::T=zero(T), atol::T=√eps(T), rtol::T=√eps(T),\n itmax::Int=0, timemax::Float64=Inf,\n verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the regularized linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ‖x‖₂²\n\nof size m × n using the Conjugate Gradient (CG) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CG to the normal equations\n\n(AᴴA + λI) x = Aᴴb\n\nbut is more stable.\n\nCGLS produces monotonic residuals ‖r‖₂ but not optimality residuals ‖Aᴴr‖₂. It is formally equivalent to LSQR, though can be slightly less accurate, but simpler to implement.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nradius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nM. R. Hestenes and E. Stiefel. Methods of conjugate gradients for solving linear systems, Journal of Research of the National Bureau of Standards, 49(6), pp. 409–436, 1952.\nA. Björck, T. Elfving and Z. Strakos, Stability of Conjugate Gradient and Lanczos Methods for Linear Least Squares Problems, SIAM Journal on Matrix Analysis and Applications, 19(3), pp. 720–736, 1998.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.cgls!","page":"Least-squares problems","title":"Krylov.cgls!","text":"solver = cgls!(solver::CglsSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of cgls.\n\nSee CglsSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#CGLS-LANCZOS-SHIFT","page":"Least-squares problems","title":"CGLS-LANCZOS-SHIFT","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"cgls_lanczos_shift\ncgls_lanczos_shift!","category":"page"},{"location":"solvers/ls/#Krylov.cgls_lanczos_shift","page":"Least-squares problems","title":"Krylov.cgls_lanczos_shift","text":"(x, stats) = cgls_lanczos_shift(A, b::AbstractVector{FC};\n M=I, λ::T=zero(T), atol::T=√eps(T), rtol::T=√eps(T),\n radius::T=zero(T), itmax::Int=0, verbose::Int=0, history::Bool=false,\n callback=solver->false)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the regularized linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ‖x‖₂²\n\nusing the Conjugate Gradient (CG) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CG to the normal equations\n\n(AᴴA + λI) x = Aᴴb\n\nbut is more stable.\n\nCGLS produces monotonic residuals ‖r‖₂ but not optimality residuals ‖Aᴴr‖₂. It is formally equivalent to LSQR, though can be slightly less accurate, but simpler to implement.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nshifts: a vector of length p.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a vector of p dense vectors, each one of length n;\nstats: statistics collected on the run in a LanczosShiftStats structure.\n\nReferences\n\nM. R. Hestenes and E. Stiefel. Methods of conjugate gradients for solving linear systems, Journal of Research of the National Bureau of Standards, 49(6), pp. 409–436, 1952.\nA. Björck, T. Elfving and Z. Strakos, Stability of Conjugate Gradient and Lanczos Methods for Linear Least Squares Problems, SIAM Journal on Matrix Analysis and Applications, 19(3), pp. 720–736, 1998.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.cgls_lanczos_shift!","page":"Least-squares problems","title":"Krylov.cgls_lanczos_shift!","text":"solver = cgls_lanczos_shift!(solver::CglsLanczosShiftSolver, A, b, shifts; kwargs...)\n\nwhere kwargs are keyword arguments of cgls_lanczos_shift.\n\nSee CglsLanczosShiftSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#CRLS","page":"Least-squares problems","title":"CRLS","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"crls\ncrls!","category":"page"},{"location":"solvers/ls/#Krylov.crls","page":"Least-squares problems","title":"Krylov.crls","text":"(x, stats) = crls(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, radius::T=zero(T),\n λ::T=zero(T), atol::T=√eps(T), rtol::T=√eps(T),\n itmax::Int=0, timemax::Float64=Inf,\n verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ‖x‖₂²\n\nof size m × n using the Conjugate Residuals (CR) method. This method is equivalent to applying MINRES to the normal equations\n\n(AᴴA + λI) x = Aᴴb.\n\nThis implementation recurs the residual r := b - Ax.\n\nCRLS produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂. It is formally equivalent to LSMR, though can be substantially less accurate, but simpler to implement.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nradius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nD. C.-L. Fong, Minimum-Residual Methods for Sparse, Least-Squares using Golubg-Kahan Bidiagonalization, Ph.D. Thesis, Stanford University, 2011.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.crls!","page":"Least-squares problems","title":"Krylov.crls!","text":"solver = crls!(solver::CrlsSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of crls.\n\nSee CrlsSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#LSLQ","page":"Least-squares problems","title":"LSLQ","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"lslq\nlslq!","category":"page"},{"location":"solvers/ls/#Krylov.lslq","page":"Least-squares problems","title":"Krylov.lslq","text":"(x, stats) = lslq(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n window::Int=5, transfer_to_lsqr::Bool=false,\n sqd::Bool=false, λ::T=zero(T),\n σ::T=zero(T), etol::T=√eps(T),\n utol::T=√eps(T), btol::T=√eps(T),\n conlim::T=1/√eps(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the regularized linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ²‖x‖₂²\n\nof size m × n using the LSLQ method, where λ ≥ 0 is a regularization parameter. LSLQ is formally equivalent to applying SYMMLQ to the normal equations\n\n(AᴴA + λ²I) x = Aᴴb\n\nbut is more stable.\n\nIf λ > 0, we solve the symmetric and quasi-definite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ -λ²F ] [ x ] = [ 0 ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹ + λ²‖x‖²_F.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LSLQ is then equivalent to applying SYMMLQ to (AᴴE⁻¹A + λ²F)x = AᴴE⁻¹b with r = E⁻¹(b - Ax).\n\nIf λ = 0, we solve the symmetric and indefinite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ 0 ] [ x ] = [ 0 ].\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹.\n\nIn this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).\n\nMain features\n\nthe solution estimate is updated along orthogonal directions\nthe norm of the solution estimate ‖xᴸₖ‖₂ is increasing\nthe error ‖eₖ‖₂ := ‖xᴸₖ - x*‖₂ is decreasing\nit is possible to transition cheaply from the LSLQ iterate to the LSQR iterate if there is an advantage (there always is in terms of error)\nif A is rank deficient, identify the minimum least-squares solution\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\nwindow: number of iterations used to accumulate a lower bound on the error;\ntransfer_to_lsqr: transfer from the LSLQ point to the LSQR point, when it exists. The transfer is based on the residual norm;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\nσ: strict lower bound on the smallest positive singular value σₘᵢₙ such as σ = (1-10⁻⁷)σₘᵢₙ;\netol: stopping tolerance based on the lower bound on the error;\nutol: stopping tolerance based on the upper bound on the error;\nbtol: stopping tolerance used to detect zero-residual problems;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a LSLQStats structure.\nstats.err_lbnds is a vector of lower bounds on the LQ error–-the vector is empty if window is set to zero\nstats.err_ubnds_lq is a vector of upper bounds on the LQ error–-the vector is empty if σ == 0 is left at zero\nstats.err_ubnds_cg is a vector of upper bounds on the CG error–-the vector is empty if σ == 0 is left at zero\nstats.error_with_bnd is a boolean indicating whether there was an error in the upper bounds computation (cancellation errors, too large σ ...)\n\nStopping conditions\n\nThe iterations stop as soon as one of the following conditions holds true:\n\nthe optimality residual is sufficiently small (stats.status = \"found approximate minimum least-squares solution\") in the sense that either\n‖Aᴴr‖ / (‖A‖ ‖r‖) ≤ atol, or\n1 + ‖Aᴴr‖ / (‖A‖ ‖r‖) ≤ 1\nan approximate zero-residual solution has been found (stats.status = \"found approximate zero-residual solution\") in the sense that either\n‖r‖ / ‖b‖ ≤ btol + atol ‖A‖ * ‖xᴸ‖ / ‖b‖, or\n1 + ‖r‖ / ‖b‖ ≤ 1\nthe estimated condition number of A is too large in the sense that either\n1/cond(A) ≤ 1/conlim (stats.status = \"condition number exceeds tolerance\"), or\n1 + 1/cond(A) ≤ 1 (stats.status = \"condition number seems too large for this machine\")\nthe lower bound on the LQ forward error is less than etol * ‖xᴸ‖\nthe upper bound on the CG forward error is less than utol * ‖xᶜ‖\n\nReferences\n\nR. Estrin, D. Orban and M. A. Saunders, Euclidean-norm error bounds for SYMMLQ and CG, SIAM Journal on Matrix Analysis and Applications, 40(1), pp. 235–253, 2019.\nR. Estrin, D. Orban and M. A. Saunders, LSLQ: An Iterative Method for Linear Least-Squares with an Error Minimization Property, SIAM Journal on Matrix Analysis and Applications, 40(1), pp. 254–275, 2019.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.lslq!","page":"Least-squares problems","title":"Krylov.lslq!","text":"solver = lslq!(solver::LslqSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of lslq.\n\nSee LslqSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#LSQR","page":"Least-squares problems","title":"LSQR","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"lsqr\nlsqr!","category":"page"},{"location":"solvers/ls/#Krylov.lsqr","page":"Least-squares problems","title":"Krylov.lsqr","text":"(x, stats) = lsqr(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n window::Int=5, sqd::Bool=false, λ::T=zero(T),\n radius::T=zero(T), etol::T=√eps(T),\n axtol::T=√eps(T), btol::T=√eps(T),\n conlim::T=1/√eps(T), atol::T=zero(T),\n rtol::T=zero(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the regularized linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ²‖x‖₂²\n\nof size m × n using the LSQR method, where λ ≥ 0 is a regularization parameter. LSQR is formally equivalent to applying CG to the normal equations\n\n(AᴴA + λ²I) x = Aᴴb\n\n(and therefore to CGLS) but is more stable.\n\nLSQR produces monotonic residuals ‖r‖₂ but not optimality residuals ‖Aᴴr‖₂. It is formally equivalent to CGLS, though can be slightly more accurate.\n\nIf λ > 0, LSQR solves the symmetric and quasi-definite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ -λ²F ] [ x ] = [ 0 ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹ + λ²‖x‖²_F.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LSQR is then equivalent to applying CG to (AᴴE⁻¹A + λ²F)x = AᴴE⁻¹b with r = E⁻¹(b - Ax).\n\nIf λ = 0, we solve the symmetric and indefinite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ 0 ] [ x ] = [ 0 ].\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹.\n\nIn this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\nwindow: number of iterations used to accumulate a lower bound on the error;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\nradius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;\netol: stopping tolerance based on the lower bound on the error;\naxtol: tolerance on the backward error;\nbtol: stopping tolerance used to detect zero-residual problems;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nC. C. Paige and M. A. Saunders, LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares, ACM Transactions on Mathematical Software, 8(1), pp. 43–71, 1982.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.lsqr!","page":"Least-squares problems","title":"Krylov.lsqr!","text":"solver = lsqr!(solver::LsqrSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of lsqr.\n\nSee LsqrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#LSMR","page":"Least-squares problems","title":"LSMR","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"lsmr\nlsmr!","category":"page"},{"location":"solvers/ls/#Krylov.lsmr","page":"Least-squares problems","title":"Krylov.lsmr","text":"(x, stats) = lsmr(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n window::Int=5, sqd::Bool=false, λ::T=zero(T),\n radius::T=zero(T), etol::T=√eps(T),\n axtol::T=√eps(T), btol::T=√eps(T),\n conlim::T=1/√eps(T), atol::T=zero(T),\n rtol::T=zero(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the regularized linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ²‖x‖₂²\n\nof size m × n using the LSMR method, where λ ≥ 0 is a regularization parameter. LSMR is formally equivalent to applying MINRES to the normal equations\n\n(AᴴA + λ²I) x = Aᴴb\n\n(and therefore to CRLS) but is more stable.\n\nLSMR produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂. It is formally equivalent to CRLS, though can be substantially more accurate.\n\nLSMR can be also used to find a null vector of a singular matrix A by solving the problem min ‖Aᴴx - b‖ with any nonzero vector b. At a minimizer, the residual vector r = b - Aᴴx will satisfy Ar = 0.\n\nIf λ > 0, we solve the symmetric and quasi-definite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ -λ²F ] [ x ] = [ 0 ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹ + λ²‖x‖²_F.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LSMR is then equivalent to applying MINRES to (AᴴE⁻¹A + λ²F)x = AᴴE⁻¹b with r = E⁻¹(b - Ax).\n\nIf λ = 0, we solve the symmetric and indefinite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ 0 ] [ x ] = [ 0 ].\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹.\n\nIn this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\nwindow: number of iterations used to accumulate a lower bound on the error;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\nradius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;\netol: stopping tolerance based on the lower bound on the error;\naxtol: tolerance on the backward error;\nbtol: stopping tolerance used to detect zero-residual problems;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a LsmrStats structure.\n\nReference\n\nD. C.-L. Fong and M. A. Saunders, LSMR: An Iterative Algorithm for Sparse Least Squares Problems, SIAM Journal on Scientific Computing, 33(5), pp. 2950–2971, 2011.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.lsmr!","page":"Least-squares problems","title":"Krylov.lsmr!","text":"solver = lsmr!(solver::LsmrSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of lsmr.\n\nSee LsmrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#USYMQR","page":"Least-squares problems","title":"USYMQR","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"usymqr\nusymqr!","category":"page"},{"location":"solvers/ls/#Krylov.usymqr","page":"Least-squares problems","title":"Krylov.usymqr","text":"(x, stats) = usymqr(A, b::AbstractVector{FC}, c::AbstractVector{FC};\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = usymqr(A, b, c, x0::AbstractVector; kwargs...)\n\nUSYMQR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nUSYMQR solves the linear least-squares problem min ‖b - Ax‖² of size m × n. USYMQR solves Ax = b if it is consistent.\n\nUSYMQR is based on the orthogonal tridiagonalization process and requires two initial nonzero vectors b and c. The vector c is only used to initialize the process and a default value can be b or Aᴴb depending on the shape of A. The residual norm ‖b - Ax‖ monotonously decreases in USYMQR. When A is Hermitian and b = c, QMR is equivalent to MINRES. USYMQR is considered as a generalization of MINRES.\n\nIt can also be applied to under-determined and over-determined problems. USYMQR finds the minimum-norm solution if problems are inconsistent.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nc: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nM. A. Saunders, H. D. Simon, and E. L. Yip, Two Conjugate-Gradient-Type Methods for Unsymmetric Linear Equations, SIAM Journal on Numerical Analysis, 25(4), pp. 927–940, 1988.\nL. Reichel and Q. Ye, A generalized LSQR algorithm, Numerical Linear Algebra with Applications, 15(7), pp. 643–660, 2008.\nA. Buttari, D. Orban, D. Ruiz and D. Titley-Peloquin, A tridiagonalization method for symmetric saddle-point and quasi-definite systems, SIAM Journal on Scientific Computing, 41(5), pp. 409–432, 2019.\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.usymqr!","page":"Least-squares problems","title":"Krylov.usymqr!","text":"solver = usymqr!(solver::UsymqrSolver, A, b, c; kwargs...)\nsolver = usymqr!(solver::UsymqrSolver, A, b, c, x0; kwargs...)\n\nwhere kwargs are keyword arguments of usymqr.\n\nSee UsymqrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"preconditioners/#preconditioners","page":"Preconditioners","title":"Preconditioners","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"The solvers in Krylov.jl support preconditioners, i.e., transformations that modify a linear system Ax = b into an equivalent form that may yield faster convergence in finite-precision arithmetic. Preconditioning can be used to reduce the condition number of the problem or cluster its eigenvalues or singular values for instance.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"The design of preconditioners is highly dependent on the origin of the problem and most preconditioners need to take application-dependent information and structure into account. Specialized preconditioners generally outperform generic preconditioners such as incomplete factorizations.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"The construction of a preconditioner necessitates trade-offs because we need to apply it at least once per iteration within a Krylov method. Hence, a preconditioner must be constructed such that it is cheap to apply, while also capturing the characteristics of the original system in some sense.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"There exist three variants of preconditioning:","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Left preconditioning Two-sided preconditioning Right preconditioning\nP_ell^-1Ax = P_ell^-1b P_ell^-1AP_r^-1y = P_ell^-1btextwithx = P_r^-1y AP_r^-1y = btextwithx = P_r^-1y","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"where P_ell and P_r are square and nonsingular.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"The left preconditioning preserves the error x_k - x^star whereas the right preconditioning keeps invariant the residual b - A x_k. Two-sided preconditioning is the only variant that allows to preserve the hermicity of a linear system.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"note: Note\nBecause det(P^-1A - lambda I) = det(A - lambda P) det(P^-1) = det(AP^-1 - lambda I), the eigenvalues of P^-1A and AP^-1 are identical. If P = LL^H, L^-1AL^-H also has the same spectrum.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"In Krylov.jl, we call P_ell^-1 and P_r^-1 the preconditioners and we assume that we can apply them with the operation y leftarrow P^-1 * x. It is also common to call P_ell and P_r the preconditioners if the equivalent operation y leftarrow Pbackslashx is available. Krylov.jl supports both approaches thanks to the argument ldiv of the Krylov solvers.","category":"page"},{"location":"preconditioners/#How-to-use-preconditioners-in-Krylov.jl?","page":"Preconditioners","title":"How to use preconditioners in Krylov.jl?","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"info: Info\nA preconditioner only needs to support the operation mul!(y, P⁻¹, x) when ldiv=false or ldiv!(y, P, x) when ldiv=true to be used in Krylov.jl.\nAdditional support for adjoint with preconditioners is required in the methods BILQ and QMR.\nThe default value of a preconditioner in Krylov.jl is the identity operator I.","category":"page"},{"location":"preconditioners/#Square-non-Hermitian-linear-systems","page":"Preconditioners","title":"Square non-Hermitian linear systems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Methods concerned: CGS, BILQ, QMR, BiCGSTAB, DQGMRES, GMRES, BLOCK-GMRES, FGMRES, DIOM and FOM.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"A Krylov method dedicated to non-Hermitian linear systems allows the three variants of preconditioning.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Preconditioners P_ell^-1 P_ell P_r^-1 P_r\nArguments M with ldiv=false M with ldiv=true N with ldiv=false N with ldiv=true","category":"page"},{"location":"preconditioners/#Hermitian-linear-systems","page":"Preconditioners","title":"Hermitian linear systems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Methods concerned: SYMMLQ, CG, CG-LANCZOS, CG-LANCZOS-SHIFT, CR, CAR, MINRES, MINRES-QLP and MINARES.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"When A is Hermitian, we can only use centered preconditioning L^-1AL^-Hy = L^-1b with x = L^-Hy. Centered preconditioning is a special case of two-sided preconditioning with P_ell = L = P_r^H that maintains hermicity. However, there is no need to specify L and one may specify P_c = LL^H or its inverse directly.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Preconditioners P_c^-1 P_c\nArguments M with ldiv=false M with ldiv=true","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"warning: Warning\nThe preconditioner M must be hermitian and positive definite.","category":"page"},{"location":"preconditioners/#Linear-least-squares-problems","page":"Preconditioners","title":"Linear least-squares problems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Methods concerned: CGLS, CGLS-LANCZOS-SHIFT, CRLS, LSLQ, LSQR and LSMR.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Formulation Without preconditioning With preconditioning\nleast-squares problem min tfrac12 b - Ax^2_2 min tfrac12 b - Ax^2_E^-1\nNormal equation A^HAx = A^Hb A^HE^-1Ax = A^HE^-1b\nAugmented system beginbmatrix I A A^H 0 endbmatrix beginbmatrix r x endbmatrix = beginbmatrix b 0 endbmatrix beginbmatrix E A A^H 0 endbmatrix beginbmatrix r x endbmatrix = beginbmatrix b 0 endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"LSLQ, LSQR and LSMR also handle regularized least-squares problems.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Formulation Without preconditioning With preconditioning\nleast-squares problem min tfrac12 b - Ax^2_2 + tfrac12 lambda^2 x^2_2 min tfrac12 b - Ax^2_E^-1 + tfrac12 lambda^2 x^2_F\nNormal equation (A^HA + lambda^2 I)x = A^Hb (A^HE^-1A + lambda^2 F)x = A^HE^-1b\nAugmented system beginbmatrix I A A^H -lambda^2 I endbmatrix beginbmatrix r x endbmatrix = beginbmatrix b 0 endbmatrix beginbmatrix E A A^H -lambda^2 F endbmatrix beginbmatrix r x endbmatrix = beginbmatrix b 0 endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Preconditioners E^-1 E F^-1 F\nArguments M with ldiv=false M with ldiv=true N with ldiv=false N with ldiv=true","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"warning: Warning\nThe preconditioners M and N must be hermitian and positive definite.","category":"page"},{"location":"preconditioners/#Linear-least-norm-problems","page":"Preconditioners","title":"Linear least-norm problems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Methods concerned: CGNE, CRMR, LNLQ, CRAIG and CRAIGMR.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Formulation Without preconditioning With preconditioning\nminimum-norm problem min tfrac12 x^2_2textstAx = b min tfrac12 x^2_FtextstAx = b\nNormal equation AA^Hy = btextwithx = A^Hy AF^-1A^Hy = btextwithx = F^-1A^Hy\nAugmented system beginbmatrix -I A^H phantom-A 0 endbmatrix beginbmatrix x y endbmatrix = beginbmatrix 0 b endbmatrix beginbmatrix -F A^H phantom-A 0 endbmatrix beginbmatrix x y endbmatrix = beginbmatrix 0 b endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"LNLQ, CRAIG and CRAIGMR also handle penalized minimum-norm problems.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Formulation Without preconditioning With preconditioning\nminimum-norm problem min tfrac12 x^2_2 + tfrac12 y^2_2textstAx + lambda^2 y = b min tfrac12 x^2_F + tfrac12 y^2_EtextstAx + lambda^2 Ey = b\nNormal equation (AA^H + lambda^2 I)y = btextwithx = A^Hy (AF^-1A^H + lambda^2 E)y = btextwithx = F^-1A^Hy\nAugmented system beginbmatrix -I A^H phantom-A lambda^2 I endbmatrix beginbmatrix x y endbmatrix = beginbmatrix 0 b endbmatrix beginbmatrix -F A^H phantom-A lambda^2 E endbmatrix beginbmatrix x y endbmatrix = beginbmatrix 0 b endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Preconditioners E^-1 E F^-1 F\nArguments M with ldiv=false M with ldiv=true N with ldiv=false N with ldiv=true","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"warning: Warning\nThe preconditioners M and N must be hermitian and positive definite.","category":"page"},{"location":"preconditioners/#Saddle-point-and-symmetric-quasi-definite-systems","page":"Preconditioners","title":"Saddle-point and symmetric quasi-definite systems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"TriCG and TriMR can take advantage of the structure of Hermitian systems Kz = d with the 2x2 block structure","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":" beginbmatrix tau E phantom-A A^H nu F endbmatrix beginbmatrix x y endbmatrix = beginbmatrix b c endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Preconditioners E^-1 E F^-1 F\nArguments M with ldiv=false M with ldiv=true N with ldiv=false N with ldiv=true","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"warning: Warning\nThe preconditioners M and N must be hermitian and positive definite.","category":"page"},{"location":"preconditioners/#Generalized-saddle-point-and-unsymmetric-partitioned-systems","page":"Preconditioners","title":"Generalized saddle-point and unsymmetric partitioned systems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"GPMR can take advantage of the structure of general square systems Kz = d with the 2x2 block structure","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":" beginbmatrix lambda M A B mu N endbmatrix beginbmatrix x y endbmatrix = beginbmatrix b c endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Relations CE = M^-1 EC = M DF = N^-1 FD = N\nArguments C and E with ldiv=false C and E with ldiv=true D and F with ldiv=false D and F with ldiv=true","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"note: Note\nOur implementations of BiLQ, QMR, BiLQR, USYMLQ, USYMQR and TriLQR don't support preconditioning.","category":"page"},{"location":"preconditioners/#Packages-that-provide-preconditioners","page":"Preconditioners","title":"Packages that provide preconditioners","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"KrylovPreconditioners.jl implements block-Jacobi, IC(0) and ILU(0) preconditioners.\nILUZero.jl is a Julia implementation of incomplete LU factorization with zero level of fill-in. \nLimitedLDLFactorizations.jl for limited-memory LDLᵀ factorization of symmetric matrices.\nAlgebraicMultigrid.jl provides two algebraic multigrid (AMG) preconditioners.\nRandomizedPreconditioners.jl uses randomized numerical linear algebra to construct approximate inverses of matrices.\nBasicLU.jl uses a sparse LU factorization to compute a maximum volume basis that can be used as a preconditioner for least-norm and least-squares problems.","category":"page"},{"location":"preconditioners/#Examples","page":"Preconditioners","title":"Examples","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using KrylovPreconditioners, Krylov\n\nP⁻¹ = BlockJacobiPreconditioner(A) # Block-Jacobi preconditioner\nx, stats = gmres(A, b, M=P⁻¹)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using Krylov\nn, m = size(A)\nd = [A[i,i] ≠ 0 ? 1 / abs(A[i,i]) : 1 for i=1:n] # Jacobi preconditioner\nP⁻¹ = diagm(d)\nx, stats = symmlq(A, b, M=P⁻¹)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using Krylov\nn, m = size(A)\nd = [1 / norm(A[:,i]) for i=1:m] # diagonal preconditioner\nP⁻¹ = diagm(d)\nx, stats = minres(A, b, M=P⁻¹)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using KrylovPreconditioners, Krylov\nPℓ = ilu(A)\nx, stats = gmres(A, b, M=Pℓ, ldiv=true) # left preconditioning","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using LimitedLDLFactorizations, Krylov\nP = lldl(A)\nP.D .= abs.(P.D)\nx, stats = cg(A, b, M=P, ldiv=true) # centered preconditioning","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using ILUZero, Krylov\nPᵣ = ilu0(A)\nx, stats = bicgstab(A, b, N=Pᵣ, ldiv=true) # right preconditioning","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using LDLFactorizations, Krylov\n\nM = ldl(E)\nN = ldl(F)\n\n# [E A] [x] = [b]\n# [Aᴴ -F] [y] [c]\nx, y, stats = tricg(A, b, c, M=M, N=N, ldiv=true)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using RandomizedPreconditioners, Krylov\n\n = NystromSketch(A, k, r)\n\nP = NystromPreconditioner(Â, μ)\nx, stats = cg(A + μ*I, b; M=P, ldiv=true)\n\nP⁻¹ = NystromPreconditionerInverse(Â, μ)\nx, stats = cg(A + μ*I, b; M=P⁻¹)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using SuiteSparse, Krylov\nimport LinearAlgebra.ldiv!\n\nM = cholesky(E)\n\n# ldiv! is not implemented for the sparse Cholesky factorization (SuiteSparse.CHOLMOD)\nldiv!(y::Vector{T}, F::SuiteSparse.CHOLMOD.Factor{T}, x::Vector{T}) where T = (y .= F \\ x)\n\n# [E A] [x] = [b]\n# [Aᴴ 0] [y] [c]\nx, y, stats = trimr(A, b, c, M=M, sp=true, ldiv=true)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using Krylov\n\nC = lu(M)\n\n# [M A] [x] = [b]\n# [B 0] [y] [c]\nx, y, stats = gpmr(A, B, b, c, C=C, gsp=true, ldiv=true)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"import BasicLU\nusing LinearOperators, Krylov\n\n# Least-squares problem\nm, n = size(A)\nAᴴ = sparse(A')\nbasis, B = BasicLU.maxvolbasis(Aᴴ)\nopA = LinearOperator(A)\nB⁻ᴴ = LinearOperator(Float64, n, n, false, false, (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'T')),\n (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'N')),\n (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'N')))\n\nd, stats = lsmr(opA * B⁻ᴴ, b) # min ‖AB⁻ᴴd - b‖₂\nx = B⁻ᴴ * d # recover the solution of min ‖Ax - b‖₂\n\n# Least-norm problem\nm, n = size(A)\nbasis, B = maxvolbasis(A)\nopA = LinearOperator(A)\nB⁻¹ = LinearOperator(Float64, m, m, false, false, (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'N')),\n (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'T')),\n (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'T')))\n\nx, y, stats = craigmr(B⁻¹ * opA, B⁻¹ * b) # min ‖x‖₂ s.t. B⁻¹Ax = B⁻¹b","category":"page"},{"location":"reference/#Reference","page":"Reference","title":"Reference","text":"","category":"section"},{"location":"reference/#Index","page":"Reference","title":"Index","text":"","category":"section"},{"location":"reference/","page":"Reference","title":"Reference","text":"","category":"page"},{"location":"reference/","page":"Reference","title":"Reference","text":"Krylov.FloatOrComplex\nKrylov.niterations\nKrylov.Aprod\nKrylov.Atprod\nKrylov.kstdout\nKrylov.extract_parameters\nBase.show","category":"page"},{"location":"reference/#Krylov.FloatOrComplex","page":"Reference","title":"Krylov.FloatOrComplex","text":"FloatOrComplex{T}\n\nUnion type of T and Complex{T} where T is an AbstractFloat.\n\n\n\n\n\n","category":"type"},{"location":"reference/#Krylov.niterations","page":"Reference","title":"Krylov.niterations","text":"niterations(solver)\n\nReturn the number of iterations performed by the Krylov method associated to solver.\n\n\n\n\n\n","category":"function"},{"location":"reference/#Krylov.Aprod","page":"Reference","title":"Krylov.Aprod","text":"Aprod(solver)\n\nReturn the number of operator-vector products with A performed by the Krylov method associated to solver.\n\n\n\n\n\n","category":"function"},{"location":"reference/#Krylov.Atprod","page":"Reference","title":"Krylov.Atprod","text":"Atprod(solver)\n\nReturn the number of operator-vector products with A' performed by the Krylov method associated to solver.\n\n\n\n\n\n","category":"function"},{"location":"reference/#Krylov.kstdout","page":"Reference","title":"Krylov.kstdout","text":"Default I/O stream for all Krylov methods.\n\n\n\n\n\n","category":"constant"},{"location":"reference/#Krylov.extract_parameters","page":"Reference","title":"Krylov.extract_parameters","text":"arguments = extract_parameters(ex::Expr)\n\nExtract the arguments of an expression that is keyword parameter tuple. Implementation suggested by Mitchell J. O'Sullivan (@mosullivan93).\n\n\n\n\n\n","category":"function"},{"location":"reference/#Base.show","page":"Reference","title":"Base.show","text":"show(io, solver; show_stats=true)\n\nStatistics of solver are displayed if show_stats is set to true.\n\n\n\n\n\n","category":"function"},{"location":"examples/block_gmres/","page":"BLOCK-GMRES","title":"BLOCK-GMRES","text":"using Krylov, MatrixMarket, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"sherman2\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nn = matrix.nrows[1]\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nB = Matrix{Float64}(I, n, 5)\nB_norm = norm(B)\n\n# Solve Ax = B\nX, stats = block_gmres(A, B)\nshow(stats)\nR = B - A * X\n@printf(\"Relative residual: %8.1e\\n\", norm(R) / B_norm)","category":"page"},{"location":"examples/cg/","page":"CG","title":"CG","text":"using Krylov, MatrixMarket, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"bcsstk09\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nn = matrix.nrows[1]\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = ones(n)\nb_norm = norm(b)\n\n# Solve Ax = b.\n(x, stats) = cg(A, b)\nshow(stats)\nr = b - A * x\n@printf(\"Relative residual: %8.1e\\n\", norm(r) / b_norm)","category":"page"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"# Non-Hermitian square linear systems","category":"page"},{"location":"solvers/unsymmetric/#BiLQ","page":"Non-Hermitian square linear systems","title":"BiLQ","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"bilq\nbilq!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.bilq","page":"Non-Hermitian square linear systems","title":"Krylov.bilq","text":"(x, stats) = bilq(A, b::AbstractVector{FC};\n c::AbstractVector{FC}=b, transfer_to_bicg::Bool=true,\n M=I, N=I, ldiv::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0, timemax::Float64=Inf,\n verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = bilq(A, b, x0::AbstractVector; kwargs...)\n\nBiLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the square linear system Ax = b of size n using BiLQ. BiLQ is based on the Lanczos biorthogonalization process and requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b. When A is Hermitian and b = c, BiLQ is equivalent to SYMMLQ. BiLQ requires support for adjoint(M) and adjoint(N) if preconditioners are provided.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nc: the second initial vector of length n required by the Lanczos biorthogonalization process;\ntransfer_to_bicg: transfer from the BiLQ point to the BiCG point, when it exists. The transfer is based on the residual norm;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\nR. Fletcher, Conjugate gradient methods for indefinite systems, Numerical Analysis, Springer, pp. 73–89, 1976.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.bilq!","page":"Non-Hermitian square linear systems","title":"Krylov.bilq!","text":"solver = bilq!(solver::BilqSolver, A, b; kwargs...)\nsolver = bilq!(solver::BilqSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of bilq.\n\nSee BilqSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#QMR","page":"Non-Hermitian square linear systems","title":"QMR","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"qmr\nqmr!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.qmr","page":"Non-Hermitian square linear systems","title":"Krylov.qmr","text":"(x, stats) = qmr(A, b::AbstractVector{FC};\n c::AbstractVector{FC}=b, M=I, N=I, ldiv::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0, timemax::Float64=Inf, verbose::Int=0,\n history::Bool=false, callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = qmr(A, b, x0::AbstractVector; kwargs...)\n\nQMR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the square linear system Ax = b of size n using QMR.\n\nQMR is based on the Lanczos biorthogonalization process and requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b. When A is Hermitian and b = c, QMR is equivalent to MINRES. QMR requires support for adjoint(M) and adjoint(N) if preconditioners are provided.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nc: the second initial vector of length n required by the Lanczos biorthogonalization process;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nR. W. Freund and N. M. Nachtigal, QMR : a quasi-minimal residual method for non-Hermitian linear systems, Numerische mathematik, Vol. 60(1), pp. 315–339, 1991.\nR. W. Freund and N. M. Nachtigal, An implementation of the QMR method based on coupled two-term recurrences, SIAM Journal on Scientific Computing, Vol. 15(2), pp. 313–337, 1994.\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.qmr!","page":"Non-Hermitian square linear systems","title":"Krylov.qmr!","text":"solver = qmr!(solver::QmrSolver, A, b; kwargs...)\nsolver = qmr!(solver::QmrSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of qmr.\n\nSee QmrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#CGS","page":"Non-Hermitian square linear systems","title":"CGS","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"cgs\ncgs!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.cgs","page":"Non-Hermitian square linear systems","title":"Krylov.cgs","text":"(x, stats) = cgs(A, b::AbstractVector{FC};\n c::AbstractVector{FC}=b, M=I, N=I,\n ldiv::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = cgs(A, b, x0::AbstractVector; kwargs...)\n\nCGS can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the consistent linear system Ax = b of size n using CGS. CGS requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b.\n\nFrom \"Iterative Methods for Sparse Linear Systems (Y. Saad)\" :\n\n«The method is based on a polynomial variant of the conjugate gradients algorithm. Although related to the so-called bi-conjugate gradients (BCG) algorithm, it does not involve adjoint matrix-vector multiplications, and the expected convergence rate is about twice that of the BCG algorithm.\n\nThe Conjugate Gradient Squared algorithm works quite well in many cases. However, one difficulty is that, since the polynomials are squared, rounding errors tend to be more damaging than in the standard BCG algorithm. In particular, very high variations of the residual vectors often cause the residual norms computed to become inaccurate.\n\nTFQMR and BICGSTAB were developed to remedy this difficulty.»\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nc: the second initial vector of length n required by the Lanczos biorthogonalization process;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nP. Sonneveld, CGS, A Fast Lanczos-Type Solver for Nonsymmetric Linear systems, SIAM Journal on Scientific and Statistical Computing, 10(1), pp. 36–52, 1989.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.cgs!","page":"Non-Hermitian square linear systems","title":"Krylov.cgs!","text":"solver = cgs!(solver::CgsSolver, A, b; kwargs...)\nsolver = cgs!(solver::CgsSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of cgs.\n\nSee CgsSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#BiCGSTAB","page":"Non-Hermitian square linear systems","title":"BiCGSTAB","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"bicgstab\nbicgstab!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.bicgstab","page":"Non-Hermitian square linear systems","title":"Krylov.bicgstab","text":"(x, stats) = bicgstab(A, b::AbstractVector{FC};\n c::AbstractVector{FC}=b, M=I, N=I,\n ldiv::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = bicgstab(A, b, x0::AbstractVector; kwargs...)\n\nBICGSTAB can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the square linear system Ax = b of size n using BICGSTAB. BICGSTAB requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b.\n\nThe Biconjugate Gradient Stabilized method is a variant of BiCG, like CGS, but using different updates for the Aᴴ-sequence in order to obtain smoother convergence than CGS.\n\nIf BICGSTAB stagnates, we recommend DQGMRES and BiLQ as alternative methods for unsymmetric square systems.\n\nBICGSTAB stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖b‖ * rtol.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nc: the second initial vector of length n required by the Lanczos biorthogonalization process;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nH. A. van der Vorst, Bi-CGSTAB: A fast and smoothly converging variant of Bi-CG for the solution of nonsymmetric linear systems, SIAM Journal on Scientific and Statistical Computing, 13(2), pp. 631–644, 1992.\nG. L.G. Sleijpen and D. R. Fokkema, BiCGstab(ℓ) for linear equations involving unsymmetric matrices with complex spectrum, Electronic Transactions on Numerical Analysis, 1, pp. 11–32, 1993.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.bicgstab!","page":"Non-Hermitian square linear systems","title":"Krylov.bicgstab!","text":"solver = bicgstab!(solver::BicgstabSolver, A, b; kwargs...)\nsolver = bicgstab!(solver::BicgstabSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of bicgstab.\n\nSee BicgstabSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#DIOM","page":"Non-Hermitian square linear systems","title":"DIOM","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"diom\ndiom!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.diom","page":"Non-Hermitian square linear systems","title":"Krylov.diom","text":"(x, stats) = diom(A, b::AbstractVector{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n reorthogonalization::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = diom(A, b, x0::AbstractVector; kwargs...)\n\nDIOM can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the consistent linear system Ax = b of size n using DIOM.\n\nDIOM only orthogonalizes the new vectors of the Krylov basis against the memory most recent vectors. If CG is well defined on Ax = b and memory = 2, DIOM is theoretically equivalent to CG. If k ≤ memory where k is the number of iterations, DIOM is theoretically equivalent to FOM. Otherwise, DIOM interpolates between CG and FOM and is similar to CG with partial reorthogonalization.\n\nAn advantage of DIOM is that non-Hermitian or Hermitian indefinite or both non-Hermitian and indefinite systems of linear equations can be handled by this single algorithm.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nmemory: the number of most recent vectors of the Krylov basis against which to orthogonalize a new vector;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against the memory most recent vectors;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nY. Saad, Practical use of some krylov subspace methods for solving indefinite and nonsymmetric linear systems, SIAM journal on scientific and statistical computing, 5(1), pp. 203–228, 1984.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.diom!","page":"Non-Hermitian square linear systems","title":"Krylov.diom!","text":"solver = diom!(solver::DiomSolver, A, b; kwargs...)\nsolver = diom!(solver::DiomSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of diom.\n\nNote that the memory keyword argument is the only exception. It's required to create a DiomSolver and can't be changed later.\n\nSee DiomSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#FOM","page":"Non-Hermitian square linear systems","title":"FOM","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"fom\nfom!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.fom","page":"Non-Hermitian square linear systems","title":"Krylov.fom","text":"(x, stats) = fom(A, b::AbstractVector{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n restart::Bool=false, reorthogonalization::Bool=false,\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = fom(A, b, x0::AbstractVector; kwargs...)\n\nFOM can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the linear system Ax = b of size n using FOM.\n\nFOM algorithm is based on the Arnoldi process and a Galerkin condition.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nmemory: if restart = true, the restarted version FOM(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\nrestart: restart the method after memory iterations;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nY. Saad, Krylov subspace methods for solving unsymmetric linear systems, Mathematics of computation, Vol. 37(155), pp. 105–126, 1981.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.fom!","page":"Non-Hermitian square linear systems","title":"Krylov.fom!","text":"solver = fom!(solver::FomSolver, A, b; kwargs...)\nsolver = fom!(solver::FomSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of fom.\n\nNote that the memory keyword argument is the only exception. It's required to create a FomSolver and can't be changed later.\n\nSee FomSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#DQGMRES","page":"Non-Hermitian square linear systems","title":"DQGMRES","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"dqgmres\ndqgmres!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.dqgmres","page":"Non-Hermitian square linear systems","title":"Krylov.dqgmres","text":"(x, stats) = dqgmres(A, b::AbstractVector{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n reorthogonalization::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = dqgmres(A, b, x0::AbstractVector; kwargs...)\n\nDQGMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the consistent linear system Ax = b of size n using DQGMRES.\n\nDQGMRES algorithm is based on the incomplete Arnoldi orthogonalization process and computes a sequence of approximate solutions with the quasi-minimal residual property.\n\nDQGMRES only orthogonalizes the new vectors of the Krylov basis against the memory most recent vectors. If MINRES is well defined on Ax = b and memory = 2, DQGMRES is theoretically equivalent to MINRES. If k ≤ memory where k is the number of iterations, DQGMRES is theoretically equivalent to GMRES. Otherwise, DQGMRES interpolates between MINRES and GMRES and is similar to MINRES with partial reorthogonalization.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nmemory: the number of most recent vectors of the Krylov basis against which to orthogonalize a new vector;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against the memory most recent vectors;\nldiv: define whether the preconditioners use ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nY. Saad and K. Wu, DQGMRES: a quasi minimal residual algorithm based on incomplete orthogonalization, Numerical Linear Algebra with Applications, Vol. 3(4), pp. 329–343, 1996.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.dqgmres!","page":"Non-Hermitian square linear systems","title":"Krylov.dqgmres!","text":"solver = dqgmres!(solver::DqgmresSolver, A, b; kwargs...)\nsolver = dqgmres!(solver::DqgmresSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of dqgmres.\n\nNote that the memory keyword argument is the only exception. It's required to create a DqgmresSolver and can't be changed later.\n\nSee DqgmresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#GMRES","page":"Non-Hermitian square linear systems","title":"GMRES","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"gmres\ngmres!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.gmres","page":"Non-Hermitian square linear systems","title":"Krylov.gmres","text":"(x, stats) = gmres(A, b::AbstractVector{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n restart::Bool=false, reorthogonalization::Bool=false,\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = gmres(A, b, x0::AbstractVector; kwargs...)\n\nGMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the linear system Ax = b of size n using GMRES.\n\nGMRES algorithm is based on the Arnoldi process and computes a sequence of approximate solutions with the minimum residual.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nmemory: if restart = true, the restarted version GMRES(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\nrestart: restart the method after memory iterations;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nY. Saad and M. H. Schultz, GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems, SIAM Journal on Scientific and Statistical Computing, Vol. 7(3), pp. 856–869, 1986.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.gmres!","page":"Non-Hermitian square linear systems","title":"Krylov.gmres!","text":"solver = gmres!(solver::GmresSolver, A, b; kwargs...)\nsolver = gmres!(solver::GmresSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of gmres.\n\nNote that the memory keyword argument is the only exception. It's required to create a GmresSolver and can't be changed later.\n\nSee GmresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#FGMRES","page":"Non-Hermitian square linear systems","title":"FGMRES","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"fgmres\nfgmres!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.fgmres","page":"Non-Hermitian square linear systems","title":"Krylov.fgmres","text":"(x, stats) = fgmres(A, b::AbstractVector{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n restart::Bool=false, reorthogonalization::Bool=false,\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = fgmres(A, b, x0::AbstractVector; kwargs...)\n\nFGMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the linear system Ax = b of size n using FGMRES.\n\nFGMRES computes a sequence of approximate solutions with minimum residual. FGMRES is a variant of GMRES that allows changes in the right preconditioner at each iteration.\n\nThis implementation allows a left preconditioner M and a flexible right preconditioner N. A situation in which the preconditioner is \"not constant\" is when a relaxation-type method, a Chebyshev iteration or another Krylov subspace method is used as a preconditioner. Compared to GMRES, there is no additional cost incurred in the arithmetic but the memory requirement almost doubles. Thus, GMRES is recommended if the right preconditioner N is constant.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nmemory: if restart = true, the restarted version FGMRES(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\nrestart: restart the method after memory iterations;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nY. Saad, A Flexible Inner-Outer Preconditioned GMRES Algorithm, SIAM Journal on Scientific Computing, Vol. 14(2), pp. 461–469, 1993.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.fgmres!","page":"Non-Hermitian square linear systems","title":"Krylov.fgmres!","text":"solver = fgmres!(solver::FgmresSolver, A, b; kwargs...)\nsolver = fgmres!(solver::FgmresSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of fgmres.\n\nNote that the memory keyword argument is the only exception. It's required to create a FgmresSolver and can't be changed later.\n\nSee FgmresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"examples/crmr/","page":"CRMR","title":"CRMR","text":"using Krylov, HarwellRutherfordBoeing, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"gemat1\")\npath = fetch_ssmc(matrix, format=\"RB\")\n\nA = RutherfordBoeingData(joinpath(path[1], \"$(matrix.name[1]).rb\")).data\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\nx_exact = A' * ones(m)\nx_exact_norm = norm(x_exact)\nx_exact /= x_exact_norm\nb = A * x_exact\n(x, stats) = crmr(A, b)\nshow(stats)\nresid = norm(A * x - b) / norm(b)\n@printf(\"CRMR: Relative residual: %7.1e\\n\", resid)\n@printf(\"CRMR: ‖x - x*‖₂: %7.1e\\n\", norm(x - x_exact))","category":"page"},{"location":"solvers/sid/","page":"Hermitian indefinite linear systems","title":"Hermitian indefinite linear systems","text":"# Hermitian indefinite linear systems","category":"page"},{"location":"solvers/sid/#SYMMLQ","page":"Hermitian indefinite linear systems","title":"SYMMLQ","text":"","category":"section"},{"location":"solvers/sid/","page":"Hermitian indefinite linear systems","title":"Hermitian indefinite linear systems","text":"symmlq\nsymmlq!","category":"page"},{"location":"solvers/sid/#Krylov.symmlq","page":"Hermitian indefinite linear systems","title":"Krylov.symmlq","text":"(x, stats) = symmlq(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, window::Int=5,\n transfer_to_cg::Bool=true, λ::T=zero(T),\n λest::T=zero(T), etol::T=√eps(T),\n conlim::T=1/√eps(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = symmlq(A, b, x0::AbstractVector; kwargs...)\n\nSYMMLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above\n\nSolve the shifted linear system\n\n(A + λI) x = b\n\nof size n using the SYMMLQ method, where λ is a shift parameter, and A is Hermitian.\n\nSYMMLQ produces monotonic errors ‖x* - x‖₂.\n\nInput arguments\n\nA: a linear operator that models a Hermitian matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nwindow: number of iterations used to accumulate a lower bound on the error;\ntransfer_to_cg: transfer from the SYMMLQ point to the CG point, when it exists. The transfer is based on the residual norm;\nλ: regularization parameter;\nλest: positive strict lower bound on the smallest eigenvalue λₘᵢₙ when solving a positive-definite system, such as λest = (1-10⁻⁷)λₘᵢₙ;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\netol: stopping tolerance based on the lower bound on the error;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SymmlqStats structure.\n\nReference\n\nC. C. Paige and M. A. Saunders, Solution of Sparse Indefinite Systems of Linear Equations, SIAM Journal on Numerical Analysis, 12(4), pp. 617–629, 1975.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#Krylov.symmlq!","page":"Hermitian indefinite linear systems","title":"Krylov.symmlq!","text":"solver = symmlq!(solver::SymmlqSolver, A, b; kwargs...)\nsolver = symmlq!(solver::SymmlqSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of symmlq.\n\nSee SymmlqSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#MINRES","page":"Hermitian indefinite linear systems","title":"MINRES","text":"","category":"section"},{"location":"solvers/sid/","page":"Hermitian indefinite linear systems","title":"Hermitian indefinite linear systems","text":"minres\nminres!","category":"page"},{"location":"solvers/sid/#Krylov.minres","page":"Hermitian indefinite linear systems","title":"Krylov.minres","text":"(x, stats) = minres(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, window::Int=5,\n λ::T=zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), etol::T=√eps(T),\n conlim::T=1/√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = minres(A, b, x0::AbstractVector; kwargs...)\n\nMINRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the shifted linear least-squares problem\n\nminimize ‖b - (A + λI)x‖₂²\n\nor the shifted linear system\n\n(A + λI) x = b\n\nof size n using the MINRES method, where λ ≥ 0 is a shift parameter, where A is Hermitian.\n\nMINRES is formally equivalent to applying CR to Ax=b when A is positive definite, but is typically more stable and also applies to the case where A is indefinite.\n\nMINRES produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂.\n\nInput arguments\n\nA: a linear operator that models a Hermitian matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nwindow: number of iterations used to accumulate a lower bound on the error;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\netol: stopping tolerance based on the lower bound on the error;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nC. C. Paige and M. A. Saunders, Solution of Sparse Indefinite Systems of Linear Equations, SIAM Journal on Numerical Analysis, 12(4), pp. 617–629, 1975.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#Krylov.minres!","page":"Hermitian indefinite linear systems","title":"Krylov.minres!","text":"solver = minres!(solver::MinresSolver, A, b; kwargs...)\nsolver = minres!(solver::MinresSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of minres.\n\nSee MinresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#MINRES-QLP","page":"Hermitian indefinite linear systems","title":"MINRES-QLP","text":"","category":"section"},{"location":"solvers/sid/","page":"Hermitian indefinite linear systems","title":"Hermitian indefinite linear systems","text":"minres_qlp\nminres_qlp!","category":"page"},{"location":"solvers/sid/#Krylov.minres_qlp","page":"Hermitian indefinite linear systems","title":"Krylov.minres_qlp","text":"(x, stats) = minres_qlp(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, Artol::T=√eps(T),\n λ::T=zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = minres_qlp(A, b, x0::AbstractVector; kwargs...)\n\nMINRES-QLP can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nMINRES-QLP is the only method based on the Lanczos process that returns the minimum-norm solution on singular inconsistent systems (A + λI)x = b of size n, where λ is a shift parameter. It is significantly more complex but can be more reliable than MINRES when A is ill-conditioned.\n\nM also indicates the weighted norm in which residuals are measured.\n\nInput arguments\n\nA: a linear operator that models a Hermitian matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nArtol: relative stopping tolerance based on the Aᴴ-residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nS.-C. T. Choi, Iterative methods for singular linear equations and least-squares problems, Ph.D. thesis, ICME, Stanford University, 2006.\nS.-C. T. Choi, C. C. Paige and M. A. Saunders, MINRES-QLP: A Krylov subspace method for indefinite or singular symmetric systems, SIAM Journal on Scientific Computing, Vol. 33(4), pp. 1810–1836, 2011.\nS.-C. T. Choi and M. A. Saunders, Algorithm 937: MINRES-QLP for symmetric and Hermitian linear equations and least-squares problems, ACM Transactions on Mathematical Software, 40(2), pp. 1–12, 2014.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#Krylov.minres_qlp!","page":"Hermitian indefinite linear systems","title":"Krylov.minres_qlp!","text":"solver = minres_qlp!(solver::MinresQlpSolver, A, b; kwargs...)\nsolver = minres_qlp!(solver::MinresQlpSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of minres_qlp.\n\nSee MinresQlpSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#MINARES","page":"Hermitian indefinite linear systems","title":"MINARES","text":"","category":"section"},{"location":"solvers/sid/","page":"Hermitian indefinite linear systems","title":"Hermitian indefinite linear systems","text":"minares\nminares!","category":"page"},{"location":"solvers/sid/#Krylov.minares","page":"Hermitian indefinite linear systems","title":"Krylov.minares","text":"(x, stats) = minares(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false,\n λ::T = zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), Artol::T = √eps(T),\n itmax::Int=0, timemax::Float64=Inf,\n verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = minares(A, b, x0::AbstractVector; kwargs...)\n\nMINARES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nMINARES solves the Hermitian linear system Ax = b of size n. MINARES minimizes ‖Arₖ‖₂ when M = Iₙ and ‖AMrₖ‖M otherwise. The estimates computed every iteration are ‖Mrₖ‖₂ and ‖AMrₖ‖M.\n\nInput arguments\n\nA: a linear operator that models a Hermitian positive definite matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nArtol: relative stopping tolerance based on the Aᴴ-residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nA. Montoison, D. Orban and M. A. Saunders, MinAres: An Iterative Solver for Symmetric Linear Systems, Cahier du GERAD G-2023-40, GERAD, Montréal, 2023.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#Krylov.minares!","page":"Hermitian indefinite linear systems","title":"Krylov.minares!","text":"solver = minares!(solver::MinaresSolver, A, b; kwargs...)\nsolver = minares!(solver::MinaresSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of minares.\n\nSee MinaresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"# Least-norm problems","category":"page"},{"location":"solvers/ln/#CGNE","page":"Least-norm problems","title":"CGNE","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"cgne\ncgne!","category":"page"},{"location":"solvers/ln/#Krylov.cgne","page":"Least-norm problems","title":"Krylov.cgne","text":"(x, stats) = cgne(A, b::AbstractVector{FC};\n N=I, ldiv::Bool=false,\n λ::T=zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the consistent linear system\n\nAx + √λs = b\n\nof size m × n using the Conjugate Gradient (CG) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CG to the normal equations of the second kind\n\n(AAᴴ + λI) y = b\n\nbut is more stable. When λ = 0, this method solves the minimum-norm problem\n\nmin ‖x‖₂ s.t. Ax = b.\n\nWhen λ > 0, it solves the problem\n\nmin ‖(x,s)‖₂ s.t. Ax + √λs = b.\n\nCGNE produces monotonic errors ‖x-x*‖₂ but not residuals ‖r‖₂. It is formally equivalent to CRAIG, though can be slightly less accurate, but simpler to implement. Only the x-part of the solution is returned.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nN: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nJ. E. Craig, The N-step iteration procedures, Journal of Mathematics and Physics, 34(1), pp. 64–73, 1955.\nJ. E. Craig, Iterations Procedures for Simultaneous Equations, Ph.D. Thesis, Department of Electrical Engineering, MIT, 1954.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.cgne!","page":"Least-norm problems","title":"Krylov.cgne!","text":"solver = cgne!(solver::CgneSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of cgne.\n\nSee CgneSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#CRMR","page":"Least-norm problems","title":"CRMR","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"crmr\ncrmr!","category":"page"},{"location":"solvers/ln/#Krylov.crmr","page":"Least-norm problems","title":"Krylov.crmr","text":"(x, stats) = crmr(A, b::AbstractVector{FC};\n N=I, ldiv::Bool=false,\n λ::T=zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the consistent linear system\n\nAx + √λs = b\n\nof size m × n using the Conjugate Residual (CR) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CR to the normal equations of the second kind\n\n(AAᴴ + λI) y = b\n\nbut is more stable. When λ = 0, this method solves the minimum-norm problem\n\nmin ‖x‖₂ s.t. x ∈ argmin ‖Ax - b‖₂.\n\nWhen λ > 0, this method solves the problem\n\nmin ‖(x,s)‖₂ s.t. Ax + √λs = b.\n\nCRMR produces monotonic residuals ‖r‖₂. It is formally equivalent to CRAIG-MR, though can be slightly less accurate, but simpler to implement. Only the x-part of the solution is returned.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nN: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nD. Orban and M. Arioli, Iterative Solution of Symmetric Quasi-Definite Linear Systems, Volume 3 of Spotlights. SIAM, Philadelphia, PA, 2017.\nD. Orban, The Projected Golub-Kahan Process for Constrained Linear Least-Squares Problems. Cahier du GERAD G-2014-15, 2014.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.crmr!","page":"Least-norm problems","title":"Krylov.crmr!","text":"solver = crmr!(solver::CrmrSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of crmr.\n\nSee CrmrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#LNLQ","page":"Least-norm problems","title":"LNLQ","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"lnlq\nlnlq!","category":"page"},{"location":"solvers/ln/#Krylov.lnlq","page":"Least-norm problems","title":"Krylov.lnlq","text":"(x, y, stats) = lnlq(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n transfer_to_craig::Bool=true,\n sqd::Bool=false, λ::T=zero(T),\n σ::T=zero(T), utolx::T=√eps(T),\n utoly::T=√eps(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nFind the least-norm solution of the consistent linear system\n\nAx + λ²y = b\n\nof size m × n using the LNLQ method, where λ ≥ 0 is a regularization parameter.\n\nFor a system in the form Ax = b, LNLQ method is equivalent to applying SYMMLQ to AAᴴy = b and recovering x = Aᴴy but is more stable. Note that y are the Lagrange multipliers of the least-norm problem\n\nminimize ‖x‖ s.t. Ax = b.\n\nIf λ > 0, LNLQ solves the symmetric and quasi-definite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A λ²E ] [ y ] = [ b ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nmin ‖x‖²_F + λ²‖y‖²_E s.t. Ax + λ²Ey = b.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LNLQ is then equivalent to applying SYMMLQ to (AF⁻¹Aᴴ + λ²E)y = b with Fx = Aᴴy.\n\nIf λ = 0, LNLQ solves the symmetric and indefinite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A 0 ] [ y ] = [ b ].\n\nThe system above represents the optimality conditions of\n\nminimize ‖x‖²_F s.t. Ax = b.\n\nIn this case, M can still be specified and indicates the weighted norm in which residuals are measured.\n\nIn this implementation, both the x and y-parts of the solution are returned.\n\nutolx and utoly are tolerances on the upper bound of the distance to the solution ‖x-x‖ and ‖y-y‖, respectively. The bound is valid if λ>0 or σ>0 where σ should be strictly smaller than the smallest positive singular value. For instance σ:=(1-1e-7)σₘᵢₙ .\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\ntransfer_to_craig: transfer from the LNLQ point to the CRAIG point, when it exists. The transfer is based on the residual norm;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\nσ: strict lower bound on the smallest positive singular value σₘᵢₙ such as σ = (1-10⁻⁷)σₘᵢₙ;\nutolx: tolerance on the upper bound on the distance to the solution ‖x-x*‖;\nutoly: tolerance on the upper bound on the distance to the solution ‖y-y*‖;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\ny: a dense vector of length m;\nstats: statistics collected on the run in a LNLQStats structure.\n\nReference\n\nR. Estrin, D. Orban, M.A. Saunders, LNLQ: An Iterative Method for Least-Norm Problems with an Error Minimization Property, SIAM Journal on Matrix Analysis and Applications, 40(3), pp. 1102–1124, 2019.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.lnlq!","page":"Least-norm problems","title":"Krylov.lnlq!","text":"solver = lnlq!(solver::LnlqSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of lnlq.\n\nSee LnlqSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#CRAIG","page":"Least-norm problems","title":"CRAIG","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"craig\ncraig!","category":"page"},{"location":"solvers/ln/#Krylov.craig","page":"Least-norm problems","title":"Krylov.craig","text":"(x, y, stats) = craig(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n transfer_to_lsqr::Bool=false, sqd::Bool=false,\n λ::T=zero(T), btol::T=√eps(T),\n conlim::T=1/√eps(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nFind the least-norm solution of the consistent linear system\n\nAx + λ²y = b\n\nof size m × n using the Golub-Kahan implementation of Craig's method, where λ ≥ 0 is a regularization parameter. This method is equivalent to CGNE but is more stable.\n\nFor a system in the form Ax = b, Craig's method is equivalent to applying CG to AAᴴy = b and recovering x = Aᴴy. Note that y are the Lagrange multipliers of the least-norm problem\n\nminimize ‖x‖ s.t. Ax = b.\n\nIf λ > 0, CRAIG solves the symmetric and quasi-definite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A λ²E ] [ y ] = [ b ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nmin ‖x‖²_F + λ²‖y‖²_E s.t. Ax + λ²Ey = b.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. CRAIG is then equivalent to applying CG to (AF⁻¹Aᴴ + λ²E)y = b with Fx = Aᴴy.\n\nIf λ = 0, CRAIG solves the symmetric and indefinite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A 0 ] [ y ] = [ b ].\n\nThe system above represents the optimality conditions of\n\nminimize ‖x‖²_F s.t. Ax = b.\n\nIn this case, M can still be specified and indicates the weighted norm in which residuals are measured.\n\nIn this implementation, both the x and y-parts of the solution are returned.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\ntransfer_to_lsqr: transfer from the LSLQ point to the LSQR point, when it exists. The transfer is based on the residual norm;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\nbtol: stopping tolerance used to detect zero-residual problems;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\ny: a dense vector of length m;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nJ. E. Craig, The N-step iteration procedures, Journal of Mathematics and Physics, 34(1-4), pp. 64–73, 1955.\nC. C. Paige and M. A. Saunders, LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares, ACM Transactions on Mathematical Software, 8(1), pp. 43–71, 1982.\nM. A. Saunders, Solutions of Sparse Rectangular Systems Using LSQR and CRAIG, BIT Numerical Mathematics, 35(4), pp. 588–604, 1995.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.craig!","page":"Least-norm problems","title":"Krylov.craig!","text":"solver = craig!(solver::CraigSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of craig.\n\nSee CraigSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#CRAIGMR","page":"Least-norm problems","title":"CRAIGMR","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"craigmr\ncraigmr!","category":"page"},{"location":"solvers/ln/#Krylov.craigmr","page":"Least-norm problems","title":"Krylov.craigmr","text":"(x, y, stats) = craigmr(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n sqd::Bool=false, λ::T=zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the consistent linear system\n\nAx + λ²y = b\n\nof size m × n using the CRAIGMR method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying the Conjugate Residuals method to the normal equations of the second kind\n\n(AAᴴ + λ²I) y = b\n\nbut is more stable. When λ = 0, this method solves the minimum-norm problem\n\nmin ‖x‖ s.t. x ∈ argmin ‖Ax - b‖.\n\nIf λ > 0, CRAIGMR solves the symmetric and quasi-definite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A λ²E ] [ y ] = [ b ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nmin ‖x‖²_F + λ²‖y‖²_E s.t. Ax + λ²Ey = b.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. CRAIGMR is then equivalent to applying MINRES to (AF⁻¹Aᴴ + λ²E)y = b with Fx = Aᴴy.\n\nIf λ = 0, CRAIGMR solves the symmetric and indefinite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A 0 ] [ y ] = [ b ].\n\nThe system above represents the optimality conditions of\n\nmin ‖x‖²_F s.t. Ax = b.\n\nIn this case, M can still be specified and indicates the weighted norm in which residuals are measured.\n\nCRAIGMR produces monotonic residuals ‖r‖₂. It is formally equivalent to CRMR, though can be slightly more accurate, and intricate to implement. Both the x- and y-parts of the solution are returned.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\ny: a dense vector of length m;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nD. Orban and M. Arioli. Iterative Solution of Symmetric Quasi-Definite Linear Systems, Volume 3 of Spotlights. SIAM, Philadelphia, PA, 2017.\nD. Orban, The Projected Golub-Kahan Process for Constrained, Linear Least-Squares Problems. Cahier du GERAD G-2014-15, 2014.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.craigmr!","page":"Least-norm problems","title":"Krylov.craigmr!","text":"solver = craigmr!(solver::CraigmrSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of craigmr.\n\nSee CraigmrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#USYMLQ","page":"Least-norm problems","title":"USYMLQ","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"usymlq\nusymlq!","category":"page"},{"location":"solvers/ln/#Krylov.usymlq","page":"Least-norm problems","title":"Krylov.usymlq","text":"(x, stats) = usymlq(A, b::AbstractVector{FC}, c::AbstractVector{FC};\n transfer_to_usymcg::Bool=true, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = usymlq(A, b, c, x0::AbstractVector; kwargs...)\n\nUSYMLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nUSYMLQ determines the least-norm solution of the consistent linear system Ax = b of size m × n.\n\nUSYMLQ is based on the orthogonal tridiagonalization process and requires two initial nonzero vectors b and c. The vector c is only used to initialize the process and a default value can be b or Aᴴb depending on the shape of A. The error norm ‖x - x*‖ monotonously decreases in USYMLQ. When A is Hermitian and b = c, USYMLQ is equivalent to SYMMLQ. USYMLQ is considered as a generalization of SYMMLQ.\n\nIt can also be applied to under-determined and over-determined problems. In all cases, problems must be consistent.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nc: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\ntransfer_to_usymcg: transfer from the USYMLQ point to the USYMCG point, when it exists. The transfer is based on the residual norm;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nM. A. Saunders, H. D. Simon, and E. L. Yip, Two Conjugate-Gradient-Type Methods for Unsymmetric Linear Equations, SIAM Journal on Numerical Analysis, 25(4), pp. 927–940, 1988.\nA. Buttari, D. Orban, D. Ruiz and D. Titley-Peloquin, A tridiagonalization method for symmetric saddle-point and quasi-definite systems, SIAM Journal on Scientific Computing, 41(5), pp. 409–432, 2019.\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.usymlq!","page":"Least-norm problems","title":"Krylov.usymlq!","text":"solver = usymlq!(solver::UsymlqSolver, A, b, c; kwargs...)\nsolver = usymlq!(solver::UsymlqSolver, A, b, c, x0; kwargs...)\n\nwhere kwargs are keyword arguments of usymlq.\n\nSee UsymlqSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"","category":"page"},{"location":"processes/#krylov-processes","page":"Krylov processes","title":"Krylov processes","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Krylov processes are the foundation of Krylov methods, they generate bases of Krylov subspaces. Depending on the Krylov subspaces generated, Krylov processes are more or less specialized for a subset of linear problems. The following table summarizes the most relevant processes for each linear problem.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Linear problems Processes\nHermitian linear systems Hermitian Lanczos\nSquare Non-Hermitian linear systems Non-Hermitian Lanczos – Arnoldi\nLeast-squares problems Golub-Kahan – Saunders-Simon-Yip\nLeast-norm problems Golub-Kahan – Saunders-Simon-Yip\nSaddle-point and Hermitian quasi-definite systems Golub-Kahan – Saunders-Simon-Yip\nGeneralized saddle-point and non-Hermitian partitioned systems Montoison-Orban","category":"page"},{"location":"processes/#Notation","page":"Krylov processes","title":"Notation","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"For a matrix A, A^H denotes the conjugate transpose of A. It coincides with A^T, the transpose of A, for real matrices. Define V_k = beginbmatrix v_1 ldots v_k endbmatrix enspace and enspace U_k = beginbmatrix u_1 ldots u_k endbmatrix.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"For a matrix C in mathbbC^n times n and a vector t in mathbbC^n, the k-th Krylov subspace generated by C and t is","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"mathcalK_k(C t) =\nleftsum_i=0^k-1 omega_i C^i t middle vert omega_i in mathbbC0 le i le k-1 right","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"For matrices C in mathbbC^n times n enspace and enspace T in mathbbC^n times p, the k-th block Krylov subspace generated by C and T is","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"mathcalK_k^square(C T) =\nleftsum_i=0^k-1 C^i T Omega_i middle vert Omega_i in mathbbC^p times p0 le i le k-1 right","category":"page"},{"location":"processes/#hermitian-lanczos","page":"Krylov processes","title":"Hermitian Lanczos","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: hermitian_lanczos)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the Hermitian Lanczos process, the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A V_k = V_k T_k + beta_k+1k v_k+1 e_k^T = V_k+1 T_k+1k \n V_k^H V_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where V_k is an orthonormal basis of the Krylov subspace mathcalK_k(Ab),","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"T_k =\nbeginbmatrix\n alpha_1 barbeta_2 \n beta_2 alpha_2 ddots \n ddots ddots barbeta_k \n beta_k alpha_k\nendbmatrix\n qquad\nT_k+1k =\nbeginbmatrix\n T_k \n beta_k+1 e_k^T\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Note that depending on how we normalize the vectors that compose V_k, T_k+1k can be a real tridiagonal matrix even if A is a complex matrix.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function hermitian_lanczos returns V_k+1, beta_1 and T_k+1k.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: SYMMLQ, CG, CR, CAR, MINRES, MINRES-QLP, MINARES, CGLS, CGLS-LANCZOS-SHIFT, CRLS, CGNE, CRMR, CG-LANCZOS and CG-LANCZOS-SHIFT.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"hermitian_lanczos(::Any, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.hermitian_lanczos-Union{Tuple{FC}, Tuple{Any, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.hermitian_lanczos","text":"V, β, T = hermitian_lanczos(A, b, k)\n\nInput arguments\n\nA: a linear operator that models an Hermitian matrix of dimension n;\nb: a vector of length n;\nk: the number of iterations of the Hermitian Lanczos process.\n\nOutput arguments\n\nV: a dense n × (k+1) matrix;\nβ: a coefficient such that βv₁ = b;\nT: a sparse (k+1) × k tridiagonal matrix.\n\nReference\n\nC. Lanczos, An Iteration Method for the Solution of the Eigenvalue Problem of Linear Differential and Integral Operators, Journal of Research of the National Bureau of Standards, 45(4), pp. 225–280, 1950.\n\n\n\n\n\n","category":"method"},{"location":"processes/#nonhermitian-lanczos","page":"Krylov processes","title":"Non-Hermitian Lanczos","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: nonhermitian_lanczos)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the non-Hermitian Lanczos process (also named the Lanczos biorthogonalization process), the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A V_k = V_k T_k + beta_k+1 v_k+1 e_k^T = V_k+1 T_k+1k \n A^H U_k = U_k T_k^H + bargamma_k+1 u_k+1 e_k^T = U_k+1 T_kk+1^H \n V_k^H U_k = U_k^H V_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where V_k and U_k are bases of the Krylov subspaces mathcalK_k (Ab) and mathcalK_k (A^Hc), respectively,","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"T_k = \nbeginbmatrix\n alpha_1 gamma_2 \n beta_2 alpha_2 ddots \n ddots ddots gamma_k \n beta_k alpha_k\nendbmatrix\n qquad\nT_k+1k =\nbeginbmatrix\n T_k \n beta_k+1 e_k^T\nendbmatrix\n qquad\nT_kk+1 =\nbeginbmatrix\n T_k gamma_k+1 e_k\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function nonhermitian_lanczos returns V_k+1, beta_1, T_k+1k, U_k+1, bargamma_1 and T_kk+1^H.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: BiLQ, QMR, BiLQR, CGS and BICGSTAB.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"note: Note\nThe scaling factors used in our implementation are beta_k = u_k^H v_k^tfrac12 and gamma_k = (u_k^H v_k) beta_k. With these scaling factors, the non-Hermitian Lanczos process coincides with the Hermitian Lanczos process when A = A^H and b = c.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"nonhermitian_lanczos(::Any, ::AbstractVector{FC}, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.nonhermitian_lanczos-Union{Tuple{FC}, Tuple{Any, AbstractVector{FC}, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.nonhermitian_lanczos","text":"V, β, T, U, γᴴ, Tᴴ = nonhermitian_lanczos(A, b, c, k)\n\nInput arguments\n\nA: a linear operator that models a square matrix of dimension n;\nb: a vector of length n;\nc: a vector of length n;\nk: the number of iterations of the non-Hermitian Lanczos process.\n\nOutput arguments\n\nV: a dense n × (k+1) matrix;\nβ: a coefficient such that βv₁ = b;\nT: a sparse (k+1) × k tridiagonal matrix;\nU: a dense n × (k+1) matrix;\nγᴴ: a coefficient such that γᴴu₁ = c;\nTᴴ: a sparse (k+1) × k tridiagonal matrix.\n\nReference\n\nC. Lanczos, An Iteration Method for the Solution of the Eigenvalue Problem of Linear Differential and Integral Operators, Journal of Research of the National Bureau of Standards, 45(4), pp. 225–280, 1950.\n\n\n\n\n\n","category":"method"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The non-Hermitian Lanczos process can be also implemented without A^H (transpose-free variant). To derive it, we can observe that beta_k+1 v_k+1 = P_k(A) b and bargamma_k+1 u_k+1 = Q_k(A^H) c where P_k and Q_k are polynomials of degree k. The polynomials are defined from the recursions:","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n P_0(A) = I_n \n P_1(A) = left(dfracA - alpha_1 I_nbeta_1right) P_0(A) \n P_k(A) = left(dfracA - alpha_k I_nbeta_kright) P_k-1(A) - dfracgamma_kbeta_k-1 P_k-2(A) quad k ge 2 \n \n Q_0(A^H) = I_n \n Q_1(A^H) = left(dfracA^H - baralpha_1 I_nbargamma_1right) Q_0(A^H) \n Q_k(A^H) = left(dfracA^H - baralpha_k I_nbargamma_kright) Q_k-1(A^H) - dfracbarbeta_kbargamma_k-1 Q_k-2(A^H) quad k ge 2\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Because alpha_k = u_k^H A v_k and (bargamma_k+1 u_k+1)^H (beta_k+1 v_k+1) = gamma_k+1 beta_k+1, we can determine the coefficients of T_k+1k and T_kk+1^H as follows:","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n alpha_k = dfrac1gamma_k beta_k langleQ_k-1(A^H) c A P_k-1(A) brangle \n = dfrac1gamma_k beta_k langlec barQ_k-1(A) A P_k-1(A) brangle \n \n beta_k+1 gamma_k+1 = langleQ_k(A^H) c P_k(A) brangle \n = langlec barQ_k(A) P_k(A) brangle\nendalign*","category":"page"},{"location":"processes/#arnoldi","page":"Krylov processes","title":"Arnoldi","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: arnoldi)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the Arnoldi process, the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A V_k = V_k H_k + h_k+1k v_k+1 e_k^T = V_k+1 H_k+1k \n V_k^H V_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where V_k is an orthonormal basis of the Krylov subspace mathcalK_k (Ab),","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"H_k =\nbeginbmatrix\n h_11 h_12 ldots h_1k \n h_21 ddots ddots vdots \n ddots ddots h_k-1k \n h_kk-1 h_kk\nendbmatrix\n qquad\nH_k+1k =\nbeginbmatrix\n H_k \n h_k+1k e_k^T\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function arnoldi returns V_k+1, beta and H_k+1k.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: DIOM, FOM, DQGMRES, GMRES and FGMRES.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"note: Note\nThe Arnoldi process coincides with the Hermitian Lanczos process when A is Hermitian.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"arnoldi(::Any, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.arnoldi-Union{Tuple{FC}, Tuple{Any, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.arnoldi","text":"V, β, H = arnoldi(A, b, k; reorthogonalization=false)\n\nInput arguments\n\nA: a linear operator that models a square matrix of dimension n;\nb: a vector of length n;\nk: the number of iterations of the Arnoldi process.\n\nKeyword argument\n\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors.\n\nOutput arguments\n\nV: a dense n × (k+1) matrix;\nβ: a coefficient such that βv₁ = b;\nH: a dense (k+1) × k upper Hessenberg matrix.\n\nReference\n\nW. E. Arnoldi, The principle of minimized iterations in the solution of the matrix eigenvalue problem, Quarterly of Applied Mathematics, 9, pp. 17–29, 1951.\n\n\n\n\n\n","category":"method"},{"location":"processes/#golub-kahan","page":"Krylov processes","title":"Golub-Kahan","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: golub_kahan)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the Golub-Kahan bidiagonalization process, the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A V_k = U_k+1 B_k \n A^H U_k+1 = V_k B_k^H + baralpha_k+1 v_k+1 e_k+1^T = V_k+1 L_k+1^H \n V_k^H V_k = U_k^H U_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where V_k and U_k are bases of the Krylov subspaces mathcalK_k (A^HAA^Hb) and mathcalK_k (AA^Hb), respectively,","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"L_k =\nbeginbmatrix\n alpha_1 \n beta_2 alpha_2 \n ddots ddots \n beta_k alpha_k\nendbmatrix\n qquad\nB_k =\nbeginbmatrix\n alpha_1 \n beta_2 alpha_2 \n ddots ddots \n beta_k alpha_k \n beta_k+1 \nendbmatrix\n=\nbeginbmatrix\n L_k \n beta_k+1 e_k^T\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Note that depending on how we normalize the vectors that compose V_k and U_k, L_k can be a real bidiagonal matrix even if A is a complex matrix.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function golub_kahan returns V_k+1, U_k+1, beta_1 and L_k+1.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: LNLQ, CRAIG, CRAIGMR, LSLQ, LSQR and LSMR.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"note: Note\nThe Golub-Kahan process coincides with the Hermitian Lanczos process applied to the normal equations A^HA x = A^Hb and AA^H x = b. It is also related to the Hermitian Lanczos process applied to beginbmatrix 0 A A^H 0 endbmatrix with initial vector beginbmatrix b 0 endbmatrix.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"golub_kahan(::Any, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.golub_kahan-Union{Tuple{FC}, Tuple{Any, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.golub_kahan","text":"V, U, β, L = golub_kahan(A, b, k)\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nk: the number of iterations of the Golub-Kahan process.\n\nOutput arguments\n\nV: a dense n × (k+1) matrix;\nU: a dense m × (k+1) matrix;\nβ: a coefficient such that βu₁ = b;\nL: a sparse (k+1) × (k+1) lower bidiagonal matrix.\n\nReferences\n\nG. H. Golub and W. Kahan, Calculating the Singular Values and Pseudo-Inverse of a Matrix, SIAM Journal on Numerical Analysis, 2(2), pp. 225–224, 1965.\nC. C. Paige, Bidiagonalization of Matrices and Solution of Linear Equations, SIAM Journal on Numerical Analysis, 11(1), pp. 197–209, 1974.\n\n\n\n\n\n","category":"method"},{"location":"processes/#saunders-simon-yip","page":"Krylov processes","title":"Saunders-Simon-Yip","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: saunders_simon_yip)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the Saunders-Simon-Yip process (also named the orthogonal tridiagonalization process), the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A U_k = V_k T_k + beta_k+1 v_k+1 e_k^T = V_k+1 T_k+1k \n A^H V_k = U_k T_k^H + bargamma_k+1 u_k+1 e_k^T = U_k+1 T_kk+1^H \n V_k^H V_k = U_k^H U_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where beginbmatrix V_k 0 0 U_k endbmatrix is an orthonormal basis of the block Krylov subspace mathcalK^square_k left(beginbmatrix 0 A A^H 0 endbmatrix beginbmatrix b 0 0 c endbmatrixright),","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"T_k = \nbeginbmatrix\n alpha_1 gamma_2 \n beta_2 alpha_2 ddots \n ddots ddots gamma_k \n beta_k alpha_k\nendbmatrix\n qquad\nT_k+1k =\nbeginbmatrix\n T_k \n beta_k+1 e_k^T\nendbmatrix\n qquad\nT_kk+1 =\nbeginbmatrix\n T_k gamma_k+1 e_k\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function saunders_simon_yip returns V_k+1, beta_1, T_k+1k, U_k+1, bargamma_1 and T_kk+1^H.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: USYMLQ, USYMQR, TriLQR, TriCG and TriMR.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"note: Note\nThe Saunders-Simon-Yip is equivalent to the block-Lanczos process applied to beginbmatrix 0 A A^H 0 endbmatrix with initial matrix beginbmatrix b 0 0 c endbmatrix.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"saunders_simon_yip(::Any, ::AbstractVector{FC}, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.saunders_simon_yip-Union{Tuple{FC}, Tuple{Any, AbstractVector{FC}, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.saunders_simon_yip","text":"V, β, T, U, γᴴ, Tᴴ = saunders_simon_yip(A, b, c, k)\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nc: a vector of length n;\nk: the number of iterations of the Saunders-Simon-Yip process.\n\nOutput arguments\n\nV: a dense m × (k+1) matrix;\nβ: a coefficient such that βv₁ = b;\nT: a sparse (k+1) × k tridiagonal matrix;\nU: a dense n × (k+1) matrix;\nγᴴ: a coefficient such that γᴴu₁ = c;\nTᴴ: a sparse (k+1) × k tridiagonal matrix.\n\nReference\n\nM. A. Saunders, H. D. Simon, and E. L. Yip, Two Conjugate-Gradient-Type Methods for Unsymmetric Linear Equations, SIAM Journal on Numerical Analysis, 25(4), pp. 927–940, 1988.\n\n\n\n\n\n","category":"method"},{"location":"processes/#montoison-orban","page":"Krylov processes","title":"Montoison-Orban","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: montoison_orban)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the Montoison-Orban process (also named the orthogonal Hessenberg reduction process), the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A U_k = V_k H_k + h_k+1k v_k+1 e_k^T = V_k+1 H_k+1k \n B V_k = U_k F_k + f_k+1k u_k+1 e_k^T = U_k+1 F_k+1k \n V_k^H V_k = U_k^H U_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where beginbmatrix V_k 0 0 U_k endbmatrix is an orthonormal basis of the block Krylov subspace mathcalK^square_k left(beginbmatrix 0 A B 0 endbmatrix beginbmatrix b 0 0 c endbmatrixright),","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"H_k =\nbeginbmatrix\n h_11 h_12 ldots h_1k \n h_21 ddots ddots vdots \n ddots ddots h_k-1k \n h_kk-1 h_kk\nendbmatrix\n qquad\nF_k =\nbeginbmatrix\n f_11 f_12 ldots f_1k \n f_21 ddots ddots vdots \n ddots ddots f_k-1k \n f_kk-1 f_kk\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"H_k+1k =\nbeginbmatrix\n H_k \n h_k+1k e_k^T\nendbmatrix\n qquad\nF_k+1k =\nbeginbmatrix\n F_k \n f_k+1k e_k^T\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function montoison_orban returns V_k+1, beta, H_k+1k, U_k+1, gamma and F_k+1k.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: GPMR.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"note: Note\nThe Montoison-Orban is equivalent to the block-Arnoldi process applied to beginbmatrix 0 A B 0 endbmatrix with initial matrix beginbmatrix b 0 0 c endbmatrix. It also coincides with the Saunders-Simon-Yip process when B = A^H.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"montoison_orban(::Any, ::Any, ::AbstractVector{FC}, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.montoison_orban-Union{Tuple{FC}, Tuple{Any, Any, AbstractVector{FC}, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.montoison_orban","text":"V, β, H, U, γ, F = montoison_orban(A, B, b, c, k; reorthogonalization=false)\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nB: a linear operator that models a matrix of dimension n × m;\nb: a vector of length m;\nc: a vector of length n;\nk: the number of iterations of the Montoison-Orban process.\n\nKeyword argument\n\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors.\n\nOutput arguments\n\nV: a dense m × (k+1) matrix;\nβ: a coefficient such that βv₁ = b;\nH: a dense (k+1) × k upper Hessenberg matrix;\nU: a dense n × (k+1) matrix;\nγ: a coefficient such that γu₁ = c;\nF: a dense (k+1) × k upper Hessenberg matrix.\n\nReference\n\nA. Montoison and D. Orban, GPMR: An Iterative Method for Unsymmetric Partitioned Linear Systems, SIAM Journal on Matrix Analysis and Applications, 44(1), pp. 293–311, 2023.\n\n\n\n\n\n","category":"method"},{"location":"examples/crls/","page":"CRLS","title":"CRLS","text":"using MatrixMarket, SuiteSparseMatrixCollection\nusing Krylov, LinearOperators\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"well1850\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1])_b.mtx\"))[:]\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\n# Define a regularization parameter.\nλ = 1.0e-3\n\n(x, stats) = crls(A, b, λ=λ)\nshow(stats)\nresid = norm(A' * (A * x - b) + λ * x) / norm(b)\n@printf(\"CRLS: Relative residual: %8.1e\\n\", resid)\n@printf(\"CRLS: ‖x‖: %8.1e\\n\", norm(x))","category":"page"},{"location":"callbacks/#callbacks","page":"Callbacks","title":"Callbacks","text":"","category":"section"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"Each Krylov method is able to call a callback function as callback(solver) at each iteration. The callback should return true if the main loop should terminate, and false otherwise. If the method terminated because of the callback, the output status will be \"user-requested exit\". For example, if the user defines minres_callback(solver::MinresSolver), it can be passed to the solver using","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"(x, stats) = minres(A, b, callback = minres_callback)","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"If you need to write a callback that uses variables that are not in a KrylovSolver, use a closure:","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"function custom_stopping_condition(solver::KrylovSolver, A, b, r, tol)\n mul!(r, A, solver.x)\n r .-= b # r := b - Ax\n bool = norm(r) ≤ tol # tolerance based on the 2-norm of the residual\n return bool\nend\n\ncg_callback(solver) = custom_stopping_condition(solver, A, b, r, tol)\n(x, stats) = cg(A, b, callback = cg_callback)","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"Alternatively, use a structure and make it callable:","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"mutable struct CallbackWorkspace{T}\n A::Matrix{T}\n b::Vector{T}\n r::Vector{T}\n tol::T\nend\n\nfunction (workspace::CallbackWorkspace)(solver::KrylovSolver)\n mul!(workspace.r, workspace.A, solver.x)\n workspace.r .-= workspace.b\n bool = norm(workspace.r) ≤ workspace.tol\n return bool\nend\n\nbicgstab_callback = CallbackWorkspace(A, b, r, tol)\n(x, stats) = bicgstab(A, b, callback = bicgstab_callback)","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"Although the main goal of a callback is to add new stopping conditions, it can also retrieve information from the workspace of a Krylov method along the iterations. We now illustrate how to store all iterates x_k of the GMRES method.","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"S = Krylov.ktypeof(b)\nglobal X = S[] # Storage for GMRES iterates\n\nfunction gmres_callback(solver)\n z = solver.z\n k = solver.inner_iter\n nr = sum(1:k)\n V = solver.V\n R = solver.R\n y = copy(z)\n\n # Solve Rk * yk = zk\n for i = k : -1 : 1\n pos = nr + i - k\n for j = k : -1 : i+1\n y[i] = y[i] - R[pos] * y[j]\n pos = pos - j + 1\n end\n y[i] = y[i] / R[pos]\n end\n\n # xk = Vk * yk\n xk = sum(V[i] * y[i] for i = 1:k)\n push!(X, xk)\n\n return false # We don't want to add new stopping conditions\nend\n\n(x, stats) = gmres(A, b, callback = gmres_callback)","category":"page"},{"location":"gpu/#gpu","page":"GPU support","title":"GPU support","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"Krylov methods are well suited for GPU computations because they only require matrix-vector products (u leftarrow Av, u leftarrow A^Hw) and vector operations (v, u^H v, v leftarrow alpha u + beta v), which are highly parallelizable.","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"The implementations in Krylov.jl are generic so as to take advantage of the multiple dispatch and broadcast features of Julia. Those allow the implementations to be specialized automatically by the compiler for both CPU and GPU. Thus, Krylov.jl works with GPU backends that build on GPUArrays.jl, such as CUDA.jl, AMDGPU.jl, oneAPI.jl or Metal.jl.","category":"page"},{"location":"gpu/#Nvidia-GPUs","page":"GPU support","title":"Nvidia GPUs","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"All solvers in Krylov.jl can be used with CUDA.jl and allow computations on Nvidia GPUs. Problems stored in CPU format (Matrix and Vector) must first be converted to the related GPU format (CuMatrix and CuVector).","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using CUDA, Krylov\n\nif CUDA.functional()\n # CPU Arrays\n A_cpu = rand(20, 20)\n b_cpu = rand(20)\n\n # GPU Arrays\n A_gpu = CuMatrix(A_cpu)\n b_gpu = CuVector(b_cpu)\n\n # Solve a square and dense system on an Nivida GPU\n x, stats = bilq(A_gpu, b_gpu)\nend","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"Sparse matrices have a specific storage on Nvidia GPUs (CuSparseMatrixCSC, CuSparseMatrixCSR or CuSparseMatrixCOO):","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using CUDA, Krylov\nusing CUDA.CUSPARSE, SparseArrays\n\nif CUDA.functional()\n # CPU Arrays\n A_cpu = sprand(200, 100, 0.3)\n b_cpu = rand(200)\n\n # GPU Arrays\n A_csc_gpu = CuSparseMatrixCSC(A_cpu)\n A_csr_gpu = CuSparseMatrixCSR(A_cpu)\n A_coo_gpu = CuSparseMatrixCOO(A_cpu)\n b_gpu = CuVector(b_cpu)\n\n # Solve a rectangular and sparse system on an Nvidia GPU\n x_csc, stats_csc = lslq(A_csc_gpu, b_gpu)\n x_csr, stats_csr = lsqr(A_csr_gpu, b_gpu)\n x_coo, stats_coo = lsmr(A_coo_gpu, b_gpu)\nend","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"If you use a Krylov method that only requires A * v products (see here), the most efficient format is CuSparseMatrixCSR. Optimized operator-vector and operator-matrix products that exploit GPU features can be also used by means of linear operators.","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"For instance, when executing sparse matrix products on NVIDIA GPUs, the mul! function utilized by Krylov.jl internally calls three routines from CUSPARSE. The first one conducts an analysis of the sparse matrix's structure, the second one determines the size of the buffer that needs to be allocated, and the last one handles the product using the allocated buffer. To circumvent the need for repeated analysis computations and buffer allocation/deallocation with each product, we introduce a KrylovOperator in KrylovPreconditioners.jl. This operator performs the analysis and allocates the buffer only once.","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using Krylov, KrylovPreconditioners\n\n# A_gpu can be a sparse COO, CSC or CSR matrix\nopA_gpu = KrylovOperator(A_gpu)\nx_gpu, stats = gmres(opA_gpu, b_gpu)","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"Preconditioners, especially incomplete Cholesky or incomplete LU factorizations that involve sparse triangular solves, can be applied directly on GPU thanks to efficient operators (like TriangularOperator) that take advantage of CUSPARSE routines.","category":"page"},{"location":"gpu/#Example-with-a-symmetric-positive-definite-system","page":"GPU support","title":"Example with a symmetric positive-definite system","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using SparseArrays, Krylov, LinearOperators\nusing CUDA, CUDA.CUSPARSE\n\nif CUDA.functional()\n # Transfer the linear system from the CPU to the GPU\n A_gpu = CuSparseMatrixCSR(A_cpu) # A_gpu = CuSparseMatrixCSC(A_cpu)\n b_gpu = CuVector(b_cpu)\n\n # IC(0) decomposition LLᴴ ≈ A for CuSparseMatrixCSC or CuSparseMatrixCSR matrices\n P = ic02(A_gpu)\n\n # Additional vector required for solving triangular systems\n n = length(b_gpu)\n T = eltype(b_gpu)\n z = CUDA.zeros(T, n)\n\n # Solve Py = x\n function ldiv_ic0!(P::CuSparseMatrixCSR, x, y, z)\n ldiv!(z, LowerTriangular(P), x) # Forward substitution with L\n ldiv!(y, LowerTriangular(P)', z) # Backward substitution with Lᴴ\n return y\n end\n\n function ldiv_ic0!(P::CuSparseMatrixCSC, x, y, z)\n ldiv!(z, UpperTriangular(P)', x) # Forward substitution with L\n ldiv!(y, UpperTriangular(P), z) # Backward substitution with Lᴴ\n return y\n end\n\n # Operator that model P⁻¹\n symmetric = hermitian = true\n opM = LinearOperator(T, n, n, symmetric, hermitian, (y, x) -> ldiv_ic0!(P, x, y, z))\n\n # Solve an Hermitian positive definite system with an IC(0) preconditioner on GPU\n x, stats = cg(A_gpu, b_gpu, M=opM)\nend","category":"page"},{"location":"gpu/#Example-with-a-general-square-system","page":"GPU support","title":"Example with a general square system","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using SparseArrays, Krylov, LinearOperators\nusing CUDA, CUDA.CUSPARSE, CUDA.CUSOLVER\n\nif CUDA.functional()\n # Optional -- Compute a permutation vector p such that A[:,p] has no zero diagonal\n p = zfd(A_cpu)\n p .+= 1\n A_cpu = A_cpu[:,p]\n\n # Transfer the linear system from the CPU to the GPU\n A_gpu = CuSparseMatrixCSR(A_cpu) # A_gpu = CuSparseMatrixCSC(A_cpu)\n b_gpu = CuVector(b_cpu)\n\n # ILU(0) decomposition LU ≈ A for CuSparseMatrixCSC or CuSparseMatrixCSR matrices\n P = ilu02(A_gpu)\n\n # Additional vector required for solving triangular systems\n n = length(b_gpu)\n T = eltype(b_gpu)\n z = CUDA.zeros(T, n)\n\n # Solve Py = x\n function ldiv_ilu0!(P::CuSparseMatrixCSR, x, y, z)\n ldiv!(z, UnitLowerTriangular(P), x) # Forward substitution with L\n ldiv!(y, UpperTriangular(P), z) # Backward substitution with U\n return y\n end\n\n function ldiv_ilu0!(P::CuSparseMatrixCSC, x, y, z)\n ldiv!(z, LowerTriangular(P), x) # Forward substitution with L\n ldiv!(y, UnitUpperTriangular(P), z) # Backward substitution with U\n return y\n end\n\n # Operator that model P⁻¹\n symmetric = hermitian = false\n opM = LinearOperator(T, n, n, symmetric, hermitian, (y, x) -> ldiv_ilu0!(P, x, y, z))\n\n # Solve a non-Hermitian system with an ILU(0) preconditioner on GPU\n x̄, stats = bicgstab(A_gpu, b_gpu, M=opM)\n\n # Recover the solution of Ax = b with the solution of A[:,p]x̄ = b\n invp = invperm(p)\n x = x̄[invp]\nend","category":"page"},{"location":"gpu/#AMD-GPUs","page":"GPU support","title":"AMD GPUs","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"All solvers in Krylov.jl can be used with AMDGPU.jl and allow computations on AMD GPUs. Problems stored in CPU format (Matrix and Vector) must first be converted to the related GPU format (ROCMatrix and ROCVector).","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using Krylov, AMDGPU\n\nif AMDGPU.functional()\n # CPU Arrays\n A_cpu = rand(ComplexF64, 20, 20)\n A_cpu = A_cpu + A_cpu'\n b_cpu = rand(ComplexF64, 20)\n\n A_gpu = ROCMatrix(A_cpu)\n b_gpu = ROCVector(b_cpu)\n\n # Solve a dense Hermitian system on an AMD GPU\n x, stats = minres(A_gpu, b_gpu)\nend","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"Sparse matrices have a specific storage on AMD GPUs (ROCSparseMatrixCSC, ROCSparseMatrixCSR or ROCSparseMatrixCOO):","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using AMDGPU, Krylov\nusing AMDGPU.rocSPARSE, SparseArrays\n\nif AMDGPU.functional()\n # CPU Arrays\n A_cpu = sprand(100, 200, 0.3)\n b_cpu = rand(100)\n\n # GPU Arrays\n A_csc_gpu = ROCSparseMatrixCSC(A_cpu)\n A_csr_gpu = ROCSparseMatrixCSR(A_cpu)\n A_coo_gpu = ROCSparseMatrixCOO(A_cpu)\n b_gpu = ROCVector(b_cpu)\n\n # Solve a rectangular and sparse system on an AMD GPU\n x_csc, y_csc, stats_csc = lnlq(A_csc_gpu, b_gpu)\n x_csr, y_csr, stats_csr = craig(A_csr_gpu, b_gpu)\n x_coo, y_coo, stats_coo = craigmr(A_coo_gpu, b_gpu)\nend","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"When executing sparse products on AMD GPUs, the mul! function utilized by Krylov.jl internally calls three routines from rocSPARSE. The first one conducts an analysis of the sparse matrix's structure, the second one determines the size of the buffer that needs to be allocated, and the last one handles the product using the allocated buffer. To circumvent the need for repeated analysis computations and buffer allocation/deallocation with each product, we introduce a KrylovOperator in KrylovPreconditioners.jl. This operator performs the analysis and allocates the buffer only once.","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using Krylov, KrylovPreconditioners\n\n# A_gpu can be a sparse COO, CSC or CSR matrix\nopA_gpu = KrylovOperator(A_gpu)\nx_gpu, stats = bicgstab(opA_gpu, b_gpu)","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"Preconditioners, especially incomplete Cholesky or incomplete LU factorizations that involve sparse triangular solves, can be applied directly on GPU thanks to efficient operators (like TriangularOperator) that take advantage of rocSPARSE routines.","category":"page"},{"location":"gpu/#Intel-GPUs","page":"GPU support","title":"Intel GPUs","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"All solvers in Krylov.jl can be used with oneAPI.jl and allow computations on Intel GPUs. Problems stored in CPU format (Matrix and Vector) must first be converted to the related GPU format (oneMatrix and oneVector).","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using Krylov, oneAPI\n\nif oneAPI.functional()\n T = Float32 # oneAPI.jl also works with ComplexF32\n m = 20\n n = 10\n\n # CPU Arrays\n A_cpu = rand(T, m, n)\n b_cpu = rand(T, m)\n\n # GPU Arrays\n A_gpu = oneMatrix(A_cpu)\n b_gpu = oneVector(b_cpu)\n\n # Solve a dense least-squares problem on an Intel GPU\n x, stats = lsqr(A_gpu, b_gpu)\nend","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"note: Note\nThe library oneMKL is interfaced in oneAPI.jl and accelerates linear algebra operations on Intel GPUs. Only dense linear systems are supported for the time being because sparse linear algebra routines are not interfaced yet.","category":"page"},{"location":"gpu/#Apple-M1-GPUs","page":"GPU support","title":"Apple M1 GPUs","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"All solvers in Krylov.jl can be used with Metal.jl and allow computations on Apple M1 GPUs. Problems stored in CPU format (Matrix and Vector) must first be converted to the related GPU format (MtlMatrix and MtlVector).","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using Krylov, Metal\n\nT = Float32 # Metal.jl also works with ComplexF32\nn = 10\nm = 20\n\n# CPU Arrays\nA_cpu = rand(T, n, m)\nb_cpu = rand(T, n)\n\n# GPU Arrays\nA_gpu = MtlMatrix(A_cpu)\nb_gpu = MtlVector(b_cpu)\n\n# Solve a dense least-norm problem on an Apple M1 GPU\nx, stats = craig(A_gpu, b_gpu)","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"warning: Warning\nMetal.jl is under heavy development and is considered experimental for now.","category":"page"},{"location":"#Home","page":"Home","title":"Krylov.jl documentation","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"This package provides implementations of certain of the most useful Krylov method for a variety of problems:","category":"page"},{"location":"","page":"Home","title":"Home","text":"1 - Square or rectangular full-rank systems","category":"page"},{"location":"","page":"Home","title":"Home","text":" Ax = b","category":"page"},{"location":"","page":"Home","title":"Home","text":"should be solved when b lies in the range space of A. This situation occurs when","category":"page"},{"location":"","page":"Home","title":"Home","text":"A is square and nonsingular,\nA is tall and has full column rank and b lies in the range of A.","category":"page"},{"location":"","page":"Home","title":"Home","text":"2 - Linear least-squares problems","category":"page"},{"location":"","page":"Home","title":"Home","text":" min b - Ax","category":"page"},{"location":"","page":"Home","title":"Home","text":"should be solved when b is not in the range of A (inconsistent systems), regardless of the shape and rank of A. This situation mainly occurs when","category":"page"},{"location":"","page":"Home","title":"Home","text":"A is square and singular,\nA is tall and thin.","category":"page"},{"location":"","page":"Home","title":"Home","text":"Underdetermined systems are less common but also occur.","category":"page"},{"location":"","page":"Home","title":"Home","text":"If there are infinitely many such x (because A is column rank-deficient), one with minimum norm is identified","category":"page"},{"location":"","page":"Home","title":"Home","text":" min x quad textsubject to quad x in argmin b - Ax","category":"page"},{"location":"","page":"Home","title":"Home","text":"3 - Linear least-norm problems","category":"page"},{"location":"","page":"Home","title":"Home","text":" min x quad textsubject to quad Ax = b","category":"page"},{"location":"","page":"Home","title":"Home","text":"should be solved when A is column rank-deficient but b is in the range of A (consistent systems), regardless of the shape of A. This situation mainly occurs when","category":"page"},{"location":"","page":"Home","title":"Home","text":"A is square and singular,\nA is short and wide.","category":"page"},{"location":"","page":"Home","title":"Home","text":"Overdetermined systems are less common but also occur.","category":"page"},{"location":"","page":"Home","title":"Home","text":"4 - Adjoint systems","category":"page"},{"location":"","page":"Home","title":"Home","text":" Ax = b quad textand quad A^H y = c","category":"page"},{"location":"","page":"Home","title":"Home","text":"where A can have any shape.","category":"page"},{"location":"","page":"Home","title":"Home","text":"5 - Saddle-point and Hermitian quasi-definite systems","category":"page"},{"location":"","page":"Home","title":"Home","text":" beginbmatrix M phantom-A A^H -N endbmatrix beginbmatrix x y endbmatrix = left(beginbmatrix b 0 endbmatrixbeginbmatrix 0 c endbmatrixbeginbmatrix b c endbmatrixright)","category":"page"},{"location":"","page":"Home","title":"Home","text":"where A can have any shape.","category":"page"},{"location":"","page":"Home","title":"Home","text":"6 - Generalized saddle-point and non-Hermitian partitioned systems","category":"page"},{"location":"","page":"Home","title":"Home","text":" beginbmatrix M A B N endbmatrix beginbmatrix x y endbmatrix = beginbmatrix b c endbmatrix","category":"page"},{"location":"","page":"Home","title":"Home","text":"where A can have any shape and B has the shape of Aᴴ. A, B, b and c must be all nonzero.","category":"page"},{"location":"","page":"Home","title":"Home","text":"Krylov solvers are particularly appropriate in situations where such problems must be solved but a factorization is not possible, either because:","category":"page"},{"location":"","page":"Home","title":"Home","text":"A is not available explicitly,\nA would be dense or would consume an excessive amount of memory if it were materialized,\nfactors would consume an excessive amount of memory.","category":"page"},{"location":"","page":"Home","title":"Home","text":"Iterative methods are recommended in either of the following situations:","category":"page"},{"location":"","page":"Home","title":"Home","text":"the problem is sufficiently large that a factorization is not feasible or would be slow,\nan effective preconditioner is known in cases where the problem has unfavorable spectral structure,\nthe operator can be represented efficiently as a sparse matrix,\nthe operator is fast, i.e., can be applied with better complexity than if it were materialized as a matrix. Certain fast operators would materialize as dense matrices.","category":"page"},{"location":"#Features","page":"Home","title":"Features","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"All solvers in Krylov.jl have in-place version, are compatible with GPU and work in any floating-point data type.","category":"page"},{"location":"#How-to-Install","page":"Home","title":"How to Install","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Krylov can be installed and tested through the Julia package manager:","category":"page"},{"location":"","page":"Home","title":"Home","text":"julia> ]\npkg> add Krylov\npkg> test Krylov","category":"page"},{"location":"#Bug-reports-and-discussions","page":"Home","title":"Bug reports and discussions","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"If you think you found a bug, feel free to open an issue. Focused suggestions and requests can also be opened as issues. Before opening a pull request, start an issue or a discussion on the topic, please.","category":"page"},{"location":"","page":"Home","title":"Home","text":"If you want to ask a question not suited for a bug report, feel free to start a discussion here. This forum is for general discussion about this repository and the JuliaSmoothOptimizers organization, so questions about any of our packages are welcome.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"","category":"page"},{"location":"matrix_free/#matrix-free","page":"Matrix-free operators","title":"Matrix-free operators","text":"","category":"section"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"All methods are matrix-free, which means that you only need to provide operator-vector products.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"The A or B input arguments of Krylov.jl solvers can be any object that represents a linear operator. That object must implement mul!, for multiplication with a vector, size() and eltype(). For certain methods it must also implement adjoint().","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"Some methods only require A * v products, whereas other ones also require A' * u products. In the latter case, adjoint(A) must also be implemented.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"A * v A * v and A' * u\nCG, CR, CAR CGLS, CRLS, CGNE, CRMR\nSYMMLQ, CG-LANCZOS, MINRES, MINRES-QLP, MINARES LSLQ, LSQR, LSMR, LNLQ, CRAIG, CRAIGMR\nDIOM, FOM, DQGMRES, GMRES, FGMRES, BLOCK-GMRES BiLQ, QMR, BiLQR, USYMLQ, USYMQR, TriLQR\nCGS, BICGSTAB TriCG, TriMR\nCG-LANCZOS-SHIFT CGLS-LANCZOS-SHIFT","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"info: Info\nGPMR is the only method that requires A * v and B * w products.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"Preconditioners M, N, C, D, E or F can be also linear operators and must implement mul! or ldiv!.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"We strongly recommend LinearOperators.jl to model matrix-free operators, but other packages such as LinearMaps.jl, DiffEqOperators.jl or your own operator can be used as well.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"With LinearOperators.jl, operators are defined as","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"A = LinearOperator(type, nrows, ncols, symmetric, hermitian, prod, tprod, ctprod)","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"where","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"type is the operator element type;\nnrow and ncol are its dimensions;\nsymmetric and hermitian should be set to true or false;\nprod(y, v), tprod(y, w) and ctprod(u, w) are called when writing mul!(y, A, v), mul!(y, transpose(A), w), and mul!(y, A', u), respectively.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"See the tutorial and the detailed documentation for more information on LinearOperators.jl.","category":"page"},{"location":"matrix_free/#Examples-with-automatic-differentiation","page":"Matrix-free operators","title":"Examples with automatic differentiation","text":"","category":"section"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"In the field of nonlinear optimization, finding critical points of a continuous function frequently involves linear systems with a Hessian or Jacobian as coefficient. Materializing such operators as matrices is expensive in terms of operations and memory consumption and is unreasonable for high-dimensional problems. However, it is often possible to implement efficient Hessian-vector and Jacobian-vector products, for example with the help of automatic differentiation tools, and used within Krylov solvers. We now illustrate variants with explicit matrices and with matrix-free operators for two well-known optimization methods.","category":"page"},{"location":"matrix_free/#Example-1:-Newton's-Method-for-convex-optimization","page":"Matrix-free operators","title":"Example 1: Newton's Method for convex optimization","text":"","category":"section"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"At each iteration of Newton's method applied to a mathcalC^2 strictly convex function f mathbbR^n rightarrow mathbbR, a descent direction direction is determined by minimizing the quadratic Taylor model of f:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"min_d in mathbbR^nf(x_k) + nabla f(x_k)^T d + tfrac12d^T nabla^2 f(x_k) d","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"which is equivalent to solving the symmetric and positive-definite system","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"nabla^2 f(x_k) d = -nabla f(x_k)","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"The system above can be solved with the conjugate gradient method as follows, using the explicit Hessian:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"using ForwardDiff, Krylov\n\nxk = -ones(4)\n\nf(x) = (x[1] - 1)^2 + (x[2] - 2)^2 + (x[3] - 3)^2 + (x[4] - 4)^2\n\ng(x) = ForwardDiff.gradient(f, x)\n\nH(x) = ForwardDiff.hessian(f, x)\n\nd, stats = cg(H(xk), -g(xk))","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"The explicit Hessian can be replaced by a linear operator that only computes Hessian-vector products:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"using ForwardDiff, LinearOperators, Krylov\n\nxk = -ones(4)\n\nf(x) = (x[1] - 1)^2 + (x[2] - 2)^2 + (x[3] - 3)^2 + (x[4] - 4)^2\n\ng(x) = ForwardDiff.gradient(f, x)\n\nH(y, v) = ForwardDiff.derivative!(y, t -> g(xk + t * v), 0)\nopH = LinearOperator(Float64, 4, 4, true, true, (y, v) -> H(y, v))\n\ncg(opH, -g(xk))","category":"page"},{"location":"matrix_free/#Example-2:-The-Gauss-Newton-Method-for-Nonlinear-Least-Squares","page":"Matrix-free operators","title":"Example 2: The Gauss-Newton Method for Nonlinear Least Squares","text":"","category":"section"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"At each iteration of the Gauss-Newton method applied to a nonlinear least-squares objective f(x) = tfrac12 F(x)^2 where F mathbbR^n rightarrow mathbbR^m is mathcalC^1, we solve the subproblem:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"min_d in mathbbR^ntfrac12J(x_k) d + F(x_k)^2","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"where J(x) is the Jacobian of F at x.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"An appropriate iterative method to solve the above linear least-squares problems is LSMR. We could pass the explicit Jacobian to LSMR as follows:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"using ForwardDiff, Krylov\n\nxk = ones(2)\n\nF(x) = [x[1]^4 - 3; exp(x[2]) - 2; log(x[1]) - x[2]^2]\n\nJ(x) = ForwardDiff.jacobian(F, x)\n\nd, stats = lsmr(J(xk), -F(xk))","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"However, the explicit Jacobian can be replaced by a linear operator that only computes Jacobian-vector and transposed Jacobian-vector products:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"using LinearAlgebra, ForwardDiff, LinearOperators, Krylov\n\nxk = ones(2)\n\nF(x) = [x[1]^4 - 3; exp(x[2]) - 2; log(x[1]) - x[2]^2]\n\nJ(y, v) = ForwardDiff.derivative!(y, t -> F(xk + t * v), 0)\nJᵀ(y, u) = ForwardDiff.gradient!(y, x -> dot(F(x), u), xk)\nopJ = LinearOperator(Float64, 3, 2, false, false, (y, v) -> J(y, v),\n (y, w) -> Jᵀ(y, w),\n (y, u) -> Jᵀ(y, u))\n\nlsmr(opJ, -F(xk))","category":"page"},{"location":"matrix_free/#Example-with-FFT-and-IFFT","page":"Matrix-free operators","title":"Example with FFT and IFFT","text":"","category":"section"},{"location":"matrix_free/#Example-3:-Solving-the-Poisson-equation-with-FFT-and-IFFT","page":"Matrix-free operators","title":"Example 3: Solving the Poisson equation with FFT and IFFT","text":"","category":"section"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"In applications related to partial differential equations (PDEs), linear systems can arise from discretizing differential operators. Storing such operators as explicit matrices is computationally expensive and unnecessary when matrix-free methods can be used, particularly with structured grids.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"The FFT is an algorithm that computes the discrete Fourier transform (DFT) of a sequence, transforming data from the spatial domain to the frequency domain. In the context of solving PDEs, it simplifies the application of differential operators like the Laplacian by converting derivatives into algebraic operations.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"For a function u(x) discretized on a periodic grid with n points, the FFT of u is:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"hatu_k = sum_j=0^n-1 u_j e^-i k x_j","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"where hatu_k represents the Fourier coefficients for the frequency k, and u_j is the value of u at the grid point x_j defined as x_j = frac2 pi jL with period L. The inverse FFT (IFFT) reconstructs u from its Fourier coefficients:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"u_j = frac1n sum_k=0^n-1 hatu_k e^i k x_j","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"In Fourier space, the Laplacian operator fracd^2dx^2 becomes a simple multiplication by -k^2, where k is the wavenumber derived from the grid size. This transforms the Poisson equation fracd^2 u(x)dx^2 = f(x) into an algebraic equation in the frequency domain:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"-k^2 hatu_k = hatf_k","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"By solving for hatu_k and applying the IFFT, we can recover the solution u(x) efficiently.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"The inverse FFT is used to convert data from the frequency domain back to the spatial domain. Once the solution in frequency space is obtained by dividing the Fourier coefficients hatf_k by -k^2 for k neq 0, the IFFT is applied to transform the result back to the original grid points in the spatial domain. At k = 0, the equation -k^2 hatu_0 = hatf_0 becomes indeterminate since k^2 = 0. This situation corresponds to the zero-frequency component hatf_0, which represents the mean of f(x). In such cases, hatu_0 is treated separately. It is typically set to 0 to remove the constant mode, or adjusted based on boundary conditions or other constraints.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"In some cases, even though the FFT provides an efficient way to apply differential operators (such as the Laplacian) in the frequency domain, a direct solution may not be feasible due to complex boundary conditions, variable coefficients, or grid irregularities. In these situations, the FFT must be coupled with a Krylov method to iteratively solve the problem.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"This example consists of solving the 1D Poisson equation on a periodic domain 0 4pi:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"fracd^2 u(x)dx^2 = f(x)","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"where u(x) is the unknown solution, and f(x) is the given source term. We solve this equation using FFTW.jl to compute the matrix-free action of the Laplacian within the conjugate gradient solver.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"Note that while a direct FFT-based approach can be used here due to the simplicity of the periodic boundary conditions, this example illustrates how a Krylov method can be employed to solve more challenging problems.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"using FFTW, Krylov, LinearAlgebra\n\n# Define the problem size and domain\nn = 32768 # Number of grid points (2^15)\nL = 4π # Length of the domain\nx = LinRange(0, L, n+1)[1:end-1] # Periodic grid (excluding the last point)\n\n# Define the source term f(x)\nf = sin.(x)\n\n# Define a matrix-free operator using FFT and IFFT\nstruct FFTPoissonOperator\n n::Int\n L::Float64\n complex::Bool\n k::Vector{Float64} # Store Fourier wave numbers\nend\n\nfunction FFTPoissonOperator(n::Int, L::Float64, complex::Bool)\n if complex\n k = Vector{Float64}(undef, n)\n else\n k = Vector{Float64}(undef, n÷2 + 1)\n end\n k[1] = sum(f) / n # average value of f(x) over the domain\n for j in 1:(n÷2)\n k[j+1] = 2 * π * j / L # Positive wave numbers\n end\n if complex\n for j in 1:(n÷2 - 1)\n k[n-j+1] = -2 * π * j / L # Negative wave numbers\n end\n end\n return FFTPoissonOperator(n, L, complex, k)\nend\n\nBase.size(A::FFTPoissonOperator) = (n, n)\n\nfunction Base.eltype(A::FFTPoissonOperator)\n type = A.complex ? ComplexF64 : Float64\n return type\nend\n\nfunction LinearAlgebra.mul!(y::Vector, A::FFTPoissonOperator, u::Vector)\n # Transform the input vector `u` to the frequency domain using `fft` or `rfft`.\n # If the operator is complex, use the full FFT; otherwise, use the real FFT.\n if A.complex\n u_hat = fft(u)\n else\n u_hat = rfft(u)\n end\n\n # In Fourier space, solve the system by multiplying with -k^2 (corresponding to the second derivative).\n # This step applies the Laplacian operator in the frequency domain.\n u_hat .= -u_hat .* (A.k .^ 2)\n\n # Transform the result back to the spatial domain using `ifft` or `irfft`.\n # If the operator is complex, use the full inverse FFT; otherwise, use the inverse real FFT.\n if A.complex\n y .= ifft(u_hat)\n else\n y .= irfft(u_hat, A.n)\n end\n\n return y\nend\n\n# Create the matrix-free operator for the Poisson equation\ncomplex = false\nA = FFTPoissonOperator(n, L, complex)\n\n# Solve the linear system using CG\nu_sol, stats = cg(A, f, atol=1e-10, rtol=0.0, verbose=1)\n\n# The exact solution is u(x) = -sin(x)\nu_star = -sin.(x)\nu_star ≈ u_sol","category":"page"},{"location":"matrix_free/#Example-with-PDE","page":"Matrix-free operators","title":"Example with PDE","text":"","category":"section"},{"location":"matrix_free/#Example-4:-Solving-the-3D-Helmholtz-equation","page":"Matrix-free operators","title":"Example 4: Solving the 3D Helmholtz equation","text":"","category":"section"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"The Helmholtz equation in 3D is a fundamental equation used in various fields like acoustics, electromagnetism, and quantum mechanics to model stationary wave phenomena. It is given by:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"nabla^2 u(xyz) + k^2 u(xyz) = f(xyz)","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"where:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"u(x y z)\nis the unknown function representing, for example, a pressure field in acoustics, a scalar potential in electromagnetism, or a wave function in quantum mechanics,\nnabla^2\nis the Laplacian operator in three dimensions,\nk\nis the wave number, related to the frequency of the wave by k = frac2pilambda, where lambda is the wavelength,\nf(xyz)\nis a source term that drives the wave phenomena, such as a forcing function or external influence.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"To discretize the Helmholtz equation, we use finite differences on a uniform 3D grid with grid spacings Delta x, Delta y, and Delta z. For a grid point (i j k), the second derivatives are approximated as follows:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"In the x-direction:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"fracpartial^2 upartial x^2 approx fracu_i+1jk - 2u_ijk + u_i-1jkDelta x^2","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"In the y-direction:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"fracpartial^2 upartial y^2 approx fracu_ij+1k - 2u_ijk + u_ij-1kDelta y^2","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"In the z-direction:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"fracpartial^2 upartial z^2 approx fracu_ijk+1 - 2u_ijk + u_ijk-1Delta z^2","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"Combining these, the discretized Helmholtz equation becomes:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"fracu_i+1jk - 2u_ijk + u_i-1jkDelta x^2 + fracu_ij+1k - 2u_ijk + u_ij-1kDelta y^2 + fracu_ijk+1 - 2u_ijk + u_ijk-1Delta z^2 + k^2 u_ijk = f_ijk","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"This discretization results in an equation at each grid point, resulting in a large and sparse linear system when assembled across the entire 3D grid. To simplify the example, we impose Dirichlet boundary conditions with the solution u(x y z) = 0 on the boundary of the cubic domain.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"Explicitly constructing this large sparse matrix is often impractical and unnecessary. Instead, we can define a function that directly applies the Helmholtz operator to the 3D grid, avoiding the need to form the matrix explicitly.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"Krylov.jl operates on vectors, so we must vectorize both the solution and the computational domain. However, we can still maintain the structure of the original 3D operator by using reshape and vec. This approach enables a simpler and efficient application of the operator in 3D while leveraging the vectorized framework for linear algebra operations.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"using Krylov, LinearAlgebra\n\n# Parameters\nL = 1.0 # Length of the cubic domain\nNx = 200 # Number of interior grid points in x\nNy = 200 # Number of interior grid points in y\nNz = 200 # Number of interior grid points in z\nΔx = L / (Nx + 1) # Grid spacing in x\nΔy = L / (Ny + 1) # Grid spacing in y\nΔz = L / (Nz + 1) # Grid spacing in z\nwavelength = 0.5 # Wavelength of the wave\nk = 2 * π / wavelength # Wave number\n\n# Create the grid points\nx = 0:Δx:L # Points in x dimension (Nx + 2)\ny = 0:Δy:L # Points in y dimension (Ny + 2)\nz = 0:Δz:L # Points in z dimension (Nz + 2)\n\n# Define a matrix-free Helmholtz operator\nstruct HelmholtzOperator\n Nx::Int\n Ny::Int\n Nz::Int\n Δx::Float64\n Δy::Float64\n Δz::Float64\n k::Float64\nend\n\nBase.size(A::HelmholtzOperator) = (A.Nx * A.Ny * A.Nz, A.Nx * A.Ny * A.Nz)\n\nBase.eltype(A::HelmholtzOperator) = Float64\n\nfunction LinearAlgebra.mul!(y::Vector, A::HelmholtzOperator, u::Vector)\n # Reshape vectors y and u into 3D arrays\n U = reshape(u, A.Nx, A.Ny, A.Nz)\n Y = reshape(y, A.Nx, A.Ny, A.Nz)\n\n # Apply the discrete Laplacian in 3D with k^2 * u\n for i in 1:A.Nx\n for j in 1:A.Ny\n for k in 1:A.Nz\n if i == 1\n dx2 = (U[i+1,j,k] -2 * U[i,j,k]) / (A.Δx)^2\n elseif i == A.Nx\n dx2 = (-2 * U[i,j,k] + U[i-1,j,k]) / (A.Δx)^2\n else\n dx2 = (U[i+1,j,k] -2 * U[i,j,k] + U[i-1,j,k]) / (A.Δx)^2\n end\n\n if j == 1\n dy2 = (U[i,j+1,k] -2 * U[i,j,k]) / (A.Δy)^2\n elseif j == A.Ny\n dy2 = (-2 * U[i,j,k] + U[i,j-1,k]) / (A.Δy)^2\n else\n dy2 = (U[i,j+1,k] -2 * U[i,j,k] + U[i,j-1,k]) / (A.Δy)^2\n end\n\n if k == 1\n dz2 = (U[i,j,k+1] -2 * U[i,j,k]) / (A.Δz)^2\n elseif k == A.Nz\n dz2 = (-2 * U[i,j,k] + U[i,j,k-1]) / (A.Δz)^2\n else\n dz2 = (U[i,j,k+1] -2 * U[i,j,k] + U[i,j,k-1]) / (A.Δz)^2\n end\n\n Y[i,j,k] = dx2 + dy2 + dz2 + (A.k)^2 * U[i,j,k]\n end\n end\n end\n\n return y\nend\n\n# Create the matrix-free operator for the Helmholtz equation\nA = HelmholtzOperator(Nx, Ny, Nz, Δx, Δy, Δz, k)\n\n# Source term f(x, y, z) = -2k² * sin(kx) * sin(ky) * sin(kz)\nF = [-2 * k^2 * sin(k * x[ii+1]) * sin(k * y[jj+1]) * sin(k * z[kk+1]) for ii in 1:Nx, jj in 1:Ny, kk in 1:Nz]\nf = vec(F)\n\n# Solve the linear system using MinAres\nu_sol, stats = minares(A, f, atol=1e-10, rtol=0.0, verbose=1)\n\n# Solution as 3D array\nU_sol = reshape(u_sol, Nx, Ny, Nz)\n\n# Exact solution u(x,y,z) = sin(kx) * sin(ky) * sin(kz)\nU_star = [sin(k * x[ii+1]) * sin(k * y[jj+1]) * sin(k * z[kk+1]) for ii in 1:Nx, jj in 1:Ny, kk in 1:Nz]\n\n# Compute the maximum error between the numerical solution U_sol and the exact solution U_star\nnorm(U_sol - U_star, Inf)","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"Note that preconditioners can be also implemented as abstract operators. For instance, we could compute the Cholesky factorization of M and N and create linear operators that perform the forward and backsolves.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"Krylov methods combined with factorization free operators allow to reduce computation time and memory requirements considerably by avoiding building and storing the system matrix. In the field of partial differential equations, the implementation of high-performance factorization free operators and assembly free preconditioning is a subject of active research.","category":"page"},{"location":"examples/cgls/","page":"CGLS","title":"CGLS","text":"using MatrixMarket, SuiteSparseMatrixCollection\nusing Krylov, LinearOperators\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"well1033\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1])_b.mtx\"))[:]\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\n# Define a regularization parameter.\nλ = 1.0e-3\n\n(x, stats) = cgls(A, b, λ=λ)\nshow(stats)\nresid = norm(A' * (A * x - b) + λ * x) / norm(b)\n@printf(\"CGLS: Relative residual: %8.1e\\n\", resid)\n@printf(\"CGLS: ‖x‖: %8.1e\\n\", norm(x))","category":"page"}] +[{"location":"examples/minares/","page":"MINARES","title":"MINARES","text":"using Krylov, MatrixMarket, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"GHS_indef\", \"laser\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nn = matrix.nrows[1]\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = ones(n)\n\n# Solve Ax = b.\nx, stats = minares(A, b)\nshow(stats)\nr = b - A * x\nAr = A * r\n@printf(\"Relative A-residual: %8.1e\\n\", norm(A * r) / norm(A * b))","category":"page"},{"location":"examples/trimr/","page":"TriMR","title":"TriMR","text":"using Krylov, LinearOperators, LDLFactorizations\nusing LinearAlgebra, Printf, SparseArrays\n\n# Identity matrix.\neye(n::Int) = sparse(1.0 * I, n, n)\n\n# Saddle-point systems\nn = m = 5\nA = [2^(i/j)*j + (-1)^(i-j) * n*(i-1) for i = 1:n, j = 1:n]\nb = ones(n)\nD = diagm(0 => [2.0 * i for i = 1:n])\nm, n = size(A)\nc = -b\n\n# [D A] [x] = [b]\n# [Aᴴ 0] [y] [c]\nllt_D = cholesky(D)\nopD⁻¹ = LinearOperator(Float64, 5, 5, true, true, (y, v) -> ldiv!(y, llt_D, v))\nopH⁻¹ = BlockDiagonalOperator(opD⁻¹, eye(n))\n(x, y, stats) = trimr(A, b, c, M=opD⁻¹, sp=true)\nK = [D A; A' zeros(n,n)]\nB = [b; c]\nr = B - K * [x; y]\nresid = sqrt(dot(r, opH⁻¹ * r))\n@printf(\"TriMR: Relative residual: %8.1e\\n\", resid)\n\n# Symmetric quasi-definite systems\nn = m = 5\nA = [2^(i/j)*j + (-1)^(i-j) * n*(i-1) for i = 1:n, j = 1:n]\nb = ones(n)\nM = diagm(0 => [3.0 * i for i = 1:n])\nN = diagm(0 => [5.0 * i for i = 1:n])\nc = -b\n\n# [I A] [x] = [b]\n# [Aᴴ -I] [y] [c]\n(x, y, stats) = trimr(A, b, c)\nK = [eye(m) A; A' -eye(n)]\nB = [b; c]\nr = B - K * [x; y]\nresid = norm(r)\n@printf(\"TriMR: Relative residual: %8.1e\\n\", resid)\n\n# [M A] [x] = [b]\n# [Aᴴ -N] [y] [c]\nldlt_M = ldl(M)\nldlt_N = ldl(N)\nopM⁻¹ = LinearOperator(Float64, size(M,1), size(M,2), true, true, (y, v) -> ldiv!(y, ldlt_M, v))\nopN⁻¹ = LinearOperator(Float64, size(N,1), size(N,2), true, true, (y, v) -> ldiv!(y, ldlt_N, v))\nopH⁻¹ = BlockDiagonalOperator(opM⁻¹, opN⁻¹)\n(x, y, stats) = trimr(A, b, c, M=opM⁻¹, N=opN⁻¹, verbose=1)\nK = [M A; A' -N]\nB = [b; c]\nr = B - K * [x; y]\nresid = sqrt(dot(r, opH⁻¹ * r))\n@printf(\"TriMR: Relative residual: %8.1e\\n\", resid)","category":"page"},{"location":"solvers/spd/","page":"Hermitian positive definite linear systems","title":"Hermitian positive definite linear systems","text":"# Hermitian positive definite linear systems","category":"page"},{"location":"solvers/spd/#CG","page":"Hermitian positive definite linear systems","title":"CG","text":"","category":"section"},{"location":"solvers/spd/","page":"Hermitian positive definite linear systems","title":"Hermitian positive definite linear systems","text":"cg\ncg!","category":"page"},{"location":"solvers/spd/#Krylov.cg","page":"Hermitian positive definite linear systems","title":"Krylov.cg","text":"(x, stats) = cg(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, radius::T=zero(T),\n linesearch::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = cg(A, b, x0::AbstractVector; kwargs...)\n\nCG can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nThe conjugate gradient method to solve the Hermitian linear system Ax = b of size n.\n\nThe method does not abort if A is not definite. M also indicates the weighted norm in which residuals are measured.\n\nInput arguments\n\nA: a linear operator that models a Hermitian positive definite matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nradius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;\nlinesearch: if true, indicate that the solution is to be used in an inexact Newton method with linesearch. If negative curvature is detected at iteration k > 0, the solution of iteration k-1 is returned. If negative curvature is detected at iteration 0, the right-hand side is returned (i.e., the negative gradient);\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nM. R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, Journal of Research of the National Bureau of Standards, 49(6), pp. 409–436, 1952.\n\n\n\n\n\n","category":"function"},{"location":"solvers/spd/#Krylov.cg!","page":"Hermitian positive definite linear systems","title":"Krylov.cg!","text":"solver = cg!(solver::CgSolver, A, b; kwargs...)\nsolver = cg!(solver::CgSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of cg.\n\nSee CgSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/spd/#CR","page":"Hermitian positive definite linear systems","title":"CR","text":"","category":"section"},{"location":"solvers/spd/","page":"Hermitian positive definite linear systems","title":"Hermitian positive definite linear systems","text":"cr\ncr!","category":"page"},{"location":"solvers/spd/#Krylov.cr","page":"Hermitian positive definite linear systems","title":"Krylov.cr","text":"(x, stats) = cr(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, radius::T=zero(T),\n linesearch::Bool=false, γ::T=√eps(T),\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = cr(A, b, x0::AbstractVector; kwargs...)\n\nCR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nA truncated version of Stiefel’s Conjugate Residual method to solve the Hermitian linear system Ax = b of size n or the least-squares problem min ‖b - Ax‖ if A is singular. The matrix A must be Hermitian semi-definite. M also indicates the weighted norm in which residuals are measured.\n\nInput arguments\n\nA: a linear operator that models a Hermitian positive definite matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nradius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;\nlinesearch: if true, indicate that the solution is to be used in an inexact Newton method with linesearch. If negative curvature is detected at iteration k > 0, the solution of iteration k-1 is returned. If negative curvature is detected at iteration 0, the right-hand side is returned (i.e., the negative gradient);\nγ: tolerance to determine that the curvature of the quadratic model is nonpositive;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nM. R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, Journal of Research of the National Bureau of Standards, 49(6), pp. 409–436, 1952.\nE. Stiefel, Relaxationsmethoden bester Strategie zur Losung linearer Gleichungssysteme, Commentarii Mathematici Helvetici, 29(1), pp. 157–179, 1955.\nD. G. Luenberger, The conjugate residual method for constrained minimization problems, SIAM Journal on Numerical Analysis, 7(3), pp. 390–398, 1970.\nM-A. Dahito and D. Orban, The Conjugate Residual Method in Linesearch and Trust-Region Methods, SIAM Journal on Optimization, 29(3), pp. 1988–2025, 2019.\n\n\n\n\n\n","category":"function"},{"location":"solvers/spd/#Krylov.cr!","page":"Hermitian positive definite linear systems","title":"Krylov.cr!","text":"solver = cr!(solver::CrSolver, A, b; kwargs...)\nsolver = cr!(solver::CrSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of cr.\n\nSee CrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/spd/#CAR","page":"Hermitian positive definite linear systems","title":"CAR","text":"","category":"section"},{"location":"solvers/spd/","page":"Hermitian positive definite linear systems","title":"Hermitian positive definite linear systems","text":"car\ncar!","category":"page"},{"location":"solvers/spd/#Krylov.car","page":"Hermitian positive definite linear systems","title":"Krylov.car","text":"(x, stats) = car(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false,\n atol::T=√eps(T), rtol::T=√eps(T),\n itmax::Int=0, timemax::Float64=Inf,\n verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = car(A, b, x0::AbstractVector; kwargs...)\n\nCAR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nCAR solves the Hermitian and positive definite linear system Ax = b of size n. CAR minimizes ‖Arₖ‖₂ when M = Iₙ and ‖AMrₖ‖M otherwise. The estimates computed every iteration are ‖Mrₖ‖₂ and ‖AMrₖ‖M.\n\nInput arguments\n\nA: a linear operator that models a Hermitian positive definite matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nA. Montoison, D. Orban and M. A. Saunders, MinAres: An Iterative Solver for Symmetric Linear Systems, Cahier du GERAD G-2023-40, GERAD, Montréal, 2023.\n\n\n\n\n\n","category":"function"},{"location":"solvers/spd/#Krylov.car!","page":"Hermitian positive definite linear systems","title":"Krylov.car!","text":"solver = car!(solver::CarSolver, A, b; kwargs...)\nsolver = car!(solver::CarSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of car.\n\nSee CarSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/spd/#CG-LANCZOS","page":"Hermitian positive definite linear systems","title":"CG-LANCZOS","text":"","category":"section"},{"location":"solvers/spd/","page":"Hermitian positive definite linear systems","title":"Hermitian positive definite linear systems","text":"cg_lanczos\ncg_lanczos!","category":"page"},{"location":"solvers/spd/#Krylov.cg_lanczos","page":"Hermitian positive definite linear systems","title":"Krylov.cg_lanczos","text":"(x, stats) = cg_lanczos(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false,\n check_curvature::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = cg_lanczos(A, b, x0::AbstractVector; kwargs...)\n\nCG-LANCZOS can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nThe Lanczos version of the conjugate gradient method to solve the Hermitian linear system Ax = b of size n.\n\nThe method does not abort if A is not definite.\n\nInput arguments\n\nA: a linear operator that models a Hermitian matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\ncheck_curvature: if true, check that the curvature of the quadratic along the search direction is positive, and abort if not, unless linesearch is also true;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a LanczosStats structure.\n\nReferences\n\nA. Frommer and P. Maass, Fast CG-Based Methods for Tikhonov-Phillips Regularization, SIAM Journal on Scientific Computing, 20(5), pp. 1831–1850, 1999.\nC. C. Paige and M. A. Saunders, Solution of Sparse Indefinite Systems of Linear Equations, SIAM Journal on Numerical Analysis, 12(4), pp. 617–629, 1975.\n\n\n\n\n\n","category":"function"},{"location":"solvers/spd/#Krylov.cg_lanczos!","page":"Hermitian positive definite linear systems","title":"Krylov.cg_lanczos!","text":"solver = cg_lanczos!(solver::CgLanczosSolver, A, b; kwargs...)\nsolver = cg_lanczos!(solver::CgLanczosSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of cg_lanczos.\n\nSee CgLanczosSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/spd/#CG-LANCZOS-SHIFT","page":"Hermitian positive definite linear systems","title":"CG-LANCZOS-SHIFT","text":"","category":"section"},{"location":"solvers/spd/","page":"Hermitian positive definite linear systems","title":"Hermitian positive definite linear systems","text":"cg_lanczos_shift\ncg_lanczos_shift!","category":"page"},{"location":"solvers/spd/#Krylov.cg_lanczos_shift","page":"Hermitian positive definite linear systems","title":"Krylov.cg_lanczos_shift","text":"(x, stats) = cg_lanczos_shift(A, b::AbstractVector{FC}, shifts::AbstractVector{T};\n M=I, ldiv::Bool=false,\n check_curvature::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nThe Lanczos version of the conjugate gradient method to solve a family of shifted systems\n\n(A + αI) x = b (α = α₁, ..., αₚ)\n\nof size n. The method does not abort if A + αI is not definite.\n\nInput arguments\n\nA: a linear operator that models a Hermitian matrix of dimension n;\nb: a vector of length n;\nshifts: a vector of length p.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\ncheck_curvature: if true, check that the curvature of the quadratic along the search direction is positive, and abort if not, unless linesearch is also true;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a vector of p dense vectors, each one of length n;\nstats: statistics collected on the run in a LanczosShiftStats structure.\n\nReferences\n\nA. Frommer and P. Maass, Fast CG-Based Methods for Tikhonov-Phillips Regularization, SIAM Journal on Scientific Computing, 20(5), pp. 1831–1850, 1999.\nC. C. Paige and M. A. Saunders, Solution of Sparse Indefinite Systems of Linear Equations, SIAM Journal on Numerical Analysis, 12(4), pp. 617–629, 1975.\n\n\n\n\n\n","category":"function"},{"location":"solvers/spd/#Krylov.cg_lanczos_shift!","page":"Hermitian positive definite linear systems","title":"Krylov.cg_lanczos_shift!","text":"solver = cg_lanczos_shift!(solver::CgLanczosShiftSolver, A, b, shifts; kwargs...)\n\nwhere kwargs are keyword arguments of cg_lanczos_shift.\n\nSee CgLanczosShiftSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"examples/craigmr/","page":"CRAIGMR","title":"CRAIGMR","text":"using Krylov, HarwellRutherfordBoeing, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"wm1\")\npath = fetch_ssmc(matrix, format=\"RB\")\n\nA = RutherfordBoeingData(joinpath(path[1], \"$(matrix.name[1]).rb\")).data\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\nx_exact = A' * ones(m)\nx_exact_norm = norm(x_exact)\nx_exact /= x_exact_norm\nb = A * x_exact\n(x, y, stats) = craigmr(A, b)\nshow(stats)\nresid = norm(A * x - b) / norm(b)\n@printf(\"CRAIGMR: Relative residual: %7.1e\\n\", resid)\n@printf(\"CRAIGMR: ‖x - x*‖₂: %7.1e\\n\", norm(x - x_exact))\n@printf(\"CRAIGMR: %d iterations\\n\", length(stats.residuals))","category":"page"},{"location":"examples/symmlq/","page":"SYMMLQ","title":"SYMMLQ","text":"using Krylov\nusing LinearAlgebra, Printf\n\nA = diagm([1.0; 2.0; 3.0; 0.0])\nn = size(A, 1)\nb = [1.0; 2.0; 3.0; 0.0]\nb_norm = norm(b)\n\n# SYMMLQ returns the minimum-norm solution of symmetric, singular and consistent systems\n(x, stats) = symmlq(A, b, transfer_to_cg=false);\nr = b - A * x;\n\n@printf(\"Residual r: %s\\n\", Krylov.vec2str(r))\n@printf(\"Relative residual norm ‖r‖: %8.1e\\n\", norm(r) / b_norm)\n@printf(\"Solution x: %s\\n\", Krylov.vec2str(x))\n@printf(\"Minimum-norm solution? %s\\n\", x ≈ [1.0; 1.0; 1.0; 0.0])","category":"page"},{"location":"examples/dqgmres/","page":"DQGMRES","title":"DQGMRES","text":"using Krylov, LinearOperators, ILUZero, MatrixMarket\nusing LinearAlgebra, Printf, SuiteSparseMatrixCollection\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"Simon\", \"raefsky1\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nn = matrix.nrows[1]\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = A * ones(n)\n\nF = ilu0(A)\n\n@printf(\"nnz(ILU) / nnz(A): %7.1e\\n\", nnz(F) / nnz(A))\n\n# Solve Ax = b with DQGMRES and an ILU(0) preconditioner\n# Remark: DIOM, FOM and GMRES can be used in the same way\nopM = LinearOperator(Float64, n, n, false, false, (y, v) -> forward_substitution!(y, F, v))\nopN = LinearOperator(Float64, n, n, false, false, (y, v) -> backward_substitution!(y, F, v))\nopP = LinearOperator(Float64, n, n, false, false, (y, v) -> ldiv!(y, F, v))\n\n# Without preconditioning\nx, stats = dqgmres(A, b, memory=50, history=true)\nr = b - A * x\n@printf(\"[Without preconditioning] Residual norm: %8.1e\\n\", norm(r))\n@printf(\"[Without preconditioning] Number of iterations: %3d\\n\", length(stats.residuals) - 1)\n\n# Split preconditioning\nx, stats = dqgmres(A, b, memory=50, history=true, M=opM, N=opN)\nr = b - A * x\n@printf(\"[Split preconditioning] Residual norm: %8.1e\\n\", norm(r))\n@printf(\"[Split preconditioning] Number of iterations: %3d\\n\", length(stats.residuals) - 1)\n\n# Left preconditioning\nx, stats = dqgmres(A, b, memory=50, history=true, M=opP)\nr = b - A * x\n@printf(\"[Left preconditioning] Residual norm: %8.1e\\n\", norm(r))\n@printf(\"[Left preconditioning] Number of iterations: %3d\\n\", length(stats.residuals) - 1)\n\n# Right preconditioning\nx, stats = dqgmres(A, b, memory=50, history=true, N=opP)\nr = b - A * x\n@printf(\"[Right preconditioning] Residual norm: %8.1e\\n\", norm(r))\n@printf(\"[Right preconditioning] Number of iterations: %3d\\n\", length(stats.residuals) - 1)","category":"page"},{"location":"solvers/sp_sqd/","page":"Saddle-point and Hermitian quasi-definite systems","title":"Saddle-point and Hermitian quasi-definite systems","text":"# Saddle-point and Hermitian quasi-definite systems","category":"page"},{"location":"solvers/sp_sqd/#TriCG","page":"Saddle-point and Hermitian quasi-definite systems","title":"TriCG","text":"","category":"section"},{"location":"solvers/sp_sqd/","page":"Saddle-point and Hermitian quasi-definite systems","title":"Saddle-point and Hermitian quasi-definite systems","text":"tricg\ntricg!","category":"page"},{"location":"solvers/sp_sqd/#Krylov.tricg","page":"Saddle-point and Hermitian quasi-definite systems","title":"Krylov.tricg","text":"(x, y, stats) = tricg(A, b::AbstractVector{FC}, c::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n spd::Bool=false, snd::Bool=false,\n flip::Bool=false, τ::T=one(T),\n ν::T=-one(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, y, stats) = tricg(A, b, c, x0::AbstractVector, y0::AbstractVector; kwargs...)\n\nTriCG can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.\n\nGiven a matrix A of dimension m × n, TriCG solves the Hermitian linear system\n\n[ τE A ] [ x ] = [ b ]\n[ Aᴴ νF ] [ y ] [ c ],\n\nof size (n+m) × (n+m) where τ and ν are real numbers, E = M⁻¹ ≻ 0 and F = N⁻¹ ≻ 0. b and c must both be nonzero. TriCG could breakdown if τ = 0 or ν = 0. It's recommended to use TriMR in these cases.\n\nBy default, TriCG solves Hermitian and quasi-definite linear systems with τ = 1 and ν = -1.\n\nTriCG is based on the preconditioned orthogonal tridiagonalization process and its relation with the preconditioned block-Lanczos process.\n\n[ M 0 ]\n[ 0 N ]\n\nindicates the weighted norm in which residuals are measured. It's the Euclidean norm when M and N are identity operators.\n\nTriCG stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖r₀‖ * rtol. atol is an absolute tolerance and rtol is a relative tolerance.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nc: a vector of length n.\n\nOptional arguments\n\nx0: a vector of length m that represents an initial guess of the solution x;\ny0: a vector of length n that represents an initial guess of the solution y.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the partitioned system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the partitioned system;\nldiv: define whether the preconditioners use ldiv! or mul!;\nspd: if true, set τ = 1 and ν = 1 for Hermitian and positive-definite linear system;\nsnd: if true, set τ = -1 and ν = -1 for Hermitian and negative-definite linear systems;\nflip: if true, set τ = -1 and ν = 1 for another known variant of Hermitian quasi-definite systems;\nτ and ν: diagonal scaling factors of the partitioned Hermitian linear system;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length m;\ny: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nA. Montoison and D. Orban, TriCG and TriMR: Two Iterative Methods for Symmetric Quasi-Definite Systems, SIAM Journal on Scientific Computing, 43(4), pp. 2502–2525, 2021.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sp_sqd/#Krylov.tricg!","page":"Saddle-point and Hermitian quasi-definite systems","title":"Krylov.tricg!","text":"solver = tricg!(solver::TricgSolver, A, b, c; kwargs...)\nsolver = tricg!(solver::TricgSolver, A, b, c, x0, y0; kwargs...)\n\nwhere kwargs are keyword arguments of tricg.\n\nSee TricgSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sp_sqd/#TriMR","page":"Saddle-point and Hermitian quasi-definite systems","title":"TriMR","text":"","category":"section"},{"location":"solvers/sp_sqd/","page":"Saddle-point and Hermitian quasi-definite systems","title":"Saddle-point and Hermitian quasi-definite systems","text":"trimr\ntrimr!","category":"page"},{"location":"solvers/sp_sqd/#Krylov.trimr","page":"Saddle-point and Hermitian quasi-definite systems","title":"Krylov.trimr","text":"(x, y, stats) = trimr(A, b::AbstractVector{FC}, c::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n spd::Bool=false, snd::Bool=false,\n flip::Bool=false, sp::Bool=false,\n τ::T=one(T), ν::T=-one(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, y, stats) = trimr(A, b, c, x0::AbstractVector, y0::AbstractVector; kwargs...)\n\nTriMR can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.\n\nGiven a matrix A of dimension m × n, TriMR solves the symmetric linear system\n\n[ τE A ] [ x ] = [ b ]\n[ Aᴴ νF ] [ y ] [ c ],\n\nof size (n+m) × (n+m) where τ and ν are real numbers, E = M⁻¹ ≻ 0, F = N⁻¹ ≻ 0. b and c must both be nonzero. TriMR handles saddle-point systems (τ = 0 or ν = 0) and adjoint systems (τ = 0 and ν = 0) without any risk of breakdown.\n\nBy default, TriMR solves symmetric and quasi-definite linear systems with τ = 1 and ν = -1.\n\nTriMR is based on the preconditioned orthogonal tridiagonalization process and its relation with the preconditioned block-Lanczos process.\n\n[ M 0 ]\n[ 0 N ]\n\nindicates the weighted norm in which residuals are measured. It's the Euclidean norm when M and N are identity operators.\n\nTriMR stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖r₀‖ * rtol. atol is an absolute tolerance and rtol is a relative tolerance.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nc: a vector of length n.\n\nOptional arguments\n\nx0: a vector of length m that represents an initial guess of the solution x;\ny0: a vector of length n that represents an initial guess of the solution y.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the partitioned system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the partitioned system;\nldiv: define whether the preconditioners use ldiv! or mul!;\nspd: if true, set τ = 1 and ν = 1 for Hermitian and positive-definite linear system;\nsnd: if true, set τ = -1 and ν = -1 for Hermitian and negative-definite linear systems;\nflip: if true, set τ = -1 and ν = 1 for another known variant of Hermitian quasi-definite systems;\nsp: if true, set τ = 1 and ν = 0 for saddle-point systems;\nτ and ν: diagonal scaling factors of the partitioned Hermitian linear system;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length m;\ny: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nA. Montoison and D. Orban, TriCG and TriMR: Two Iterative Methods for Symmetric Quasi-Definite Systems, SIAM Journal on Scientific Computing, 43(4), pp. 2502–2525, 2021.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sp_sqd/#Krylov.trimr!","page":"Saddle-point and Hermitian quasi-definite systems","title":"Krylov.trimr!","text":"solver = trimr!(solver::TrimrSolver, A, b, c; kwargs...)\nsolver = trimr!(solver::TrimrSolver, A, b, c, x0, y0; kwargs...)\n\nwhere kwargs are keyword arguments of trimr.\n\nSee TrimrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"examples/car/","page":"CAR","title":"CAR","text":"using Krylov, MatrixMarket, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"Nasa\", \"nasa2146\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nn = matrix.nrows[1]\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = ones(n)\nb_norm = norm(b)\n\n# Solve Ax = b.\n(x, stats) = car(A, b)\nshow(stats)\nr = b - A * x\n@printf(\"Relative residual: %8.1e\\n\", norm(r) / b_norm)","category":"page"},{"location":"examples/bicgstab/","page":"BICGSTAB","title":"BICGSTAB","text":"using Krylov, LinearOperators, KrylovPreconditioners, HarwellRutherfordBoeing\nusing LinearAlgebra, Printf, SuiteSparseMatrixCollection, SparseArrays\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"sherman5\")\npath = fetch_ssmc(matrix, format=\"RB\")\n\nn = matrix.nrows[1]\nA = RutherfordBoeingData(joinpath(path[1], \"$(matrix.name[1]).rb\")).data\nb = A * ones(n)\n\nF = ilu(A, τ = 0.05)\n\n@printf(\"nnz(ILU) / nnz(A): %7.1e\\n\", nnz(F) / nnz(A))\n\n# Solve Ax = b with BICGSTAB and an incomplete LU factorization\n# Remark: CGS can be used in the same way\nopM = LinearOperator(Float64, n, n, false, false, (y, v) -> forward_substitution!(y, F, v))\nopN = LinearOperator(Float64, n, n, false, false, (y, v) -> backward_substitution!(y, F, v))\nopP = LinearOperator(Float64, n, n, false, false, (y, v) -> ldiv!(y, F, v))\n\n# Without preconditioning\nx, stats = bicgstab(A, b, history=true)\nr = b - A * x\n@printf(\"[Without preconditioning] Residual norm: %8.1e\\n\", norm(r))\n@printf(\"[Without preconditioning] Number of iterations: %3d\\n\", length(stats.residuals) - 1)\n\n# Split preconditioning\nx, stats = bicgstab(A, b, history=true, M=opM, N=opN)\nr = b - A * x\n@printf(\"[Split preconditioning] Residual norm: %8.1e\\n\", norm(r))\n@printf(\"[Split preconditioning] Number of iterations: %3d\\n\", length(stats.residuals) - 1)\n\n# Left preconditioning\nx, stats = bicgstab(A, b, history=true, M=opP)\nr = b - A * x\n@printf(\"[Left preconditioning] Residual norm: %8.1e\\n\", norm(r))\n@printf(\"[Left preconditioning] Number of iterations: %3d\\n\", length(stats.residuals) - 1)\n\n# Right preconditioning\nx, stats = bicgstab(A, b, history=true, N=opP)\nr = b - A * x\n@printf(\"[Right preconditioning] Residual norm: %8.1e\\n\", norm(r))\n@printf(\"[Right preconditioning] Number of iterations: %3d\\n\", length(stats.residuals) - 1)","category":"page"},{"location":"examples/cgls_lanczos_shift/","page":"CGLS-LANCZOS-SHIFT","title":"CGLS-LANCZOS-SHIFT","text":"using MatrixMarket, SuiteSparseMatrixCollection\nusing Krylov, LinearOperators\nusing LinearAlgebra, Printf\n\nfunction residuals(A, b, shifts, x)\n nshifts = length(shifts)\n r = [ A' * (A * x[i] - b) + shifts[i] * x[i] for i = 1 : nshifts ]\n return r\nend\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"well1033\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1])_b.mtx\"))[:]\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\n# Define regularization parameters.\nshifts = [1.0, 2.0, 3.0, 4.0]\n\n(x, stats) = cgls_lanczos_shift(A, b, shifts)\nshow(stats)\nr = residuals(A, b, shifts, x)\n\nresids = map(norm, r) / norm(b)\n@printf(\"CGLS: Relative residuals with shifts:\\n\")\nfor resid in resids\n @printf(\" %8.1e\", resid)\nend\n@printf(\"\\n\")","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"The most expensive procedures in Krylov methods are matrix-vector products (y ← Ax) and vector operations (dot products, vector norms, y ← αx + βy). Therefore they directly affect the efficiency of the methods provided by Krylov.jl. In this section, we present various optimizations based on multithreading to speed up these procedures. Multithreading is a form of shared memory parallelism that makes use of multiple cores to perform tasks.","category":"page"},{"location":"tips/#Multi-threaded-BLAS-and-LAPACK-operations","page":"Performance tips","title":"Multi-threaded BLAS and LAPACK operations","text":"","category":"section"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"OPENBLAS_NUM_THREADS=N julia # Set the number of OpenBLAS threads to N\nMKL_NUM_THREADS=N julia # Set the number of MKL threads to N","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"If you don't know the maximum number of threads available on your computer, you can obtain it with","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"NMAX = Sys.CPU_THREADS","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"and define the number of OpenBLAS/MKL threads at runtime with","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"BLAS.set_num_threads(N) # 1 ≤ N ≤ NMAX\nBLAS.get_num_threads()","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"The recommended number of BLAS threads is the number of physical and not logical cores, which is in general N = NMAX / 2 if your CPU supports simultaneous multithreading (SMT).","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"By default Julia ships with OpenBLAS but it's also possible to use Intel MKL BLAS and LAPACK with MKL.jl. If your operating system is MacOS 13.4 or later, it's recommended to use Accelerate BLAS and LAPACK with AppleAccelerate.jl.","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"using LinearAlgebra\nBLAS.get_config() # BLAS.vendor() for Julia 1.6","category":"page"},{"location":"tips/#Multi-threaded-sparse-matrix-vector-products","page":"Performance tips","title":"Multi-threaded sparse matrix-vector products","text":"","category":"section"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"For sparse matrices, the Julia implementation of mul! of SparseArrays library is not parallelized. A significant speed-up can be observed with the multhreaded mul! of MKLSparse.jl or ThreadedSparseCSR.jl.","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"It's also possible to implement a generic multithreaded julia version. For instance, the following function can be used for symmetric matrices","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"using Base.Threads\n\nfunction threaded_mul!(y::Vector{T}, A::SparseMatrixCSC{T}, x::Vector{T}) where T <: Number\n A.m == A.n || error(\"A is not a square matrix!\")\n @threads for i = 1 : A.n\n tmp = zero(T)\n @inbounds for j = A.colptr[i] : (A.colptr[i+1] - 1)\n tmp += A.nzval[j] * x[A.rowval[j]]\n end\n @inbounds y[i] = tmp\n end\n return y\nend","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"and wrapped inside a linear operator to solve symmetric linear systems","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"using LinearOperators\n\nn, m = size(A)\nsym = herm = true\nT = eltype(A)\nopA = LinearOperator(T, n, m, sym, herm, (y, v) -> threaded_mul!(y, A, v))","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"To enable multi-threading with Julia, you can start julia with the environment variable JULIA_NUM_THREADS or the options -t and --threads","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"julia -t auto # alternative: --threads auto\njulia -t N # alternative: --threads N\n\nJULIA_NUM_THREADS=N julia","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"Thereafter, you can verify the number of threads usable by Julia","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"using Base.Threads\nnthreads()","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"The following benchmarks illustrate the time required in seconds to compute 1000 sparse matrix-vector products with symmetric matrices of the SuiteSparse Matrix Collection. The computer used for the benchmarks has 2 physical cores and Julia was launched with JULIA_NUM_THREADS=2.","category":"page"},{"location":"tips/","page":"Performance tips","title":"Performance tips","text":"(Image: benchmarks)","category":"page"},{"location":"solvers/gsp/","page":"Generalized saddle-point and non-Hermitian partitioned systems","title":"Generalized saddle-point and non-Hermitian partitioned systems","text":"# Generalized saddle-point and non-Hermitian partitioned systems","category":"page"},{"location":"solvers/gsp/#GPMR","page":"Generalized saddle-point and non-Hermitian partitioned systems","title":"GPMR","text":"","category":"section"},{"location":"solvers/gsp/","page":"Generalized saddle-point and non-Hermitian partitioned systems","title":"Generalized saddle-point and non-Hermitian partitioned systems","text":"gpmr\ngpmr!","category":"page"},{"location":"solvers/gsp/#Krylov.gpmr","page":"Generalized saddle-point and non-Hermitian partitioned systems","title":"Krylov.gpmr","text":"(x, y, stats) = gpmr(A, B, b::AbstractVector{FC}, c::AbstractVector{FC};\n memory::Int=20, C=I, D=I, E=I, F=I,\n ldiv::Bool=false, gsp::Bool=false,\n λ::FC=one(FC), μ::FC=one(FC),\n reorthogonalization::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, y, stats) = gpmr(A, B, b, c, x0::AbstractVector, y0::AbstractVector; kwargs...)\n\nGPMR can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.\n\nGiven matrices A of dimension m × n and B of dimension n × m, GPMR solves the non-Hermitian partitioned linear system\n\n[ λIₘ A ] [ x ] = [ b ]\n[ B μIₙ ] [ y ] [ c ],\n\nof size (n+m) × (n+m) where λ and μ are real or complex numbers. A can have any shape and B has the shape of Aᴴ. A, B, b and c must be all nonzero.\n\nThis implementation allows left and right block diagonal preconditioners\n\n[ C ] [ λM A ] [ E ] [ E⁻¹x ] = [ Cb ]\n[ D ] [ B μN ] [ F ] [ F⁻¹y ] [ Dc ],\n\nand can solve\n\n[ λM A ] [ x ] = [ b ]\n[ B μN ] [ y ] [ c ]\n\nwhen CE = M⁻¹ and DF = N⁻¹.\n\nBy default, GPMR solves unsymmetric linear systems with λ = 1 and μ = 1.\n\nGPMR is based on the orthogonal Hessenberg reduction process and its relations with the block-Arnoldi process. The residual norm ‖rₖ‖ is monotonically decreasing in GPMR.\n\nGPMR stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖r₀‖ * rtol. atol is an absolute tolerance and rtol is a relative tolerance.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nB: a linear operator that models a matrix of dimension n × m;\nb: a vector of length m;\nc: a vector of length n.\n\nOptional arguments\n\nx0: a vector of length m that represents an initial guess of the solution x;\ny0: a vector of length n that represents an initial guess of the solution y.\n\nKeyword arguments\n\nmemory: if restart = true, the restarted version GPMR(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;\nC: linear operator that models a nonsingular matrix of size m, and represents the first term of the block-diagonal left preconditioner;\nD: linear operator that models a nonsingular matrix of size n, and represents the second term of the block-diagonal left preconditioner;\nE: linear operator that models a nonsingular matrix of size m, and represents the first term of the block-diagonal right preconditioner;\nF: linear operator that models a nonsingular matrix of size n, and represents the second term of the block-diagonal right preconditioner;\nldiv: define whether the preconditioners use ldiv! or mul!;\ngsp: if true, set λ = 1 and μ = 0 for generalized saddle-point systems;\nλ and μ: diagonal scaling factors of the partitioned linear system;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length m;\ny: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nA. Montoison and D. Orban, GPMR: An Iterative Method for Unsymmetric Partitioned Linear Systems, SIAM Journal on Matrix Analysis and Applications, 44(1), pp. 293–311, 2023.\n\n\n\n\n\n","category":"function"},{"location":"solvers/gsp/#Krylov.gpmr!","page":"Generalized saddle-point and non-Hermitian partitioned systems","title":"Krylov.gpmr!","text":"solver = gpmr!(solver::GpmrSolver, A, B, b, c; kwargs...)\nsolver = gpmr!(solver::GpmrSolver, A, B, b, c, x0, y0; kwargs...)\n\nwhere kwargs are keyword arguments of gpmr.\n\nNote that the memory keyword argument is the only exception. It's required to create a GpmrSolver and can't be changed later.\n\nSee GpmrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"examples/lsmr/","page":"LSMR","title":"LSMR","text":"using MatrixMarket, SuiteSparseMatrixCollection\nusing Krylov, LinearOperators\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"illc1850\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1])_b.mtx\"))[:]\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\n# Define a regularization parameter.\nλ = 1.0e-3\n\n(x, stats) = lsmr(A, b, λ=λ, atol=0.0, btol=0.0)\nshow(stats)\nresid = norm(A' * (A * x - b) + λ * x) / norm(b)\n@printf(\"LSMR: Relative residual: %8.1e\\n\", resid)\n@printf(\"LSMR: ‖x‖: %8.1e\\n\", norm(x))","category":"page"},{"location":"examples/cg_lanczos_shift/","page":"CG-LANCZOS-SHIFT","title":"CG-LANCZOS-SHIFT","text":"using Krylov, MatrixMarket, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nfunction residuals(A, b, shifts, x)\n nshifts = length(shifts)\n r = [ (b - A * x[i] - shifts[i] * x[i]) for i = 1 : nshifts ]\n return r\nend\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"1138_bus\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nn, m = size(A)\nb = ones(n)\n\n# Solve (A + αI)x = b.\nshifts = [1.0, 2.0, 3.0, 4.0]\n(x, stats) = cg_lanczos_shift(A, b, shifts)\nshow(stats)\nr = residuals(A, b, shifts, x)\nresids = map(norm, r) / norm(b)\n@printf(\"Relative residuals with shifts:\\n\")\nfor resid in resids\n @printf(\" %8.1e\", resid)\nend\n@printf(\"\\n\")","category":"page"},{"location":"examples/minres_qlp/","page":"MINRES-QLP","title":"MINRES-QLP","text":"using Krylov\nusing LinearAlgebra, Printf\n\nA = diagm([1.0; 2.0; 3.0; 0.0])\nn = size(A, 1)\nb = [1.0; 2.0; 3.0; 4.0]\nb_norm = norm(b)\n\n# MINRES-QLP returns the minimum-norm solution of symmetric, singular and inconsistent systems\n(x, stats) = minres_qlp(A, b);\nr = b - A * x;\n\n@printf(\"Residual r: %s\\n\", Krylov.vec2str(r))\n@printf(\"Relative residual norm ‖r‖: %8.1e\\n\", norm(r) / b_norm)\n@printf(\"Solution x: %s\\n\", Krylov.vec2str(x))\n@printf(\"Minimum-norm solution? %s\\n\", x ≈ [1.0; 1.0; 1.0; 0.0])","category":"page"},{"location":"examples/lsqr/","page":"LSQR","title":"LSQR","text":"using MatrixMarket, SuiteSparseMatrixCollection\nusing Krylov, LinearOperators\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"illc1033\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1])_b.mtx\"))[:]\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\n# Define a regularization parameter.\nλ = 1.0e-3\n\n(x, stats) = lsqr(A, b, λ=λ, atol=0.0, btol=0.0)\nshow(stats)\nresid = norm(A' * (A * x - b) + λ * x) / norm(b)\n@printf(\"LSQR: Relative residual: %8.1e\\n\", resid)\n@printf(\"LSQR: ‖x‖: %8.1e\\n\", norm(x))","category":"page"},{"location":"inplace/#in-place","page":"In-place methods","title":"In-place methods","text":"","category":"section"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"All solvers in Krylov.jl have an in-place variant implemented in a method whose name ends with !. A workspace (KrylovSolver), which contains the storage needed by a Krylov method, can be used to solve multiple linear systems with the same dimensions and the same floating-point precision. The section storage requirements specifies the memory needed for each Krylov method. Each KrylovSolver has two constructors:","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"XyzSolver(A, b)\nXyzSolver(m, n, S)","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"Xyz is the name of the Krylov method with lowercase letters except its first one (Cg, Minres, Lsmr, Bicgstab, ...). Given an operator A and a right-hand side b, you can create a KrylovSolver based on the size of A and the type of b or explicitly give the dimensions (m, n) and the storage type S.","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"For example, use S = Vector{Float64} if you want to solve linear systems in double precision on the CPU and S = CuVector{Float32} if you want to solve linear systems in single precision on an Nvidia GPU.","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"note: Note\nDiomSolver, FomSolver, DqgmresSolver, GmresSolver, BlockGmresSolver, FgmresSolver, GpmrSolver, CgLanczosShiftSolver and CglsLanczosShiftSolver require an additional argument (memory or nshifts).","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"The workspace is always the first argument of the in-place methods:","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"minres_solver = MinresSolver(n, n, Vector{Float64})\nminres!(minres_solver, A1, b1)\n\ndqgmres_solver = DqgmresSolver(n, n, memory, Vector{BigFloat})\ndqgmres!(dqgmres_solver, A2, b2)\n\nlsqr_solver = LsqrSolver(m, n, CuVector{Float32})\nlsqr!(lsqr_solver, A3, b3)","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"A generic function solve! is also available and dispatches to the appropriate Krylov method.","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"Krylov.solve!","category":"page"},{"location":"inplace/#Krylov.solve!","page":"In-place methods","title":"Krylov.solve!","text":"solve!(solver, args...; kwargs...)\n\nUse the in-place Krylov method associated to solver.\n\n\n\n\n\n","category":"function"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"In-place methods return an updated solver workspace. Solutions and statistics can be recovered via solver.x, solver.y and solver.stats. Functions solution, statistics and results can be also used.","category":"page"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"Krylov.nsolution\nKrylov.issolved\nKrylov.solution\nKrylov.statistics\nKrylov.results","category":"page"},{"location":"inplace/#Krylov.nsolution","page":"In-place methods","title":"Krylov.nsolution","text":"nsolution(solver)\n\nReturn the number of outputs of solution(solver).\n\n\n\n\n\n","category":"function"},{"location":"inplace/#Krylov.issolved","page":"In-place methods","title":"Krylov.issolved","text":"issolved(solver)\n\nReturn a boolean that determines whether the Krylov method associated to solver succeeded.\n\n\n\n\n\n","category":"function"},{"location":"inplace/#Krylov.solution","page":"In-place methods","title":"Krylov.solution","text":"solution(solver)\n\nReturn the solution(s) stored in the solver. Optionally you can specify which solution you want to recover, solution(solver, 1) returns x and solution(solver, 2) returns y.\n\n\n\n\n\n","category":"function"},{"location":"inplace/#Krylov.statistics","page":"In-place methods","title":"Krylov.statistics","text":"statistics(solver)\n\nReturn the statistics stored in the solver.\n\n\n\n\n\n","category":"function"},{"location":"inplace/#Krylov.results","page":"In-place methods","title":"Krylov.results","text":"results(solver)\n\nReturn a tuple containing the solution(s) and the statistics associated with the solver. Allows retrieving the output arguments of an out-of-place method from the in-place method.\n\nFor example, instead of x, stats = cg(A, b), you can use:\n\n solver = CgSolver(A, b)\n cg!(solver, A, b)\n x, stats = results(solver)\n\n\n\n\n\n","category":"function"},{"location":"inplace/#Examples","page":"In-place methods","title":"Examples","text":"","category":"section"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"We illustrate the use of in-place Krylov solvers with two well-known optimization methods. The details of the optimization methods are described in the section about Factorization-free operators.","category":"page"},{"location":"inplace/#Example-1:-Newton's-method-for-convex-optimization-without-linesearch","page":"In-place methods","title":"Example 1: Newton's method for convex optimization without linesearch","text":"","category":"section"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"using Krylov\n\nfunction newton(∇f, ∇²f, x₀; itmax = 200, tol = 1e-8)\n\n n = length(x₀)\n x = copy(x₀)\n gx = ∇f(x)\n \n iter = 0\n S = typeof(x)\n solver = CgSolver(n, n, S)\n Δx = solver.x\n\n solved = false\n tired = false\n\n while !(solved || tired)\n \n Hx = ∇²f(x) # Compute ∇²f(xₖ)\n cg!(solver, Hx, -gx) # Solve ∇²f(xₖ)Δx = -∇f(xₖ)\n x = x + Δx # Update xₖ₊₁ = xₖ + Δx\n gx = ∇f(x) # ∇f(xₖ₊₁)\n \n iter += 1\n solved = norm(gx) ≤ tol\n tired = iter ≥ itmax\n end\n return x\nend","category":"page"},{"location":"inplace/#Example-2:-The-Gauss-Newton-method-for-nonlinear-least-squares-without-linesearch","page":"In-place methods","title":"Example 2: The Gauss-Newton method for nonlinear least squares without linesearch","text":"","category":"section"},{"location":"inplace/","page":"In-place methods","title":"In-place methods","text":"using Krylov\n\nfunction gauss_newton(F, JF, x₀; itmax = 200, tol = 1e-8)\n\n n = length(x₀)\n x = copy(x₀)\n Fx = F(x)\n m = length(Fx)\n \n iter = 0\n S = typeof(x)\n solver = LsmrSolver(m, n, S)\n Δx = solver.x\n\n solved = false\n tired = false\n\n while !(solved || tired)\n \n Jx = JF(x) # Compute J(xₖ)\n lsmr!(solver, Jx, -Fx) # Minimize ‖J(xₖ)Δx + F(xₖ)‖\n x = x + Δx # Update xₖ₊₁ = xₖ + Δx\n Fx_old = Fx # F(xₖ)\n Fx = F(x) # F(xₖ₊₁)\n \n iter += 1\n solved = norm(Fx - Fx_old) / norm(Fx) ≤ tol\n tired = iter ≥ itmax\n end\n return x\nend","category":"page"},{"location":"warm-start/#warm-start","page":"Warm-start","title":"Warm-start","text":"","category":"section"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"Most Krylov methods in this module accept a starting point as argument. The starting point is used as initial approximation to a solution.","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"solver = CgSolver(A, b)\ncg!(solver, A, b, itmax=100)\nif !issolved(solver)\n cg!(solver, A, b, solver.x, itmax=100) # cg! uses the approximate solution `solver.x` as starting point\nend","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"If the user has an initial guess x0, it can be provided directly.","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"cg(A, b, x0)","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"It is also possible to use the warm_start! function to feed the starting point into the solver.","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"warm_start!(solver, x0)\ncg!(solver, A, b)\n# the previous two lines are equivalent to cg!(solver, A, b, x0)","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"If a Krylov method doesn't have the option to warm start, it can still be done explicitly. We provide an example with cg_lanczos!.","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"solver = CgLanczosSolver(A, b)\ncg_lanczos!(solver, A, b)\nx₀ = solver.x # Ax₀ ≈ b\nr = b - A * x₀ # r = b - Ax₀\ncg_lanczos!(solver, A, r)\nΔx = solver.x # AΔx = r\nx = x₀ + Δx # Ax = b","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"Explicit restarts cannot be avoided in certain block methods, such as TriMR, due to the preconditioners.","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"# [E A] [x] = [b]\n# [Aᴴ F] [y] [c]\nM = inv(E)\nN = inv(F)\nx₀, y₀, stats = trimr(A, b, c, M=M, N=N)\n\n# E and F are not available inside TriMR\nb₀ = b - Ex₀ - Ay\nc₀ = c - Aᴴx₀ - Fy\n\nΔx, Δy, stats = trimr(A, b₀, c₀, M=M, N=N)\nx = x₀ + Δx\ny = y₀ + Δy","category":"page"},{"location":"warm-start/","page":"Warm-start","title":"Warm-start","text":"# ## Restarted methods\n#\n# The storage requirements of Krylov methods based on the Arnoldi process, such as FOM and GMRES, increase as the iteration progresses.\n# For very large problems, the storage costs become prohibitive after only few iterations and restarted variants FOM(k) and GMRES(k) are preferred.\n# In this section, we show how to use warm starts to implement GMRES(k) and FOM(k).\n#\n# ```julia\n# k = 50\n# solver = GmresSolver(A, b, k) # FomSolver(A, b, k)\n# solver.x .= 0 # solver.x .= x₀ \n# nrestart = 0\n# while !issolved(solver) || nrestart ≤ 10\n# solve!(solver, A, b, solver.x, itmax=k)\n# nrestart += 1\n# end\n# ```","category":"page"},{"location":"block_krylov/#Block-GMRES","page":"Block-Krylov methods","title":"Block-GMRES","text":"","category":"section"},{"location":"block_krylov/","page":"Block-Krylov methods","title":"Block-Krylov methods","text":"note: Note\nblock_gmres works on GPUs with Julia 1.11.","category":"page"},{"location":"block_krylov/","page":"Block-Krylov methods","title":"Block-Krylov methods","text":"If you want to use block_gmres on previous Julia versions, you can overload the function Krylov.copy_triangle with the following code:","category":"page"},{"location":"block_krylov/","page":"Block-Krylov methods","title":"Block-Krylov methods","text":"using KernelAbstractions, Krylov\n\n@kernel function copy_triangle_kernel!(dest, src)\n i, j = @index(Global, NTuple)\n if j >= i\n @inbounds dest[i, j] = src[i, j]\n end\nend\n\nfunction Krylov.copy_triangle(Q::AbstractMatrix{FC}, R::AbstractMatrix{FC}, k::Int) where FC <: Krylov.FloatOrComplex\n backend = get_backend(Q)\n ndrange = (k, k)\n copy_triangle_kernel!(backend)(R, Q; ndrange=ndrange)\n KernelAbstractions.synchronize(backend)\nend","category":"page"},{"location":"block_krylov/","page":"Block-Krylov methods","title":"Block-Krylov methods","text":"block_gmres\nblock_gmres!","category":"page"},{"location":"block_krylov/#Krylov.block_gmres","page":"Block-Krylov methods","title":"Krylov.block_gmres","text":"(X, stats) = block_gmres(A, b::AbstractMatrix{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n restart::Bool=false, reorthogonalization::Bool=false,\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(X, stats) = block_gmres(A, B, X0::AbstractMatrix; kwargs...)\n\nBlock-GMRES can be warm-started from an initial guess X0 where kwargs are the same keyword arguments as above.\n\nSolve the linear system AX = B of size n with p right-hand sides using block-GMRES.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nB: a matrix of size n × p.\n\nOptional argument\n\nX0: a matrix of size n × p that represents an initial guess of the solution X.\n\nKeyword arguments\n\nmemory: if restart = true, the restarted version block-GMRES(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\nrestart: restart the method after memory iterations;\nreorthogonalization: reorthogonalize the new matrices of the block-Krylov basis against all previous matrix;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2 * div(n,p);\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms;\ncallback: function or functor called as callback(solver) that returns true if the block-Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nX: a dense matrix of size n × p;\nstats: statistics collected on the run in a SimpleStats structure.\n\n\n\n\n\n","category":"function"},{"location":"block_krylov/#Krylov.block_gmres!","page":"Block-Krylov methods","title":"Krylov.block_gmres!","text":"solver = block_gmres!(solver::BlockGmresSolver, B; kwargs...)\nsolver = block_gmres!(solver::BlockGmresSolver, B, X0; kwargs...)\n\nwhere kwargs are keyword arguments of block_gmres.\n\nSee BlockGmresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"block_processes/#block-krylov-processes","page":"Block Krylov processes","title":"Block Krylov processes","text":"","category":"section"},{"location":"block_processes/#block-hermitian-lanczos","page":"Block Krylov processes","title":"Block Hermitian Lanczos","text":"","category":"section"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"If the vector b in the Hermitian Lanczos process is replaced by a matrix B with p columns, we can derive the block Hermitian Lanczos process.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"(Image: block_hermitian_lanczos)","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"After k iterations of the block Hermitian Lanczos process, the situation may be summarized as","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"beginalign*\n A V_k = V_k+1 T_k+1k \n V_k^H V_k = I_pk\nendalign*","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"where V_k is an orthonormal basis of the block Krylov subspace mathcalK_k^square(AB),","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"T_k+1k =\nbeginbmatrix\n Omega_1 Psi_2^H \n Psi_2 Omega_2 ddots \n ddots ddots Psi_k^H \n Psi_k Omega_k \n Psi_k+1\nendbmatrix","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"The function hermitian_lanczos returns V_k+1, Psi_1 and T_k+1k.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"hermitian_lanczos(::Any, ::AbstractMatrix{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"block_processes/#Krylov.hermitian_lanczos-Union{Tuple{FC}, Tuple{Any, AbstractMatrix{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Block Krylov processes","title":"Krylov.hermitian_lanczos","text":"V, Ψ, T = hermitian_lanczos(A, B, k; algo=\"householder\")\n\nInput arguments\n\nA: a linear operator that models an Hermitian matrix of dimension n;\nB: a matrix of size n × p;\nk: the number of iterations of the block Hermitian Lanczos process.\n\nKeyword arguments\n\nalgo: the algorithm to perform reduced QR factorizations (\"gs\", \"mgs\", \"givens\" or \"householder\").\n\nOutput arguments\n\nV: a dense n × p(k+1) matrix;\nΨ: a dense p × p upper triangular matrix such that V₁Ψ = B;\nT: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p.\n\n\n\n\n\n","category":"method"},{"location":"block_processes/#block-nonhermitian-lanczos","page":"Block Krylov processes","title":"Block Non-Hermitian Lanczos","text":"","category":"section"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"If the vectors b and c in the non-Hermitian Lanczos process are replaced by matrices B and C with both p columns, we can derive the block non-Hermitian Lanczos process.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"(Image: block_nonhermitian_lanczos)","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"After k iterations of the block non-Hermitian Lanczos process, the situation may be summarized as","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"beginalign*\n A V_k = V_k+1 T_k+1k \n A^H U_k = U_k+1 T_kk+1^H \n V_k^H U_k = U_k^H V_k = I_pk\nendalign*","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"where V_k and U_k are bases of the block Krylov subspaces mathcalK^square_k(AB) and mathcalK^square_k (A^HC), respectively,","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"T_k+1k =\nbeginbmatrix\n Omega_1 Phi_2 \n Psi_2 Omega_2 ddots \n ddots ddots Phi_k \n Psi_k Omega_k \n Psi_k+1\nendbmatrix\n qquad\nT_kk+1^H =\nbeginbmatrix\n Omega_1^H Psi_2^H \n Phi_2^H Omega_2^H ddots \n ddots ddots Psi_k^H \n Phi_k^H Omega_k^H \n Phi_k+1^H\nendbmatrix","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"The function nonhermitian_lanczos returns V_k+1, Psi_1, T_k+1k, U_k+1 Phi_1^H and T_kk+1^H.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"nonhermitian_lanczos(::Any, ::AbstractMatrix{FC}, ::AbstractMatrix{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"block_processes/#Krylov.nonhermitian_lanczos-Union{Tuple{FC}, Tuple{Any, AbstractMatrix{FC}, AbstractMatrix{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Block Krylov processes","title":"Krylov.nonhermitian_lanczos","text":"V, Ψ, T, U, Φᴴ, Tᴴ = nonhermitian_lanczos(A, B, C, k)\n\nInput arguments\n\nA: a linear operator that models a square matrix of dimension n;\nB: a matrix of size n × p;\nC: a matrix of size n × p;\nk: the number of iterations of the block non-Hermitian Lanczos process.\n\nOutput arguments\n\nV: a dense n × p(k+1) matrix;\nΨ: a dense p × p upper triangular matrix such that V₁Ψ = B;\nT: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p;\nU: a dense n × p(k+1) matrix;\nΦᴴ: a dense p × p upper triangular matrix such that U₁Φᴴ = C;\nTᴴ: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p.\n\n\n\n\n\n","category":"method"},{"location":"block_processes/#block-arnoldi","page":"Block Krylov processes","title":"Block Arnoldi","text":"","category":"section"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"If the vector b in the Arnoldi process is replaced by a matrix B with p columns, we can derive the block Arnoldi process.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"(Image: block_arnoldi)","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"After k iterations of the block Arnoldi process, the situation may be summarized as","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"beginalign*\n A V_k = V_k+1 H_k+1k \n V_k^H V_k = I_pk\nendalign*","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"where V_k is an orthonormal basis of the block Krylov subspace mathcalK_k^square(AB),","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"H_k+1k =\nbeginbmatrix\n Psi_11 Psi_12 ldots Psi_1k \n Psi_21 ddots ddots vdots \n ddots ddots Psi_k-1k \n Psi_kk-1 Psi_kk \n Psi_k+1k\nendbmatrix","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"The function arnoldi returns V_k+1, Gamma, and H_k+1k.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"Related method: BLOCK-GMRES.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"arnoldi(::Any, ::AbstractMatrix{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"block_processes/#Krylov.arnoldi-Union{Tuple{FC}, Tuple{Any, AbstractMatrix{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Block Krylov processes","title":"Krylov.arnoldi","text":"V, Γ, H = arnoldi(A, B, k; algo=\"householder\", reorthogonalization=false)\n\nInput arguments\n\nA: a linear operator that models a square matrix of dimension n;\nB: a matrix of size n × p;\nk: the number of iterations of the block Arnoldi process.\n\nKeyword arguments\n\nalgo: the algorithm to perform reduced QR factorizations (\"gs\", \"mgs\", \"givens\" or \"householder\").\nreorthogonalization: reorthogonalize the new matrices of the block Krylov basis against all previous matrices.\n\nOutput arguments\n\nV: a dense n × p(k+1) matrix;\nΓ: a dense p × p upper triangular matrix such that V₁Γ = B;\nH: a dense p(k+1) × pk block upper Hessenberg matrix with a lower bandwidth p.\n\n\n\n\n\n","category":"method"},{"location":"block_processes/#block-golub-kahan","page":"Block Krylov processes","title":"Block Golub-Kahan","text":"","category":"section"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"If the vector b in the Golub-Kahan process is replaced by a matrix B with p columns, we can derive the block Golub-Kahan process.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"(Image: block_golub_kahan)","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"After k iterations of the block Golub-Kahan process, the situation may be summarized as","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"beginalign*\n A V_k = U_k+1 B_k \n A^H U_k+1 = V_k+1 L_k+1^H \n V_k^H V_k = U_k^H U_k = I_pk\nendalign*","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"where V_k and U_k are bases of the block Krylov subspaces mathcalK_k^square(A^HAA^HB) and mathcalK_k^square(AA^HB), respectively,","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"B_k =\nbeginbmatrix\n Omega_1 \n Psi_2 Omega_2 \n ddots ddots \n Psi_k Omega_k \n Psi_k+1 \nendbmatrix\n qquad\nL_k+1^H =\nbeginbmatrix\n Omega_1^H Psi_2^H \n Omega_2^H ddots \n ddots Psi_k^H \n Omega_k^H Psi_k+1^H \n Omega_k+1^H \nendbmatrix","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"The function golub_kahan returns V_k+1, U_k+1, Psi_1 and L_k+1.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"golub_kahan(::Any, ::AbstractMatrix{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"block_processes/#Krylov.golub_kahan-Union{Tuple{FC}, Tuple{Any, AbstractMatrix{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Block Krylov processes","title":"Krylov.golub_kahan","text":"V, U, Ψ, L = golub_kahan(A, B, k; algo=\"householder\")\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nB: a matrix of size m × p;\nk: the number of iterations of the block Golub-Kahan process.\n\nKeyword argument\n\nalgo: the algorithm to perform reduced QR factorizations (\"gs\", \"mgs\", \"givens\" or \"householder\").\n\nOutput arguments\n\nV: a dense n × p(k+1) matrix;\nU: a dense m × p(k+1) matrix;\nΨ: a dense p × p upper triangular matrix such that U₁Ψ = B;\nL: a sparse p(k+1) × p(k+1) block lower bidiagonal matrix with a lower bandwidth p.\n\n\n\n\n\n","category":"method"},{"location":"block_processes/#block-saunders-simon-yip","page":"Block Krylov processes","title":"Block Saunders-Simon-Yip","text":"","category":"section"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"If the vectors b and c in the Saunders-Simon-Yip process are replaced by matrices B and C with both p columns, we can derive the block Saunders-Simon-Yip process.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"(Image: block_saunders_simon_yip)","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"After k iterations of the block Saunders-Simon-Yip process, the situation may be summarized as","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"beginalign*\n A U_k = V_k+1 T_k+1k \n A^H V_k = U_k+1 T_kk+1^H \n V_k^H V_k = U_k^H U_k = I_pk\nendalign*","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"where beginbmatrix V_k 0 0 U_k endbmatrix is an orthonormal basis of the block Krylov subspace mathcalK^square_k left(beginbmatrix 0 A A^H 0 endbmatrix beginbmatrix B 0 0 C endbmatrixright),","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"T_k+1k =\nbeginbmatrix\n Omega_1 Phi_2 \n Psi_2 Omega_2 ddots \n ddots ddots Phi_k \n Psi_k Omega_k \n Psi_k+1\nendbmatrix\n qquad\nT_kk+1^H =\nbeginbmatrix\n Omega_1^H Psi_2^H \n Phi_2^H Omega_2^H ddots \n ddots ddots Psi_k^H \n Phi_k^H Omega_k^H \n Phi_k+1^H\nendbmatrix","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"The function saunders_simon_yip returns V_k+1, Psi_1, T_k+1k, U_k+1, Phi_1^H and T_kk+1^H.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"saunders_simon_yip(::Any, ::AbstractMatrix{FC}, ::AbstractMatrix{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"block_processes/#Krylov.saunders_simon_yip-Union{Tuple{FC}, Tuple{Any, AbstractMatrix{FC}, AbstractMatrix{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Block Krylov processes","title":"Krylov.saunders_simon_yip","text":"V, Ψ, T, U, Φᴴ, Tᴴ = saunders_simon_yip(A, B, C, k; algo=\"householder\")\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nB: a matrix of size m × p;\nC: a matrix of size n × p;\nk: the number of iterations of the block Saunders-Simon-Yip process.\n\nKeyword argument\n\nalgo: the algorithm to perform reduced QR factorizations (\"gs\", \"mgs\", \"givens\" or \"householder\").\n\nOutput arguments\n\nV: a dense m × p(k+1) matrix;\nΨ: a dense p × p upper triangular matrix such that V₁Ψ = B;\nT: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p;\nU: a dense n × p(k+1) matrix;\nΦᴴ: a dense p × p upper triangular matrix such that U₁Φᴴ = C;\nTᴴ: a sparse p(k+1) × pk block tridiagonal matrix with a bandwidth p.\n\n\n\n\n\n","category":"method"},{"location":"block_processes/#block-montoison-orban","page":"Block Krylov processes","title":"Block Montoison-Orban","text":"","category":"section"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"If the vectors b and c in the Montoison-Orban process are replaced by matrices D and C with both p columns, we can derive the block Montoison-Orban process.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"(Image: block_montoison_orban)","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"After k iterations of the block Montoison-Orban process, the situation may be summarized as","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"beginalign*\n A U_k = V_k+1 H_k+1k \n B V_k = U_k+1 F_k+1k \n V_k^H V_k = U_k^H U_k = I_pk\nendalign*","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"where beginbmatrix V_k 0 0 U_k endbmatrix is an orthonormal basis of the block Krylov subspace mathcalK^square_k left(beginbmatrix 0 A B 0 endbmatrix beginbmatrix D 0 0 C endbmatrixright),","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"H_k+1k =\nbeginbmatrix\n Psi_11 Psi_12 ldots Psi_1k \n Psi_21 ddots ddots vdots \n ddots ddots Psi_k-1k \n Psi_kk-1 Psi_kk \n Psi_k+1k\nendbmatrix\n qquad\nF_k+1k =\nbeginbmatrix\n Phi_11 Phi_12 ldots Phi_1k \n Phi_21 ddots ddots vdots \n ddots ddots Phi_k-1k \n Phi_kk-1 Phi_kk \n Phi_k+1k\nendbmatrix","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"The function montoison_orban returns V_k+1, Gamma, H_k+1k, U_k+1, Lambda, and F_k+1k.","category":"page"},{"location":"block_processes/","page":"Block Krylov processes","title":"Block Krylov processes","text":"montoison_orban(::Any, ::Any, ::AbstractMatrix{FC}, ::AbstractMatrix{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"block_processes/#Krylov.montoison_orban-Union{Tuple{FC}, Tuple{Any, Any, AbstractMatrix{FC}, AbstractMatrix{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Block Krylov processes","title":"Krylov.montoison_orban","text":"V, Γ, H, U, Λ, F = montoison_orban(A, B, D, C, k; algo=\"householder\", reorthogonalization=false)\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nB: a linear operator that models a matrix of dimension n × m;\nD: a matrix of size m × p;\nC: a matrix of size n × p;\nk: the number of iterations of the block Montoison-Orban process.\n\nKeyword arguments\n\nalgo: the algorithm to perform reduced QR factorizations (\"gs\", \"mgs\", \"givens\" or \"householder\").\nreorthogonalization: reorthogonalize the new matrices of the block Krylov basis against all previous matrices.\n\nOutput arguments\n\nV: a dense m × p(k+1) matrix;\nΓ: a dense p × p upper triangular matrix such that V₁Γ = D;\nH: a dense p(k+1) × pk block upper Hessenberg matrix with a lower bandwidth p;\nU: a dense n × p(k+1) matrix;\nΛ: a dense p × p upper triangular matrix such that U₁Λ = C;\nF: a dense p(k+1) × pk block upper Hessenberg matrix with a lower bandwidth p.\n\n\n\n\n\n","category":"method"},{"location":"examples/tricg/","page":"TriCG","title":"TriCG","text":"using Krylov, LinearOperators\nusing LinearAlgebra, Printf, SparseArrays\n\n# Identity matrix.\neye(n::Int) = sparse(1.0 * I, n, n)\n\n# Symmetric quasi-definite systems and variants\nn = m = 5\nA = [2^(i/j)*j + (-1)^(i-j) * n*(i-1) for i = 1:n, j = 1:n]\nb = ones(n)\nM = diagm(0 => [3.0 * i for i = 1:n])\nN = diagm(0 => [5.0 * i for i = 1:n])\nc = -b\n\n# [I A] [x] = [b]\n# [Aᴴ -I] [y] [c]\n(x, y, stats) = tricg(A, b, c)\nK = [eye(m) A; A' -eye(n)]\nB = [b; c]\nr = B - K * [x; y]\nresid = norm(r)\n@printf(\"TriCG: Relative residual: %8.1e\\n\", resid)\n\n# [-I A] [x] = [b]\n# [ Aᴴ I] [y] [c]\n(x, y, stats) = tricg(A, b, c, flip=true)\nK = [-eye(m) A; A' eye(n)]\nB = [b; c]\nr = B - K * [x; y]\nresid = norm(r)\n@printf(\"TriCG: Relative residual: %8.1e\\n\", resid)\n\n# [I A] [x] = [b]\n# [Aᴴ I] [y] [c]\n(x, y, stats) = tricg(A, b, c, spd=true)\nK = [eye(m) A; A' eye(n)]\nB = [b; c]\nr = B - K * [x; y]\nresid = norm(r)\n@printf(\"TriCG: Relative residual: %8.1e\\n\", resid)\n\n# [-I A] [x] = [b]\n# [ Aᴴ -I] [y] [c]\n(x, y, stats) = tricg(A, b, c, snd=true)\nK = [-eye(m) A; A' -eye(n)]\nB = [b; c]\nr = B - K * [x; y]\nresid = norm(r)\n@printf(\"TriCG: Relative residual: %8.1e\\n\", resid)\n\n# [τI A] [x] = [b]\n# [ Aᴴ νI] [y] [c]\n(τ, ν) = (1e-4, 1e2)\n(x, y, stats) = tricg(A, b, c, τ=τ, ν=ν)\nK = [τ*eye(m) A; A' ν*eye(n)]\nB = [b; c]\nr = B - K * [x; y]\nresid = norm(r)\n@printf(\"TriCG: Relative residual: %8.1e\\n\", resid)\n\n# [M⁻¹ A ] [x] = [b]\n# [Aᴴ -N⁻¹] [y] [c]\n(x, y, stats) = tricg(A, b, c, M=M, N=N, verbose=1)\nK = [inv(M) A; A' -inv(N)]\nH = BlockDiagonalOperator(M, N)\nB = [b; c]\nr = B - K * [x; y]\nresid = sqrt(dot(r, H * r))\n@printf(\"TriCG: Relative residual: %8.1e\\n\", resid)","category":"page"},{"location":"solvers/as/","page":"Adjoint systems","title":"Adjoint systems","text":"# Adjoint systems","category":"page"},{"location":"solvers/as/#BiLQR","page":"Adjoint systems","title":"BiLQR","text":"","category":"section"},{"location":"solvers/as/","page":"Adjoint systems","title":"Adjoint systems","text":"bilqr\nbilqr!","category":"page"},{"location":"solvers/as/#Krylov.bilqr","page":"Adjoint systems","title":"Krylov.bilqr","text":"(x, y, stats) = bilqr(A, b::AbstractVector{FC}, c::AbstractVector{FC};\n transfer_to_bicg::Bool=true, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, y, stats) = bilqr(A, b, c, x0::AbstractVector, y0::AbstractVector; kwargs...)\n\nBiLQR can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.\n\nCombine BiLQ and QMR to solve adjoint systems.\n\n[0 A] [y] = [b]\n[Aᴴ 0] [x] [c]\n\nThe relation bᴴc ≠ 0 must be satisfied. BiLQ is used for solving primal system Ax = b of size n. QMR is used for solving dual system Aᴴy = c of size n.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n;\nc: a vector of length n.\n\nOptional arguments\n\nx0: a vector of length n that represents an initial guess of the solution x;\ny0: a vector of length n that represents an initial guess of the solution y.\n\nKeyword arguments\n\ntransfer_to_bicg: transfer from the BiLQ point to the BiCG point, when it exists. The transfer is based on the residual norm;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\ny: a dense vector of length n;\nstats: statistics collected on the run in an AdjointStats structure.\n\nReference\n\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\n\n\n\n\n\n","category":"function"},{"location":"solvers/as/#Krylov.bilqr!","page":"Adjoint systems","title":"Krylov.bilqr!","text":"solver = bilqr!(solver::BilqrSolver, A, b, c; kwargs...)\nsolver = bilqr!(solver::BilqrSolver, A, b, c, x0, y0; kwargs...)\n\nwhere kwargs are keyword arguments of bilqr.\n\nSee BilqrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/as/#TriLQR","page":"Adjoint systems","title":"TriLQR","text":"","category":"section"},{"location":"solvers/as/","page":"Adjoint systems","title":"Adjoint systems","text":"trilqr\ntrilqr!","category":"page"},{"location":"solvers/as/#Krylov.trilqr","page":"Adjoint systems","title":"Krylov.trilqr","text":"(x, y, stats) = trilqr(A, b::AbstractVector{FC}, c::AbstractVector{FC};\n transfer_to_usymcg::Bool=true, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, y, stats) = trilqr(A, b, c, x0::AbstractVector, y0::AbstractVector; kwargs...)\n\nTriLQR can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.\n\nCombine USYMLQ and USYMQR to solve adjoint systems.\n\n[0 A] [y] = [b]\n[Aᴴ 0] [x] [c]\n\nUSYMLQ is used for solving primal system Ax = b of size m × n. USYMQR is used for solving dual system Aᴴy = c of size n × m.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nc: a vector of length n.\n\nOptional arguments\n\nx0: a vector of length n that represents an initial guess of the solution x;\ny0: a vector of length m that represents an initial guess of the solution y.\n\nKeyword arguments\n\ntransfer_to_usymcg: transfer from the USYMLQ point to the USYMCG point, when it exists. The transfer is based on the residual norm;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\ny: a dense vector of length m;\nstats: statistics collected on the run in an AdjointStats structure.\n\nReference\n\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\n\n\n\n\n\n","category":"function"},{"location":"solvers/as/#Krylov.trilqr!","page":"Adjoint systems","title":"Krylov.trilqr!","text":"solver = trilqr!(solver::TrilqrSolver, A, b, c; kwargs...)\nsolver = trilqr!(solver::TrilqrSolver, A, b, c, x0, y0; kwargs...)\n\nwhere kwargs are keyword arguments of trilqr.\n\nSee TrilqrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"api/#Stats-Types","page":"API","title":"Stats Types","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"Krylov.KrylovStats\nKrylov.SimpleStats\nKrylov.LanczosStats\nKrylov.LanczosShiftStats\nKrylov.SymmlqStats\nKrylov.AdjointStats\nKrylov.LNLQStats\nKrylov.LSLQStats\nKrylov.LsmrStats","category":"page"},{"location":"api/#Krylov.KrylovStats","page":"API","title":"Krylov.KrylovStats","text":"Abstract type for statistics returned by a solver\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.SimpleStats","page":"API","title":"Krylov.SimpleStats","text":"Type for statistics returned by the majority of Krylov solvers, the attributes are:\n\nniter\nsolved\ninconsistent\nresiduals\nAresiduals\nAcond\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LanczosStats","page":"API","title":"Krylov.LanczosStats","text":"Type for statistics returned by CG-LANCZOS, the attributes are:\n\nniter\nsolved\nresiduals\nindefinite\nAnorm\nAcond\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LanczosShiftStats","page":"API","title":"Krylov.LanczosShiftStats","text":"Type for statistics returned by CG-LANCZOS with shifts, the attributes are:\n\nniter\nsolved\nresiduals\nindefinite\nAnorm\nAcond\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.SymmlqStats","page":"API","title":"Krylov.SymmlqStats","text":"Type for statistics returned by SYMMLQ, the attributes are:\n\nniter\nsolved\nresiduals\nresidualscg\nerrors\nerrorscg\nAnorm\nAcond\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.AdjointStats","page":"API","title":"Krylov.AdjointStats","text":"Type for statistics returned by adjoint systems solvers BiLQR and TriLQR, the attributes are:\n\nniter\nsolved_primal\nsolved_dual\nresiduals_primal\nresiduals_dual\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LNLQStats","page":"API","title":"Krylov.LNLQStats","text":"Type for statistics returned by the LNLQ method, the attributes are:\n\nniter\nsolved\nresiduals\nerrorwithbnd\nerrorbndx\nerrorbndy\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LSLQStats","page":"API","title":"Krylov.LSLQStats","text":"Type for statistics returned by the LSLQ method, the attributes are:\n\nniter\nsolved\ninconsistent\nresiduals\nAresiduals\nerr_lbnds\nerrorwithbnd\nerrubndslq\nerrubndscg\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LsmrStats","page":"API","title":"Krylov.LsmrStats","text":"Type for statistics returned by LSMR. The attributes are:\n\nniter\nsolved\ninconsistent\nresiduals\nAresiduals\nAcond\nAnorm\nxNorm\ntimer\nstatus\n\n\n\n\n\n","category":"type"},{"location":"api/#Solver-Types","page":"API","title":"Solver Types","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"KrylovSolver\nBlockKrylovSolver\nMinresSolver\nMinaresSolver\nCgSolver\nCrSolver\nCarSolver\nSymmlqSolver\nCgLanczosSolver\nCgLanczosShiftSolver\nMinresQlpSolver\nDiomSolver\nFomSolver\nDqgmresSolver\nGmresSolver\nUsymlqSolver\nUsymqrSolver\nTricgSolver\nTrimrSolver\nTrilqrSolver\nCgsSolver\nBicgstabSolver\nBilqSolver\nQmrSolver\nBilqrSolver\nCglsSolver\nCglsLanczosShiftSolver\nCrlsSolver\nCgneSolver\nCrmrSolver\nLslqSolver\nLsqrSolver\nLsmrSolver\nLnlqSolver\nCraigSolver\nCraigmrSolver\nGpmrSolver\nFgmresSolver\nBlockGmresSolver","category":"page"},{"location":"api/#Krylov.KrylovSolver","page":"API","title":"Krylov.KrylovSolver","text":"Abstract type for using Krylov solvers in-place\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.BlockKrylovSolver","page":"API","title":"Krylov.BlockKrylovSolver","text":"Abstract type for using block Krylov solvers in-place\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.MinresSolver","page":"API","title":"Krylov.MinresSolver","text":"Type for storing the vectors required by the in-place version of MINRES.\n\nThe outer constructors\n\nsolver = MinresSolver(m, n, S; window :: Int=5)\nsolver = MinresSolver(A, b; window :: Int=5)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.MinaresSolver","page":"API","title":"Krylov.MinaresSolver","text":"Type for storing the vectors required by the in-place version of MINARES.\n\nThe outer constructors\n\nsolver = MinaresSolver(m, n, S)\nsolver = MinaresSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CgSolver","page":"API","title":"Krylov.CgSolver","text":"Type for storing the vectors required by the in-place version of CG.\n\nThe outer constructors\n\nsolver = CgSolver(m, n, S)\nsolver = CgSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CrSolver","page":"API","title":"Krylov.CrSolver","text":"Type for storing the vectors required by the in-place version of CR.\n\nThe outer constructors\n\nsolver = CrSolver(m, n, S)\nsolver = CrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CarSolver","page":"API","title":"Krylov.CarSolver","text":"Type for storing the vectors required by the in-place version of CAR.\n\nThe outer constructors\n\nsolver = CarSolver(m, n, S)\nsolver = CarSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.SymmlqSolver","page":"API","title":"Krylov.SymmlqSolver","text":"Type for storing the vectors required by the in-place version of SYMMLQ.\n\nThe outer constructors\n\nsolver = SymmlqSolver(m, n, S)\nsolver = SymmlqSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CgLanczosSolver","page":"API","title":"Krylov.CgLanczosSolver","text":"Type for storing the vectors required by the in-place version of CG-LANCZOS.\n\nThe outer constructors\n\nsolver = CgLanczosSolver(m, n, S)\nsolver = CgLanczosSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CgLanczosShiftSolver","page":"API","title":"Krylov.CgLanczosShiftSolver","text":"Type for storing the vectors required by the in-place version of CG-LANCZOS-SHIFT.\n\nThe outer constructors\n\nsolver = CgLanczosShiftSolver(m, n, nshifts, S)\nsolver = CgLanczosShiftSolver(A, b, nshifts)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.MinresQlpSolver","page":"API","title":"Krylov.MinresQlpSolver","text":"Type for storing the vectors required by the in-place version of MINRES-QLP.\n\nThe outer constructors\n\nsolver = MinresQlpSolver(m, n, S)\nsolver = MinresQlpSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.DiomSolver","page":"API","title":"Krylov.DiomSolver","text":"Type for storing the vectors required by the in-place version of DIOM.\n\nThe outer constructors\n\nsolver = DiomSolver(m, n, memory, S)\nsolver = DiomSolver(A, b, memory = 20)\n\nmay be used in order to create these vectors. memory is set to n if the value given is larger than n. memory is an optional argument in the second constructor.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.FomSolver","page":"API","title":"Krylov.FomSolver","text":"Type for storing the vectors required by the in-place version of FOM.\n\nThe outer constructors\n\nsolver = FomSolver(m, n, memory, S)\nsolver = FomSolver(A, b, memory = 20)\n\nmay be used in order to create these vectors. memory is set to n if the value given is larger than n. memory is an optional argument in the second constructor.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.DqgmresSolver","page":"API","title":"Krylov.DqgmresSolver","text":"Type for storing the vectors required by the in-place version of DQGMRES.\n\nThe outer constructors\n\nsolver = DqgmresSolver(m, n, memory, S)\nsolver = DqgmresSolver(A, b, memory = 20)\n\nmay be used in order to create these vectors. memory is set to n if the value given is larger than n. memory is an optional argument in the second constructor.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.GmresSolver","page":"API","title":"Krylov.GmresSolver","text":"Type for storing the vectors required by the in-place version of GMRES.\n\nThe outer constructors\n\nsolver = GmresSolver(m, n, memory, S)\nsolver = GmresSolver(A, b, memory = 20)\n\nmay be used in order to create these vectors. memory is set to n if the value given is larger than n. memory is an optional argument in the second constructor.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.UsymlqSolver","page":"API","title":"Krylov.UsymlqSolver","text":"Type for storing the vectors required by the in-place version of USYMLQ.\n\nThe outer constructors\n\nsolver = UsymlqSolver(m, n, S)\nsolver = UsymlqSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.UsymqrSolver","page":"API","title":"Krylov.UsymqrSolver","text":"Type for storing the vectors required by the in-place version of USYMQR.\n\nThe outer constructors\n\nsolver = UsymqrSolver(m, n, S)\nsolver = UsymqrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.TricgSolver","page":"API","title":"Krylov.TricgSolver","text":"Type for storing the vectors required by the in-place version of TRICG.\n\nThe outer constructors\n\nsolver = TricgSolver(m, n, S)\nsolver = TricgSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.TrimrSolver","page":"API","title":"Krylov.TrimrSolver","text":"Type for storing the vectors required by the in-place version of TRIMR.\n\nThe outer constructors\n\nsolver = TrimrSolver(m, n, S)\nsolver = TrimrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.TrilqrSolver","page":"API","title":"Krylov.TrilqrSolver","text":"Type for storing the vectors required by the in-place version of TRILQR.\n\nThe outer constructors\n\nsolver = TrilqrSolver(m, n, S)\nsolver = TrilqrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CgsSolver","page":"API","title":"Krylov.CgsSolver","text":"Type for storing the vectors required by the in-place version of CGS.\n\nThe outer constructorss\n\nsolver = CgsSolver(m, n, S)\nsolver = CgsSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.BicgstabSolver","page":"API","title":"Krylov.BicgstabSolver","text":"Type for storing the vectors required by the in-place version of BICGSTAB.\n\nThe outer constructors\n\nsolver = BicgstabSolver(m, n, S)\nsolver = BicgstabSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.BilqSolver","page":"API","title":"Krylov.BilqSolver","text":"Type for storing the vectors required by the in-place version of BILQ.\n\nThe outer constructors\n\nsolver = BilqSolver(m, n, S)\nsolver = BilqSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.QmrSolver","page":"API","title":"Krylov.QmrSolver","text":"Type for storing the vectors required by the in-place version of QMR.\n\nThe outer constructors\n\nsolver = QmrSolver(m, n, S)\nsolver = QmrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.BilqrSolver","page":"API","title":"Krylov.BilqrSolver","text":"Type for storing the vectors required by the in-place version of BILQR.\n\nThe outer constructors\n\nsolver = BilqrSolver(m, n, S)\nsolver = BilqrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CglsSolver","page":"API","title":"Krylov.CglsSolver","text":"Type for storing the vectors required by the in-place version of CGLS.\n\nThe outer constructors\n\nsolver = CglsSolver(m, n, S)\nsolver = CglsSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CglsLanczosShiftSolver","page":"API","title":"Krylov.CglsLanczosShiftSolver","text":"Workspace for the in-place version of CGLS-LANCZOS-SHIFT.\n\nThe outer constructors:\n\nsolver = CglsLanczosShiftSolver(m, n, nshifts, S)\nsolver = CglsLanczosShiftSolver(A, b, nshifts)\n\ncan be used to initialize this workspace.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CrlsSolver","page":"API","title":"Krylov.CrlsSolver","text":"Type for storing the vectors required by the in-place version of CRLS.\n\nThe outer constructors\n\nsolver = CrlsSolver(m, n, S)\nsolver = CrlsSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CgneSolver","page":"API","title":"Krylov.CgneSolver","text":"Type for storing the vectors required by the in-place version of CGNE.\n\nThe outer constructors\n\nsolver = CgneSolver(m, n, S)\nsolver = CgneSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CrmrSolver","page":"API","title":"Krylov.CrmrSolver","text":"Type for storing the vectors required by the in-place version of CRMR.\n\nThe outer constructors\n\nsolver = CrmrSolver(m, n, S)\nsolver = CrmrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LslqSolver","page":"API","title":"Krylov.LslqSolver","text":"Type for storing the vectors required by the in-place version of LSLQ.\n\nThe outer constructors\n\nsolver = LslqSolver(m, n, S)\nsolver = LslqSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LsqrSolver","page":"API","title":"Krylov.LsqrSolver","text":"Type for storing the vectors required by the in-place version of LSQR.\n\nThe outer constructors\n\nsolver = LsqrSolver(m, n, S)\nsolver = LsqrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LsmrSolver","page":"API","title":"Krylov.LsmrSolver","text":"Type for storing the vectors required by the in-place version of LSMR.\n\nThe outer constructors\n\nsolver = LsmrSolver(m, n, S)\nsolver = LsmrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.LnlqSolver","page":"API","title":"Krylov.LnlqSolver","text":"Type for storing the vectors required by the in-place version of LNLQ.\n\nThe outer constructors\n\nsolver = LnlqSolver(m, n, S)\nsolver = LnlqSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CraigSolver","page":"API","title":"Krylov.CraigSolver","text":"Type for storing the vectors required by the in-place version of CRAIG.\n\nThe outer constructors\n\nsolver = CraigSolver(m, n, S)\nsolver = CraigSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.CraigmrSolver","page":"API","title":"Krylov.CraigmrSolver","text":"Type for storing the vectors required by the in-place version of CRAIGMR.\n\nThe outer constructors\n\nsolver = CraigmrSolver(m, n, S)\nsolver = CraigmrSolver(A, b)\n\nmay be used in order to create these vectors.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.GpmrSolver","page":"API","title":"Krylov.GpmrSolver","text":"Type for storing the vectors required by the in-place version of GPMR.\n\nThe outer constructors\n\nsolver = GpmrSolver(m, n, memory, S)\nsolver = GpmrSolver(A, b, memory = 20)\n\nmay be used in order to create these vectors. memory is set to n + m if the value given is larger than n + m. memory is an optional argument in the second constructor.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.FgmresSolver","page":"API","title":"Krylov.FgmresSolver","text":"Type for storing the vectors required by the in-place version of FGMRES.\n\nThe outer constructors\n\nsolver = FgmresSolver(m, n, memory, S)\nsolver = FgmresSolver(A, b, memory = 20)\n\nmay be used in order to create these vectors. memory is set to n if the value given is larger than n. memory is an optional argument in the second constructor.\n\n\n\n\n\n","category":"type"},{"location":"api/#Krylov.BlockGmresSolver","page":"API","title":"Krylov.BlockGmresSolver","text":"Type for storing the vectors required by the in-place version of BLOCK-GMRES.\n\nThe outer constructors\n\nsolver = BlockGmresSolver(m, n, p, memory, SV, SM)\nsolver = BlockGmresSolver(A, B, memory = 5)\n\nmay be used in order to create these vectors. memory is set to div(n,p) if the value given is larger than div(n,p). memory is an optional argument in the second constructor.\n\n\n\n\n\n","category":"type"},{"location":"api/#Utilities","page":"API","title":"Utilities","text":"","category":"section"},{"location":"api/","page":"API","title":"API","text":"Krylov.roots_quadratic\nKrylov.sym_givens\nKrylov.to_boundary\nKrylov.vec2str\nKrylov.ktypeof\nKrylov.kzeros\nKrylov.kones\nKrylov.vector_to_matrix\nKrylov.matrix_to_vector","category":"page"},{"location":"api/#Krylov.roots_quadratic","page":"API","title":"Krylov.roots_quadratic","text":"roots = roots_quadratic(q₂, q₁, q₀; nitref)\n\nFind the real roots of the quadratic\n\nq(x) = q₂ x² + q₁ x + q₀,\n\nwhere q₂, q₁ and q₀ are real. Care is taken to avoid numerical cancellation. Optionally, nitref steps of iterative refinement may be performed to improve accuracy. By default, nitref=1.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.sym_givens","page":"API","title":"Krylov.sym_givens","text":"(c, s, ρ) = sym_givens(a, b)\n\nNumerically stable symmetric Givens reflection. Given a and b reals, return (c, s, ρ) such that\n\n[ c s ] [ a ] = [ ρ ]\n[ s -c ] [ b ] = [ 0 ].\n\n\n\n\n\nNumerically stable symmetric Givens reflection. Given a and b complexes, return (c, s, ρ) with c real and (s, ρ) complexes such that\n\n[ c s ] [ a ] = [ ρ ]\n[ s̅ -c ] [ b ] = [ 0 ].\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.to_boundary","page":"API","title":"Krylov.to_boundary","text":"roots = to_boundary(n, x, d, radius; flip, xNorm2, dNorm2)\n\nGiven a trust-region radius radius, a vector x lying inside the trust-region and a direction d, return σ1 and σ2 such that\n\n‖x + σi d‖ = radius, i = 1, 2\n\nin the Euclidean norm. n is the length of vectors x and d. If known, ‖x‖² and ‖d‖² may be supplied with xNorm2 and dNorm2.\n\nIf flip is set to true, σ1 and σ2 are computed such that\n\n‖x - σi d‖ = radius, i = 1, 2.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.vec2str","page":"API","title":"Krylov.vec2str","text":"s = vec2str(x; ndisp)\n\nDisplay an array in the form\n\n[ -3.0e-01 -5.1e-01 1.9e-01 ... -2.3e-01 -4.4e-01 2.4e-01 ]\n\nwith (ndisp - 1)/2 elements on each side.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.ktypeof","page":"API","title":"Krylov.ktypeof","text":"S = ktypeof(v)\n\nReturn the most relevant storage type S based on the type of v.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.kzeros","page":"API","title":"Krylov.kzeros","text":"v = kzeros(S, n)\n\nCreate a vector of storage type S of length n only composed of zero.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.kones","page":"API","title":"Krylov.kones","text":"v = kones(S, n)\n\nCreate a vector of storage type S of length n only composed of one.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.vector_to_matrix","page":"API","title":"Krylov.vector_to_matrix","text":"M = vector_to_matrix(S)\n\nReturn the dense matrix storage type M related to the dense vector storage type S.\n\n\n\n\n\n","category":"function"},{"location":"api/#Krylov.matrix_to_vector","page":"API","title":"Krylov.matrix_to_vector","text":"S = matrix_to_vector(M)\n\nReturn the dense vector storage type S related to the dense matrix storage type M.\n\n\n\n\n\n","category":"function"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"# Thanks Morten Piibeleht for the hack with the tables!","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"","category":"page"},{"location":"storage/#storage-requirements","page":"Storage requirements","title":"Storage requirements","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"This section provides the storage requirements of all Krylov methods available in Krylov.jl.","category":"page"},{"location":"storage/#Notation","page":"Storage requirements","title":"Notation","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"We denote by m and n the number of rows and columns of the linear problem. The memory parameter of DIOM, FOM, DQGMRES, GMRES, FGMRES and GPMR is k. The numbers of shifts of CG-LANCZOS-SHIFT and CGLS-LANCZOS-SHIFT is p.","category":"page"},{"location":"storage/#Theoretical-storage-requirements","page":"Storage requirements","title":"Theoretical storage requirements","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"The following tables provide the number of coefficients that must be allocated for each Krylov method. The coefficients have the same type as those that compose the linear problem we seek to solve. Each table summarizes the storage requirements of Krylov methods recommended to a specific linear problem.","category":"page"},{"location":"storage/#Hermitian-positive-definite-linear-systems","page":"Storage requirements","title":"Hermitian positive definite linear systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods CG CR CAR CG-LANCZOS CG-LANCZOS-SHIFT\nStorage 4n 5n 7n 5n 3n + 2np + 5p","category":"page"},{"location":"storage/#Hermitian-indefinite-linear-systems","page":"Storage requirements","title":"Hermitian indefinite linear systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods SYMMLQ MINRES MINRES-QLP MINARES\nStorage 5n 6n 6n 8n","category":"page"},{"location":"storage/#Non-Hermitian-square-linear-systems","page":"Storage requirements","title":"Non-Hermitian square linear systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods CGS BICGSTAB BiLQ QMR\nStorage 6n 6n 8n 9n","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods DIOM DQGMRES\nStorage n(2k+1) + 2k - 1 n(2k+2) + 3k + 1","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods FOM GMRES FGMRES\nStoragedfrac n(2+k) +2k + dfrack(k + 1)2 n(2+k) + 3k + dfrack(k + 1)2 n(2+2k) + 3k + dfrack(k + 1)2","category":"page"},{"location":"storage/#Least-norm-problems","page":"Storage requirements","title":"Least-norm problems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods USYMLQ CGNE CRMR LNLQ CRAIG CRAIGMR\nStorage 5n + 3m 3n + 2m 3n + 2m 3n + 4m 3n + 4m 4n + 5m","category":"page"},{"location":"storage/#Least-squares-problems","page":"Storage requirements","title":"Least-squares problems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods USYMQR CGLS CGLS-LANCZOS-SHIFT CRLS LSLQ LSQR LSMR\nStorage 6n + 3m 3n + 2m 3n + 2m + 5p + 2np 4n + 3m 4n + 2m 4n + 2m 5n + 2m","category":"page"},{"location":"storage/#Adjoint-systems","page":"Storage requirements","title":"Adjoint systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods BiLQR TriLQR\nStorage 11n 6m + 5n","category":"page"},{"location":"storage/#Saddle-point-and-Hermitian-quasi-definite-systems","page":"Storage requirements","title":"Saddle-point and Hermitian quasi-definite systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Methods TriCG TriMR\nStorage 6n + 6m 8n + 8m","category":"page"},{"location":"storage/#Generalized-saddle-point-and-non-Hermitian-partitioned-systems","page":"Storage requirements","title":"Generalized saddle-point and non-Hermitian partitioned systems","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Method GPMR\nStorage (2+k)(n+m) + 2k^2 + 11k","category":"page"},{"location":"storage/#Practical-storage-requirements","page":"Storage requirements","title":"Practical storage requirements","text":"","category":"section"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Each method has its own KrylovSolver that contains all the storage needed by the method. In the REPL, the size in bytes of each attribute and the total amount of memory allocated by the solver are displayed when we show a KrylovSolver.","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"using Krylov\n\nm = 5000\nn = 12000\nA = rand(Float64, m, n)\nb = rand(Float64, m)\nsolver = LsmrSolver(A, b)\nshow(stdout, solver, show_stats=false)","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"If we want the total number of bytes used by the solver, we can call nbytes = sizeof(solver).","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"nbytes = sizeof(solver)","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Thereafter, we can use Base.format_bytes(nbytes) to recover what is displayed in the REPL.","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Base.format_bytes(nbytes)","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"To verify that we match the theoretical results, we just need to multiply the storage requirement of a method by the number of bytes associated to the precision of the linear problem. For instance, we need 4 bytes for the precision Float32, 8 bytes for precisions Float64 and ComplexF32, and 16 bytes for the precision ComplexF64.","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"FC = Float64 # precision of the least-squares problem\nncoefs_lsmr = 5*n + 2*m # number of coefficients\nnbytes_lsmr = sizeof(FC) * ncoefs_lsmr # number of bytes","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"Therefore, you can check that you have enough memory in RAM to allocate a KrylovSolver.","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"free_nbytes = Sys.free_memory()\nBase.format_bytes(free_nbytes) # Total free memory in RAM in bytes.","category":"page"},{"location":"storage/","page":"Storage requirements","title":"Storage requirements","text":"note: Note\nBeyond having faster operations, using low precisions, such as simple precision, allows to store more coefficients in RAM and solve larger linear problems.\nIn the file test_allocations.jl, we use the macro @allocated to test that we match the expected storage requirement of each method with a tolerance of 2%.","category":"page"},{"location":"examples/craig/","page":"CRAIG","title":"CRAIG","text":"using Krylov\nusing LinearAlgebra, Printf\n\nm = 5\nn = 8\nλ = 1.0e-3\nA = rand(m, n)\nb = A * ones(n)\nxy_exact = [A λ*I] \\ b # In Julia, this is the min-norm solution!\n\n(x, y, stats) = craig(A, b, λ=λ, atol=0.0, rtol=1.0e-20, verbose=1)\nshow(stats)\n\n# Check that we have a minimum-norm solution.\n# When λ > 0 we solve min ‖(x,s)‖ s.t. Ax + λs = b, and we get s = λy.\n@printf(\"Primal feasibility: %7.1e\\n\", norm(b - A * x - λ^2 * y) / norm(b))\n@printf(\"Dual feasibility: %7.1e\\n\", norm(x - A' * y) / norm(x))\n@printf(\"Error in x: %7.1e\\n\", norm(x - xy_exact[1:n]) / norm(xy_exact[1:n]))\nif λ > 0.0\n @printf(\"Error in y: %7.1e\\n\", norm(λ * y - xy_exact[n+1:n+m]) / norm(xy_exact[n+1:n+m]))\nend","category":"page"},{"location":"examples/cgne/","page":"CGNE","title":"CGNE","text":"using Krylov, HarwellRutherfordBoeing, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"wm2\")\npath = fetch_ssmc(matrix, format=\"RB\")\n\nA = RutherfordBoeingData(joinpath(path[1], \"$(matrix.name[1]).rb\")).data\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\nx_exact = A' * ones(m)\nx_exact_norm = norm(x_exact)\nx_exact /= x_exact_norm\nb = A * x_exact\n(x, stats) = cgne(A, b)\nshow(stats)\nresid = norm(A * x - b) / norm(b)\n@printf(\"CGNE: Relative residual: %7.1e\\n\", resid)\n@printf(\"CGNE: ‖x - x*‖₂: %7.1e\\n\", norm(x - x_exact))","category":"page"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"# Least-squares problems","category":"page"},{"location":"solvers/ls/#CGLS","page":"Least-squares problems","title":"CGLS","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"cgls\ncgls!","category":"page"},{"location":"solvers/ls/#Krylov.cgls","page":"Least-squares problems","title":"Krylov.cgls","text":"(x, stats) = cgls(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, radius::T=zero(T),\n λ::T=zero(T), atol::T=√eps(T), rtol::T=√eps(T),\n itmax::Int=0, timemax::Float64=Inf,\n verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the regularized linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ‖x‖₂²\n\nof size m × n using the Conjugate Gradient (CG) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CG to the normal equations\n\n(AᴴA + λI) x = Aᴴb\n\nbut is more stable.\n\nCGLS produces monotonic residuals ‖r‖₂ but not optimality residuals ‖Aᴴr‖₂. It is formally equivalent to LSQR, though can be slightly less accurate, but simpler to implement.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nradius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nM. R. Hestenes and E. Stiefel. Methods of conjugate gradients for solving linear systems, Journal of Research of the National Bureau of Standards, 49(6), pp. 409–436, 1952.\nA. Björck, T. Elfving and Z. Strakos, Stability of Conjugate Gradient and Lanczos Methods for Linear Least Squares Problems, SIAM Journal on Matrix Analysis and Applications, 19(3), pp. 720–736, 1998.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.cgls!","page":"Least-squares problems","title":"Krylov.cgls!","text":"solver = cgls!(solver::CglsSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of cgls.\n\nSee CglsSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#CGLS-LANCZOS-SHIFT","page":"Least-squares problems","title":"CGLS-LANCZOS-SHIFT","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"cgls_lanczos_shift\ncgls_lanczos_shift!","category":"page"},{"location":"solvers/ls/#Krylov.cgls_lanczos_shift","page":"Least-squares problems","title":"Krylov.cgls_lanczos_shift","text":"(x, stats) = cgls_lanczos_shift(A, b::AbstractVector{FC};\n M=I, λ::T=zero(T), atol::T=√eps(T), rtol::T=√eps(T),\n radius::T=zero(T), itmax::Int=0, verbose::Int=0, history::Bool=false,\n callback=solver->false)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the regularized linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ‖x‖₂²\n\nusing the Conjugate Gradient (CG) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CG to the normal equations\n\n(AᴴA + λI) x = Aᴴb\n\nbut is more stable.\n\nCGLS produces monotonic residuals ‖r‖₂ but not optimality residuals ‖Aᴴr‖₂. It is formally equivalent to LSQR, though can be slightly less accurate, but simpler to implement.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nshifts: a vector of length p.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a vector of p dense vectors, each one of length n;\nstats: statistics collected on the run in a LanczosShiftStats structure.\n\nReferences\n\nM. R. Hestenes and E. Stiefel. Methods of conjugate gradients for solving linear systems, Journal of Research of the National Bureau of Standards, 49(6), pp. 409–436, 1952.\nA. Björck, T. Elfving and Z. Strakos, Stability of Conjugate Gradient and Lanczos Methods for Linear Least Squares Problems, SIAM Journal on Matrix Analysis and Applications, 19(3), pp. 720–736, 1998.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.cgls_lanczos_shift!","page":"Least-squares problems","title":"Krylov.cgls_lanczos_shift!","text":"solver = cgls_lanczos_shift!(solver::CglsLanczosShiftSolver, A, b, shifts; kwargs...)\n\nwhere kwargs are keyword arguments of cgls_lanczos_shift.\n\nSee CglsLanczosShiftSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#CRLS","page":"Least-squares problems","title":"CRLS","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"crls\ncrls!","category":"page"},{"location":"solvers/ls/#Krylov.crls","page":"Least-squares problems","title":"Krylov.crls","text":"(x, stats) = crls(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, radius::T=zero(T),\n λ::T=zero(T), atol::T=√eps(T), rtol::T=√eps(T),\n itmax::Int=0, timemax::Float64=Inf,\n verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ‖x‖₂²\n\nof size m × n using the Conjugate Residuals (CR) method. This method is equivalent to applying MINRES to the normal equations\n\n(AᴴA + λI) x = Aᴴb.\n\nThis implementation recurs the residual r := b - Ax.\n\nCRLS produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂. It is formally equivalent to LSMR, though can be substantially less accurate, but simpler to implement.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nradius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nD. C.-L. Fong, Minimum-Residual Methods for Sparse, Least-Squares using Golubg-Kahan Bidiagonalization, Ph.D. Thesis, Stanford University, 2011.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.crls!","page":"Least-squares problems","title":"Krylov.crls!","text":"solver = crls!(solver::CrlsSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of crls.\n\nSee CrlsSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#LSLQ","page":"Least-squares problems","title":"LSLQ","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"lslq\nlslq!","category":"page"},{"location":"solvers/ls/#Krylov.lslq","page":"Least-squares problems","title":"Krylov.lslq","text":"(x, stats) = lslq(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n window::Int=5, transfer_to_lsqr::Bool=false,\n sqd::Bool=false, λ::T=zero(T),\n σ::T=zero(T), etol::T=√eps(T),\n utol::T=√eps(T), btol::T=√eps(T),\n conlim::T=1/√eps(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the regularized linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ²‖x‖₂²\n\nof size m × n using the LSLQ method, where λ ≥ 0 is a regularization parameter. LSLQ is formally equivalent to applying SYMMLQ to the normal equations\n\n(AᴴA + λ²I) x = Aᴴb\n\nbut is more stable.\n\nIf λ > 0, we solve the symmetric and quasi-definite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ -λ²F ] [ x ] = [ 0 ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹ + λ²‖x‖²_F.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LSLQ is then equivalent to applying SYMMLQ to (AᴴE⁻¹A + λ²F)x = AᴴE⁻¹b with r = E⁻¹(b - Ax).\n\nIf λ = 0, we solve the symmetric and indefinite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ 0 ] [ x ] = [ 0 ].\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹.\n\nIn this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).\n\nMain features\n\nthe solution estimate is updated along orthogonal directions\nthe norm of the solution estimate ‖xᴸₖ‖₂ is increasing\nthe error ‖eₖ‖₂ := ‖xᴸₖ - x*‖₂ is decreasing\nit is possible to transition cheaply from the LSLQ iterate to the LSQR iterate if there is an advantage (there always is in terms of error)\nif A is rank deficient, identify the minimum least-squares solution\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\nwindow: number of iterations used to accumulate a lower bound on the error;\ntransfer_to_lsqr: transfer from the LSLQ point to the LSQR point, when it exists. The transfer is based on the residual norm;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\nσ: strict lower bound on the smallest positive singular value σₘᵢₙ such as σ = (1-10⁻⁷)σₘᵢₙ;\netol: stopping tolerance based on the lower bound on the error;\nutol: stopping tolerance based on the upper bound on the error;\nbtol: stopping tolerance used to detect zero-residual problems;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a LSLQStats structure.\nstats.err_lbnds is a vector of lower bounds on the LQ error–-the vector is empty if window is set to zero\nstats.err_ubnds_lq is a vector of upper bounds on the LQ error–-the vector is empty if σ == 0 is left at zero\nstats.err_ubnds_cg is a vector of upper bounds on the CG error–-the vector is empty if σ == 0 is left at zero\nstats.error_with_bnd is a boolean indicating whether there was an error in the upper bounds computation (cancellation errors, too large σ ...)\n\nStopping conditions\n\nThe iterations stop as soon as one of the following conditions holds true:\n\nthe optimality residual is sufficiently small (stats.status = \"found approximate minimum least-squares solution\") in the sense that either\n‖Aᴴr‖ / (‖A‖ ‖r‖) ≤ atol, or\n1 + ‖Aᴴr‖ / (‖A‖ ‖r‖) ≤ 1\nan approximate zero-residual solution has been found (stats.status = \"found approximate zero-residual solution\") in the sense that either\n‖r‖ / ‖b‖ ≤ btol + atol ‖A‖ * ‖xᴸ‖ / ‖b‖, or\n1 + ‖r‖ / ‖b‖ ≤ 1\nthe estimated condition number of A is too large in the sense that either\n1/cond(A) ≤ 1/conlim (stats.status = \"condition number exceeds tolerance\"), or\n1 + 1/cond(A) ≤ 1 (stats.status = \"condition number seems too large for this machine\")\nthe lower bound on the LQ forward error is less than etol * ‖xᴸ‖\nthe upper bound on the CG forward error is less than utol * ‖xᶜ‖\n\nReferences\n\nR. Estrin, D. Orban and M. A. Saunders, Euclidean-norm error bounds for SYMMLQ and CG, SIAM Journal on Matrix Analysis and Applications, 40(1), pp. 235–253, 2019.\nR. Estrin, D. Orban and M. A. Saunders, LSLQ: An Iterative Method for Linear Least-Squares with an Error Minimization Property, SIAM Journal on Matrix Analysis and Applications, 40(1), pp. 254–275, 2019.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.lslq!","page":"Least-squares problems","title":"Krylov.lslq!","text":"solver = lslq!(solver::LslqSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of lslq.\n\nSee LslqSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#LSQR","page":"Least-squares problems","title":"LSQR","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"lsqr\nlsqr!","category":"page"},{"location":"solvers/ls/#Krylov.lsqr","page":"Least-squares problems","title":"Krylov.lsqr","text":"(x, stats) = lsqr(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n window::Int=5, sqd::Bool=false, λ::T=zero(T),\n radius::T=zero(T), etol::T=√eps(T),\n axtol::T=√eps(T), btol::T=√eps(T),\n conlim::T=1/√eps(T), atol::T=zero(T),\n rtol::T=zero(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the regularized linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ²‖x‖₂²\n\nof size m × n using the LSQR method, where λ ≥ 0 is a regularization parameter. LSQR is formally equivalent to applying CG to the normal equations\n\n(AᴴA + λ²I) x = Aᴴb\n\n(and therefore to CGLS) but is more stable.\n\nLSQR produces monotonic residuals ‖r‖₂ but not optimality residuals ‖Aᴴr‖₂. It is formally equivalent to CGLS, though can be slightly more accurate.\n\nIf λ > 0, LSQR solves the symmetric and quasi-definite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ -λ²F ] [ x ] = [ 0 ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹ + λ²‖x‖²_F.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LSQR is then equivalent to applying CG to (AᴴE⁻¹A + λ²F)x = AᴴE⁻¹b with r = E⁻¹(b - Ax).\n\nIf λ = 0, we solve the symmetric and indefinite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ 0 ] [ x ] = [ 0 ].\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹.\n\nIn this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\nwindow: number of iterations used to accumulate a lower bound on the error;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\nradius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;\netol: stopping tolerance based on the lower bound on the error;\naxtol: tolerance on the backward error;\nbtol: stopping tolerance used to detect zero-residual problems;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nC. C. Paige and M. A. Saunders, LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares, ACM Transactions on Mathematical Software, 8(1), pp. 43–71, 1982.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.lsqr!","page":"Least-squares problems","title":"Krylov.lsqr!","text":"solver = lsqr!(solver::LsqrSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of lsqr.\n\nSee LsqrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#LSMR","page":"Least-squares problems","title":"LSMR","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"lsmr\nlsmr!","category":"page"},{"location":"solvers/ls/#Krylov.lsmr","page":"Least-squares problems","title":"Krylov.lsmr","text":"(x, stats) = lsmr(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n window::Int=5, sqd::Bool=false, λ::T=zero(T),\n radius::T=zero(T), etol::T=√eps(T),\n axtol::T=√eps(T), btol::T=√eps(T),\n conlim::T=1/√eps(T), atol::T=zero(T),\n rtol::T=zero(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the regularized linear least-squares problem\n\nminimize ‖b - Ax‖₂² + λ²‖x‖₂²\n\nof size m × n using the LSMR method, where λ ≥ 0 is a regularization parameter. LSMR is formally equivalent to applying MINRES to the normal equations\n\n(AᴴA + λ²I) x = Aᴴb\n\n(and therefore to CRLS) but is more stable.\n\nLSMR produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂. It is formally equivalent to CRLS, though can be substantially more accurate.\n\nLSMR can be also used to find a null vector of a singular matrix A by solving the problem min ‖Aᴴx - b‖ with any nonzero vector b. At a minimizer, the residual vector r = b - Aᴴx will satisfy Ar = 0.\n\nIf λ > 0, we solve the symmetric and quasi-definite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ -λ²F ] [ x ] = [ 0 ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹ + λ²‖x‖²_F.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LSMR is then equivalent to applying MINRES to (AᴴE⁻¹A + λ²F)x = AᴴE⁻¹b with r = E⁻¹(b - Ax).\n\nIf λ = 0, we solve the symmetric and indefinite system\n\n[ E A ] [ r ] [ b ]\n[ Aᴴ 0 ] [ x ] = [ 0 ].\n\nThe system above represents the optimality conditions of\n\nminimize ‖b - Ax‖²_E⁻¹.\n\nIn this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\nwindow: number of iterations used to accumulate a lower bound on the error;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\nradius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;\netol: stopping tolerance based on the lower bound on the error;\naxtol: tolerance on the backward error;\nbtol: stopping tolerance used to detect zero-residual problems;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a LsmrStats structure.\n\nReference\n\nD. C.-L. Fong and M. A. Saunders, LSMR: An Iterative Algorithm for Sparse Least Squares Problems, SIAM Journal on Scientific Computing, 33(5), pp. 2950–2971, 2011.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.lsmr!","page":"Least-squares problems","title":"Krylov.lsmr!","text":"solver = lsmr!(solver::LsmrSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of lsmr.\n\nSee LsmrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#USYMQR","page":"Least-squares problems","title":"USYMQR","text":"","category":"section"},{"location":"solvers/ls/","page":"Least-squares problems","title":"Least-squares problems","text":"usymqr\nusymqr!","category":"page"},{"location":"solvers/ls/#Krylov.usymqr","page":"Least-squares problems","title":"Krylov.usymqr","text":"(x, stats) = usymqr(A, b::AbstractVector{FC}, c::AbstractVector{FC};\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = usymqr(A, b, c, x0::AbstractVector; kwargs...)\n\nUSYMQR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nUSYMQR solves the linear least-squares problem min ‖b - Ax‖² of size m × n. USYMQR solves Ax = b if it is consistent.\n\nUSYMQR is based on the orthogonal tridiagonalization process and requires two initial nonzero vectors b and c. The vector c is only used to initialize the process and a default value can be b or Aᴴb depending on the shape of A. The residual norm ‖b - Ax‖ monotonously decreases in USYMQR. When A is Hermitian and b = c, QMR is equivalent to MINRES. USYMQR is considered as a generalization of MINRES.\n\nIt can also be applied to under-determined and over-determined problems. USYMQR finds the minimum-norm solution if problems are inconsistent.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nc: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nM. A. Saunders, H. D. Simon, and E. L. Yip, Two Conjugate-Gradient-Type Methods for Unsymmetric Linear Equations, SIAM Journal on Numerical Analysis, 25(4), pp. 927–940, 1988.\nL. Reichel and Q. Ye, A generalized LSQR algorithm, Numerical Linear Algebra with Applications, 15(7), pp. 643–660, 2008.\nA. Buttari, D. Orban, D. Ruiz and D. Titley-Peloquin, A tridiagonalization method for symmetric saddle-point and quasi-definite systems, SIAM Journal on Scientific Computing, 41(5), pp. 409–432, 2019.\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ls/#Krylov.usymqr!","page":"Least-squares problems","title":"Krylov.usymqr!","text":"solver = usymqr!(solver::UsymqrSolver, A, b, c; kwargs...)\nsolver = usymqr!(solver::UsymqrSolver, A, b, c, x0; kwargs...)\n\nwhere kwargs are keyword arguments of usymqr.\n\nSee UsymqrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"preconditioners/#preconditioners","page":"Preconditioners","title":"Preconditioners","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"The solvers in Krylov.jl support preconditioners, i.e., transformations that modify a linear system Ax = b into an equivalent form that may yield faster convergence in finite-precision arithmetic. Preconditioning can be used to reduce the condition number of the problem or cluster its eigenvalues or singular values for instance.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"The design of preconditioners is highly dependent on the origin of the problem and most preconditioners need to take application-dependent information and structure into account. Specialized preconditioners generally outperform generic preconditioners such as incomplete factorizations.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"The construction of a preconditioner necessitates trade-offs because we need to apply it at least once per iteration within a Krylov method. Hence, a preconditioner must be constructed such that it is cheap to apply, while also capturing the characteristics of the original system in some sense.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"There exist three variants of preconditioning:","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Left preconditioning Two-sided preconditioning Right preconditioning\nP_ell^-1Ax = P_ell^-1b P_ell^-1AP_r^-1y = P_ell^-1btextwithx = P_r^-1y AP_r^-1y = btextwithx = P_r^-1y","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"where P_ell and P_r are square and nonsingular.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"The left preconditioning preserves the error x_k - x^star whereas the right preconditioning keeps invariant the residual b - A x_k. Two-sided preconditioning is the only variant that allows to preserve the hermicity of a linear system.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"note: Note\nBecause det(P^-1A - lambda I) = det(A - lambda P) det(P^-1) = det(AP^-1 - lambda I), the eigenvalues of P^-1A and AP^-1 are identical. If P = LL^H, L^-1AL^-H also has the same spectrum.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"In Krylov.jl, we call P_ell^-1 and P_r^-1 the preconditioners and we assume that we can apply them with the operation y leftarrow P^-1 * x. It is also common to call P_ell and P_r the preconditioners if the equivalent operation y leftarrow Pbackslashx is available. Krylov.jl supports both approaches thanks to the argument ldiv of the Krylov solvers.","category":"page"},{"location":"preconditioners/#How-to-use-preconditioners-in-Krylov.jl?","page":"Preconditioners","title":"How to use preconditioners in Krylov.jl?","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"info: Info\nA preconditioner only needs to support the operation mul!(y, P⁻¹, x) when ldiv=false or ldiv!(y, P, x) when ldiv=true to be used in Krylov.jl.\nAdditional support for adjoint with preconditioners is required in the methods BILQ and QMR.\nThe default value of a preconditioner in Krylov.jl is the identity operator I.","category":"page"},{"location":"preconditioners/#Square-non-Hermitian-linear-systems","page":"Preconditioners","title":"Square non-Hermitian linear systems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Methods concerned: CGS, BILQ, QMR, BiCGSTAB, DQGMRES, GMRES, BLOCK-GMRES, FGMRES, DIOM and FOM.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"A Krylov method dedicated to non-Hermitian linear systems allows the three variants of preconditioning.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Preconditioners P_ell^-1 P_ell P_r^-1 P_r\nArguments M with ldiv=false M with ldiv=true N with ldiv=false N with ldiv=true","category":"page"},{"location":"preconditioners/#Hermitian-linear-systems","page":"Preconditioners","title":"Hermitian linear systems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Methods concerned: SYMMLQ, CG, CG-LANCZOS, CG-LANCZOS-SHIFT, CR, CAR, MINRES, MINRES-QLP and MINARES.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"When A is Hermitian, we can only use centered preconditioning L^-1AL^-Hy = L^-1b with x = L^-Hy. Centered preconditioning is a special case of two-sided preconditioning with P_ell = L = P_r^H that maintains hermicity. However, there is no need to specify L and one may specify P_c = LL^H or its inverse directly.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Preconditioners P_c^-1 P_c\nArguments M with ldiv=false M with ldiv=true","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"warning: Warning\nThe preconditioner M must be hermitian and positive definite.","category":"page"},{"location":"preconditioners/#Linear-least-squares-problems","page":"Preconditioners","title":"Linear least-squares problems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Methods concerned: CGLS, CGLS-LANCZOS-SHIFT, CRLS, LSLQ, LSQR and LSMR.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Formulation Without preconditioning With preconditioning\nleast-squares problem min tfrac12 b - Ax^2_2 min tfrac12 b - Ax^2_E^-1\nNormal equation A^HAx = A^Hb A^HE^-1Ax = A^HE^-1b\nAugmented system beginbmatrix I A A^H 0 endbmatrix beginbmatrix r x endbmatrix = beginbmatrix b 0 endbmatrix beginbmatrix E A A^H 0 endbmatrix beginbmatrix r x endbmatrix = beginbmatrix b 0 endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"LSLQ, LSQR and LSMR also handle regularized least-squares problems.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Formulation Without preconditioning With preconditioning\nleast-squares problem min tfrac12 b - Ax^2_2 + tfrac12 lambda^2 x^2_2 min tfrac12 b - Ax^2_E^-1 + tfrac12 lambda^2 x^2_F\nNormal equation (A^HA + lambda^2 I)x = A^Hb (A^HE^-1A + lambda^2 F)x = A^HE^-1b\nAugmented system beginbmatrix I A A^H -lambda^2 I endbmatrix beginbmatrix r x endbmatrix = beginbmatrix b 0 endbmatrix beginbmatrix E A A^H -lambda^2 F endbmatrix beginbmatrix r x endbmatrix = beginbmatrix b 0 endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Preconditioners E^-1 E F^-1 F\nArguments M with ldiv=false M with ldiv=true N with ldiv=false N with ldiv=true","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"warning: Warning\nThe preconditioners M and N must be hermitian and positive definite.","category":"page"},{"location":"preconditioners/#Linear-least-norm-problems","page":"Preconditioners","title":"Linear least-norm problems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Methods concerned: CGNE, CRMR, LNLQ, CRAIG and CRAIGMR.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Formulation Without preconditioning With preconditioning\nminimum-norm problem min tfrac12 x^2_2textstAx = b min tfrac12 x^2_FtextstAx = b\nNormal equation AA^Hy = btextwithx = A^Hy AF^-1A^Hy = btextwithx = F^-1A^Hy\nAugmented system beginbmatrix -I A^H phantom-A 0 endbmatrix beginbmatrix x y endbmatrix = beginbmatrix 0 b endbmatrix beginbmatrix -F A^H phantom-A 0 endbmatrix beginbmatrix x y endbmatrix = beginbmatrix 0 b endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"LNLQ, CRAIG and CRAIGMR also handle penalized minimum-norm problems.","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Formulation Without preconditioning With preconditioning\nminimum-norm problem min tfrac12 x^2_2 + tfrac12 y^2_2textstAx + lambda^2 y = b min tfrac12 x^2_F + tfrac12 y^2_EtextstAx + lambda^2 Ey = b\nNormal equation (AA^H + lambda^2 I)y = btextwithx = A^Hy (AF^-1A^H + lambda^2 E)y = btextwithx = F^-1A^Hy\nAugmented system beginbmatrix -I A^H phantom-A lambda^2 I endbmatrix beginbmatrix x y endbmatrix = beginbmatrix 0 b endbmatrix beginbmatrix -F A^H phantom-A lambda^2 E endbmatrix beginbmatrix x y endbmatrix = beginbmatrix 0 b endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Preconditioners E^-1 E F^-1 F\nArguments M with ldiv=false M with ldiv=true N with ldiv=false N with ldiv=true","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"warning: Warning\nThe preconditioners M and N must be hermitian and positive definite.","category":"page"},{"location":"preconditioners/#Saddle-point-and-symmetric-quasi-definite-systems","page":"Preconditioners","title":"Saddle-point and symmetric quasi-definite systems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"TriCG and TriMR can take advantage of the structure of Hermitian systems Kz = d with the 2x2 block structure","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":" beginbmatrix tau E phantom-A A^H nu F endbmatrix beginbmatrix x y endbmatrix = beginbmatrix b c endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Preconditioners E^-1 E F^-1 F\nArguments M with ldiv=false M with ldiv=true N with ldiv=false N with ldiv=true","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"warning: Warning\nThe preconditioners M and N must be hermitian and positive definite.","category":"page"},{"location":"preconditioners/#Generalized-saddle-point-and-unsymmetric-partitioned-systems","page":"Preconditioners","title":"Generalized saddle-point and unsymmetric partitioned systems","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"GPMR can take advantage of the structure of general square systems Kz = d with the 2x2 block structure","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":" beginbmatrix lambda M A B mu N endbmatrix beginbmatrix x y endbmatrix = beginbmatrix b c endbmatrix","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"Relations CE = M^-1 EC = M DF = N^-1 FD = N\nArguments C and E with ldiv=false C and E with ldiv=true D and F with ldiv=false D and F with ldiv=true","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"note: Note\nOur implementations of BiLQ, QMR, BiLQR, USYMLQ, USYMQR and TriLQR don't support preconditioning.","category":"page"},{"location":"preconditioners/#Packages-that-provide-preconditioners","page":"Preconditioners","title":"Packages that provide preconditioners","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"KrylovPreconditioners.jl implements block-Jacobi, IC(0) and ILU(0) preconditioners.\nILUZero.jl is a Julia implementation of incomplete LU factorization with zero level of fill-in. \nLimitedLDLFactorizations.jl for limited-memory LDLᵀ factorization of symmetric matrices.\nAlgebraicMultigrid.jl provides two algebraic multigrid (AMG) preconditioners.\nRandomizedPreconditioners.jl uses randomized numerical linear algebra to construct approximate inverses of matrices.\nBasicLU.jl uses a sparse LU factorization to compute a maximum volume basis that can be used as a preconditioner for least-norm and least-squares problems.","category":"page"},{"location":"preconditioners/#Examples","page":"Preconditioners","title":"Examples","text":"","category":"section"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using KrylovPreconditioners, Krylov\n\nP⁻¹ = BlockJacobiPreconditioner(A) # Block-Jacobi preconditioner\nx, stats = gmres(A, b, M=P⁻¹)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using Krylov\nn, m = size(A)\nd = [A[i,i] ≠ 0 ? 1 / abs(A[i,i]) : 1 for i=1:n] # Jacobi preconditioner\nP⁻¹ = diagm(d)\nx, stats = symmlq(A, b, M=P⁻¹)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using Krylov\nn, m = size(A)\nd = [1 / norm(A[:,i]) for i=1:m] # diagonal preconditioner\nP⁻¹ = diagm(d)\nx, stats = minres(A, b, M=P⁻¹)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using KrylovPreconditioners, Krylov\nPℓ = ilu(A)\nx, stats = gmres(A, b, M=Pℓ, ldiv=true) # left preconditioning","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using LimitedLDLFactorizations, Krylov\nP = lldl(A)\nP.D .= abs.(P.D)\nx, stats = cg(A, b, M=P, ldiv=true) # centered preconditioning","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using ILUZero, Krylov\nPᵣ = ilu0(A)\nx, stats = bicgstab(A, b, N=Pᵣ, ldiv=true) # right preconditioning","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using LDLFactorizations, Krylov\n\nM = ldl(E)\nN = ldl(F)\n\n# [E A] [x] = [b]\n# [Aᴴ -F] [y] [c]\nx, y, stats = tricg(A, b, c, M=M, N=N, ldiv=true)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using RandomizedPreconditioners, Krylov\n\n = NystromSketch(A, k, r)\n\nP = NystromPreconditioner(Â, μ)\nx, stats = cg(A + μ*I, b; M=P, ldiv=true)\n\nP⁻¹ = NystromPreconditionerInverse(Â, μ)\nx, stats = cg(A + μ*I, b; M=P⁻¹)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using SuiteSparse, Krylov\nimport LinearAlgebra.ldiv!\n\nM = cholesky(E)\n\n# ldiv! is not implemented for the sparse Cholesky factorization (SuiteSparse.CHOLMOD)\nldiv!(y::Vector{T}, F::SuiteSparse.CHOLMOD.Factor{T}, x::Vector{T}) where T = (y .= F \\ x)\n\n# [E A] [x] = [b]\n# [Aᴴ 0] [y] [c]\nx, y, stats = trimr(A, b, c, M=M, sp=true, ldiv=true)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"using Krylov\n\nC = lu(M)\n\n# [M A] [x] = [b]\n# [B 0] [y] [c]\nx, y, stats = gpmr(A, B, b, c, C=C, gsp=true, ldiv=true)","category":"page"},{"location":"preconditioners/","page":"Preconditioners","title":"Preconditioners","text":"import BasicLU\nusing LinearOperators, Krylov\n\n# Least-squares problem\nm, n = size(A)\nAᴴ = sparse(A')\nbasis, B = BasicLU.maxvolbasis(Aᴴ)\nopA = LinearOperator(A)\nB⁻ᴴ = LinearOperator(Float64, n, n, false, false, (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'T')),\n (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'N')),\n (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'N')))\n\nd, stats = lsmr(opA * B⁻ᴴ, b) # min ‖AB⁻ᴴd - b‖₂\nx = B⁻ᴴ * d # recover the solution of min ‖Ax - b‖₂\n\n# Least-norm problem\nm, n = size(A)\nbasis, B = maxvolbasis(A)\nopA = LinearOperator(A)\nB⁻¹ = LinearOperator(Float64, m, m, false, false, (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'N')),\n (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'T')),\n (y, v) -> (y .= v ; BasicLU.solve!(B, y, 'T')))\n\nx, y, stats = craigmr(B⁻¹ * opA, B⁻¹ * b) # min ‖x‖₂ s.t. B⁻¹Ax = B⁻¹b","category":"page"},{"location":"reference/#Reference","page":"Reference","title":"Reference","text":"","category":"section"},{"location":"reference/#Index","page":"Reference","title":"Index","text":"","category":"section"},{"location":"reference/","page":"Reference","title":"Reference","text":"","category":"page"},{"location":"reference/","page":"Reference","title":"Reference","text":"Krylov.FloatOrComplex\nKrylov.niterations\nKrylov.Aprod\nKrylov.Atprod\nKrylov.kstdout\nKrylov.extract_parameters\nBase.show","category":"page"},{"location":"reference/#Krylov.FloatOrComplex","page":"Reference","title":"Krylov.FloatOrComplex","text":"FloatOrComplex{T}\n\nUnion type of T and Complex{T} where T is an AbstractFloat.\n\n\n\n\n\n","category":"type"},{"location":"reference/#Krylov.niterations","page":"Reference","title":"Krylov.niterations","text":"niterations(solver)\n\nReturn the number of iterations performed by the Krylov method associated to solver.\n\n\n\n\n\n","category":"function"},{"location":"reference/#Krylov.Aprod","page":"Reference","title":"Krylov.Aprod","text":"Aprod(solver)\n\nReturn the number of operator-vector products with A performed by the Krylov method associated to solver.\n\n\n\n\n\n","category":"function"},{"location":"reference/#Krylov.Atprod","page":"Reference","title":"Krylov.Atprod","text":"Atprod(solver)\n\nReturn the number of operator-vector products with A' performed by the Krylov method associated to solver.\n\n\n\n\n\n","category":"function"},{"location":"reference/#Krylov.kstdout","page":"Reference","title":"Krylov.kstdout","text":"Default I/O stream for all Krylov methods.\n\n\n\n\n\n","category":"constant"},{"location":"reference/#Krylov.extract_parameters","page":"Reference","title":"Krylov.extract_parameters","text":"arguments = extract_parameters(ex::Expr)\n\nExtract the arguments of an expression that is keyword parameter tuple. Implementation suggested by Mitchell J. O'Sullivan (@mosullivan93).\n\n\n\n\n\n","category":"function"},{"location":"reference/#Base.show","page":"Reference","title":"Base.show","text":"show(io, solver; show_stats=true)\n\nStatistics of solver are displayed if show_stats is set to true.\n\n\n\n\n\n","category":"function"},{"location":"examples/block_gmres/","page":"BLOCK-GMRES","title":"BLOCK-GMRES","text":"using Krylov, MatrixMarket, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"sherman2\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nn = matrix.nrows[1]\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nB = Matrix{Float64}(I, n, 5)\nB_norm = norm(B)\n\n# Solve Ax = B\nX, stats = block_gmres(A, B)\nshow(stats)\nR = B - A * X\n@printf(\"Relative residual: %8.1e\\n\", norm(R) / B_norm)","category":"page"},{"location":"examples/cg/","page":"CG","title":"CG","text":"using Krylov, MatrixMarket, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"bcsstk09\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nn = matrix.nrows[1]\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = ones(n)\nb_norm = norm(b)\n\n# Solve Ax = b.\n(x, stats) = cg(A, b)\nshow(stats)\nr = b - A * x\n@printf(\"Relative residual: %8.1e\\n\", norm(r) / b_norm)","category":"page"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"# Non-Hermitian square linear systems","category":"page"},{"location":"solvers/unsymmetric/#BiLQ","page":"Non-Hermitian square linear systems","title":"BiLQ","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"bilq\nbilq!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.bilq","page":"Non-Hermitian square linear systems","title":"Krylov.bilq","text":"(x, stats) = bilq(A, b::AbstractVector{FC};\n c::AbstractVector{FC}=b, transfer_to_bicg::Bool=true,\n M=I, N=I, ldiv::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0, timemax::Float64=Inf,\n verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = bilq(A, b, x0::AbstractVector; kwargs...)\n\nBiLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the square linear system Ax = b of size n using BiLQ. BiLQ is based on the Lanczos biorthogonalization process and requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b. When A is Hermitian and b = c, BiLQ is equivalent to SYMMLQ. BiLQ requires support for adjoint(M) and adjoint(N) if preconditioners are provided.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nc: the second initial vector of length n required by the Lanczos biorthogonalization process;\ntransfer_to_bicg: transfer from the BiLQ point to the BiCG point, when it exists. The transfer is based on the residual norm;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\nR. Fletcher, Conjugate gradient methods for indefinite systems, Numerical Analysis, Springer, pp. 73–89, 1976.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.bilq!","page":"Non-Hermitian square linear systems","title":"Krylov.bilq!","text":"solver = bilq!(solver::BilqSolver, A, b; kwargs...)\nsolver = bilq!(solver::BilqSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of bilq.\n\nSee BilqSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#QMR","page":"Non-Hermitian square linear systems","title":"QMR","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"qmr\nqmr!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.qmr","page":"Non-Hermitian square linear systems","title":"Krylov.qmr","text":"(x, stats) = qmr(A, b::AbstractVector{FC};\n c::AbstractVector{FC}=b, M=I, N=I, ldiv::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0, timemax::Float64=Inf, verbose::Int=0,\n history::Bool=false, callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = qmr(A, b, x0::AbstractVector; kwargs...)\n\nQMR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the square linear system Ax = b of size n using QMR.\n\nQMR is based on the Lanczos biorthogonalization process and requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b. When A is Hermitian and b = c, QMR is equivalent to MINRES. QMR requires support for adjoint(M) and adjoint(N) if preconditioners are provided.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nc: the second initial vector of length n required by the Lanczos biorthogonalization process;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nR. W. Freund and N. M. Nachtigal, QMR : a quasi-minimal residual method for non-Hermitian linear systems, Numerische mathematik, Vol. 60(1), pp. 315–339, 1991.\nR. W. Freund and N. M. Nachtigal, An implementation of the QMR method based on coupled two-term recurrences, SIAM Journal on Scientific Computing, Vol. 15(2), pp. 313–337, 1994.\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.qmr!","page":"Non-Hermitian square linear systems","title":"Krylov.qmr!","text":"solver = qmr!(solver::QmrSolver, A, b; kwargs...)\nsolver = qmr!(solver::QmrSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of qmr.\n\nSee QmrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#CGS","page":"Non-Hermitian square linear systems","title":"CGS","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"cgs\ncgs!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.cgs","page":"Non-Hermitian square linear systems","title":"Krylov.cgs","text":"(x, stats) = cgs(A, b::AbstractVector{FC};\n c::AbstractVector{FC}=b, M=I, N=I,\n ldiv::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = cgs(A, b, x0::AbstractVector; kwargs...)\n\nCGS can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the consistent linear system Ax = b of size n using CGS. CGS requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b.\n\nFrom \"Iterative Methods for Sparse Linear Systems (Y. Saad)\" :\n\n«The method is based on a polynomial variant of the conjugate gradients algorithm. Although related to the so-called bi-conjugate gradients (BCG) algorithm, it does not involve adjoint matrix-vector multiplications, and the expected convergence rate is about twice that of the BCG algorithm.\n\nThe Conjugate Gradient Squared algorithm works quite well in many cases. However, one difficulty is that, since the polynomials are squared, rounding errors tend to be more damaging than in the standard BCG algorithm. In particular, very high variations of the residual vectors often cause the residual norms computed to become inaccurate.\n\nTFQMR and BICGSTAB were developed to remedy this difficulty.»\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nc: the second initial vector of length n required by the Lanczos biorthogonalization process;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nP. Sonneveld, CGS, A Fast Lanczos-Type Solver for Nonsymmetric Linear systems, SIAM Journal on Scientific and Statistical Computing, 10(1), pp. 36–52, 1989.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.cgs!","page":"Non-Hermitian square linear systems","title":"Krylov.cgs!","text":"solver = cgs!(solver::CgsSolver, A, b; kwargs...)\nsolver = cgs!(solver::CgsSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of cgs.\n\nSee CgsSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#BiCGSTAB","page":"Non-Hermitian square linear systems","title":"BiCGSTAB","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"bicgstab\nbicgstab!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.bicgstab","page":"Non-Hermitian square linear systems","title":"Krylov.bicgstab","text":"(x, stats) = bicgstab(A, b::AbstractVector{FC};\n c::AbstractVector{FC}=b, M=I, N=I,\n ldiv::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = bicgstab(A, b, x0::AbstractVector; kwargs...)\n\nBICGSTAB can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the square linear system Ax = b of size n using BICGSTAB. BICGSTAB requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b.\n\nThe Biconjugate Gradient Stabilized method is a variant of BiCG, like CGS, but using different updates for the Aᴴ-sequence in order to obtain smoother convergence than CGS.\n\nIf BICGSTAB stagnates, we recommend DQGMRES and BiLQ as alternative methods for unsymmetric square systems.\n\nBICGSTAB stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖b‖ * rtol.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nc: the second initial vector of length n required by the Lanczos biorthogonalization process;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nH. A. van der Vorst, Bi-CGSTAB: A fast and smoothly converging variant of Bi-CG for the solution of nonsymmetric linear systems, SIAM Journal on Scientific and Statistical Computing, 13(2), pp. 631–644, 1992.\nG. L.G. Sleijpen and D. R. Fokkema, BiCGstab(ℓ) for linear equations involving unsymmetric matrices with complex spectrum, Electronic Transactions on Numerical Analysis, 1, pp. 11–32, 1993.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.bicgstab!","page":"Non-Hermitian square linear systems","title":"Krylov.bicgstab!","text":"solver = bicgstab!(solver::BicgstabSolver, A, b; kwargs...)\nsolver = bicgstab!(solver::BicgstabSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of bicgstab.\n\nSee BicgstabSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#DIOM","page":"Non-Hermitian square linear systems","title":"DIOM","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"diom\ndiom!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.diom","page":"Non-Hermitian square linear systems","title":"Krylov.diom","text":"(x, stats) = diom(A, b::AbstractVector{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n reorthogonalization::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = diom(A, b, x0::AbstractVector; kwargs...)\n\nDIOM can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the consistent linear system Ax = b of size n using DIOM.\n\nDIOM only orthogonalizes the new vectors of the Krylov basis against the memory most recent vectors. If CG is well defined on Ax = b and memory = 2, DIOM is theoretically equivalent to CG. If k ≤ memory where k is the number of iterations, DIOM is theoretically equivalent to FOM. Otherwise, DIOM interpolates between CG and FOM and is similar to CG with partial reorthogonalization.\n\nAn advantage of DIOM is that non-Hermitian or Hermitian indefinite or both non-Hermitian and indefinite systems of linear equations can be handled by this single algorithm.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nmemory: the number of most recent vectors of the Krylov basis against which to orthogonalize a new vector;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against the memory most recent vectors;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nY. Saad, Practical use of some krylov subspace methods for solving indefinite and nonsymmetric linear systems, SIAM journal on scientific and statistical computing, 5(1), pp. 203–228, 1984.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.diom!","page":"Non-Hermitian square linear systems","title":"Krylov.diom!","text":"solver = diom!(solver::DiomSolver, A, b; kwargs...)\nsolver = diom!(solver::DiomSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of diom.\n\nNote that the memory keyword argument is the only exception. It's required to create a DiomSolver and can't be changed later.\n\nSee DiomSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#FOM","page":"Non-Hermitian square linear systems","title":"FOM","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"fom\nfom!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.fom","page":"Non-Hermitian square linear systems","title":"Krylov.fom","text":"(x, stats) = fom(A, b::AbstractVector{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n restart::Bool=false, reorthogonalization::Bool=false,\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = fom(A, b, x0::AbstractVector; kwargs...)\n\nFOM can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the linear system Ax = b of size n using FOM.\n\nFOM algorithm is based on the Arnoldi process and a Galerkin condition.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nmemory: if restart = true, the restarted version FOM(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\nrestart: restart the method after memory iterations;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nY. Saad, Krylov subspace methods for solving unsymmetric linear systems, Mathematics of computation, Vol. 37(155), pp. 105–126, 1981.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.fom!","page":"Non-Hermitian square linear systems","title":"Krylov.fom!","text":"solver = fom!(solver::FomSolver, A, b; kwargs...)\nsolver = fom!(solver::FomSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of fom.\n\nNote that the memory keyword argument is the only exception. It's required to create a FomSolver and can't be changed later.\n\nSee FomSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#DQGMRES","page":"Non-Hermitian square linear systems","title":"DQGMRES","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"dqgmres\ndqgmres!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.dqgmres","page":"Non-Hermitian square linear systems","title":"Krylov.dqgmres","text":"(x, stats) = dqgmres(A, b::AbstractVector{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n reorthogonalization::Bool=false, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = dqgmres(A, b, x0::AbstractVector; kwargs...)\n\nDQGMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the consistent linear system Ax = b of size n using DQGMRES.\n\nDQGMRES algorithm is based on the incomplete Arnoldi orthogonalization process and computes a sequence of approximate solutions with the quasi-minimal residual property.\n\nDQGMRES only orthogonalizes the new vectors of the Krylov basis against the memory most recent vectors. If MINRES is well defined on Ax = b and memory = 2, DQGMRES is theoretically equivalent to MINRES. If k ≤ memory where k is the number of iterations, DQGMRES is theoretically equivalent to GMRES. Otherwise, DQGMRES interpolates between MINRES and GMRES and is similar to MINRES with partial reorthogonalization.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nmemory: the number of most recent vectors of the Krylov basis against which to orthogonalize a new vector;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against the memory most recent vectors;\nldiv: define whether the preconditioners use ldiv! or mul!;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nY. Saad and K. Wu, DQGMRES: a quasi minimal residual algorithm based on incomplete orthogonalization, Numerical Linear Algebra with Applications, Vol. 3(4), pp. 329–343, 1996.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.dqgmres!","page":"Non-Hermitian square linear systems","title":"Krylov.dqgmres!","text":"solver = dqgmres!(solver::DqgmresSolver, A, b; kwargs...)\nsolver = dqgmres!(solver::DqgmresSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of dqgmres.\n\nNote that the memory keyword argument is the only exception. It's required to create a DqgmresSolver and can't be changed later.\n\nSee DqgmresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#GMRES","page":"Non-Hermitian square linear systems","title":"GMRES","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"gmres\ngmres!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.gmres","page":"Non-Hermitian square linear systems","title":"Krylov.gmres","text":"(x, stats) = gmres(A, b::AbstractVector{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n restart::Bool=false, reorthogonalization::Bool=false,\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = gmres(A, b, x0::AbstractVector; kwargs...)\n\nGMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the linear system Ax = b of size n using GMRES.\n\nGMRES algorithm is based on the Arnoldi process and computes a sequence of approximate solutions with the minimum residual.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nmemory: if restart = true, the restarted version GMRES(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\nrestart: restart the method after memory iterations;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nY. Saad and M. H. Schultz, GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems, SIAM Journal on Scientific and Statistical Computing, Vol. 7(3), pp. 856–869, 1986.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.gmres!","page":"Non-Hermitian square linear systems","title":"Krylov.gmres!","text":"solver = gmres!(solver::GmresSolver, A, b; kwargs...)\nsolver = gmres!(solver::GmresSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of gmres.\n\nNote that the memory keyword argument is the only exception. It's required to create a GmresSolver and can't be changed later.\n\nSee GmresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#FGMRES","page":"Non-Hermitian square linear systems","title":"FGMRES","text":"","category":"section"},{"location":"solvers/unsymmetric/","page":"Non-Hermitian square linear systems","title":"Non-Hermitian square linear systems","text":"fgmres\nfgmres!","category":"page"},{"location":"solvers/unsymmetric/#Krylov.fgmres","page":"Non-Hermitian square linear systems","title":"Krylov.fgmres","text":"(x, stats) = fgmres(A, b::AbstractVector{FC};\n memory::Int=20, M=I, N=I, ldiv::Bool=false,\n restart::Bool=false, reorthogonalization::Bool=false,\n atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = fgmres(A, b, x0::AbstractVector; kwargs...)\n\nFGMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the linear system Ax = b of size n using FGMRES.\n\nFGMRES computes a sequence of approximate solutions with minimum residual. FGMRES is a variant of GMRES that allows changes in the right preconditioner at each iteration.\n\nThis implementation allows a left preconditioner M and a flexible right preconditioner N. A situation in which the preconditioner is \"not constant\" is when a relaxation-type method, a Chebyshev iteration or another Krylov subspace method is used as a preconditioner. Compared to GMRES, there is no additional cost incurred in the arithmetic but the memory requirement almost doubles. Thus, GMRES is recommended if the right preconditioner N is constant.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nmemory: if restart = true, the restarted version FGMRES(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;\nM: linear operator that models a nonsingular matrix of size n used for left preconditioning;\nN: linear operator that models a nonsingular matrix of size n used for right preconditioning;\nldiv: define whether the preconditioners use ldiv! or mul!;\nrestart: restart the method after memory iterations;\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nY. Saad, A Flexible Inner-Outer Preconditioned GMRES Algorithm, SIAM Journal on Scientific Computing, Vol. 14(2), pp. 461–469, 1993.\n\n\n\n\n\n","category":"function"},{"location":"solvers/unsymmetric/#Krylov.fgmres!","page":"Non-Hermitian square linear systems","title":"Krylov.fgmres!","text":"solver = fgmres!(solver::FgmresSolver, A, b; kwargs...)\nsolver = fgmres!(solver::FgmresSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of fgmres.\n\nNote that the memory keyword argument is the only exception. It's required to create a FgmresSolver and can't be changed later.\n\nSee FgmresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"examples/crmr/","page":"CRMR","title":"CRMR","text":"using Krylov, HarwellRutherfordBoeing, SuiteSparseMatrixCollection\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"gemat1\")\npath = fetch_ssmc(matrix, format=\"RB\")\n\nA = RutherfordBoeingData(joinpath(path[1], \"$(matrix.name[1]).rb\")).data\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\nx_exact = A' * ones(m)\nx_exact_norm = norm(x_exact)\nx_exact /= x_exact_norm\nb = A * x_exact\n(x, stats) = crmr(A, b)\nshow(stats)\nresid = norm(A * x - b) / norm(b)\n@printf(\"CRMR: Relative residual: %7.1e\\n\", resid)\n@printf(\"CRMR: ‖x - x*‖₂: %7.1e\\n\", norm(x - x_exact))","category":"page"},{"location":"solvers/sid/","page":"Hermitian indefinite linear systems","title":"Hermitian indefinite linear systems","text":"# Hermitian indefinite linear systems","category":"page"},{"location":"solvers/sid/#SYMMLQ","page":"Hermitian indefinite linear systems","title":"SYMMLQ","text":"","category":"section"},{"location":"solvers/sid/","page":"Hermitian indefinite linear systems","title":"Hermitian indefinite linear systems","text":"symmlq\nsymmlq!","category":"page"},{"location":"solvers/sid/#Krylov.symmlq","page":"Hermitian indefinite linear systems","title":"Krylov.symmlq","text":"(x, stats) = symmlq(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, window::Int=5,\n transfer_to_cg::Bool=true, λ::T=zero(T),\n λest::T=zero(T), etol::T=√eps(T),\n conlim::T=1/√eps(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = symmlq(A, b, x0::AbstractVector; kwargs...)\n\nSYMMLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above\n\nSolve the shifted linear system\n\n(A + λI) x = b\n\nof size n using the SYMMLQ method, where λ is a shift parameter, and A is Hermitian.\n\nSYMMLQ produces monotonic errors ‖x* - x‖₂.\n\nInput arguments\n\nA: a linear operator that models a Hermitian matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nwindow: number of iterations used to accumulate a lower bound on the error;\ntransfer_to_cg: transfer from the SYMMLQ point to the CG point, when it exists. The transfer is based on the residual norm;\nλ: regularization parameter;\nλest: positive strict lower bound on the smallest eigenvalue λₘᵢₙ when solving a positive-definite system, such as λest = (1-10⁻⁷)λₘᵢₙ;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\netol: stopping tolerance based on the lower bound on the error;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SymmlqStats structure.\n\nReference\n\nC. C. Paige and M. A. Saunders, Solution of Sparse Indefinite Systems of Linear Equations, SIAM Journal on Numerical Analysis, 12(4), pp. 617–629, 1975.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#Krylov.symmlq!","page":"Hermitian indefinite linear systems","title":"Krylov.symmlq!","text":"solver = symmlq!(solver::SymmlqSolver, A, b; kwargs...)\nsolver = symmlq!(solver::SymmlqSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of symmlq.\n\nSee SymmlqSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#MINRES","page":"Hermitian indefinite linear systems","title":"MINRES","text":"","category":"section"},{"location":"solvers/sid/","page":"Hermitian indefinite linear systems","title":"Hermitian indefinite linear systems","text":"minres\nminres!","category":"page"},{"location":"solvers/sid/#Krylov.minres","page":"Hermitian indefinite linear systems","title":"Krylov.minres","text":"(x, stats) = minres(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, window::Int=5,\n λ::T=zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), etol::T=√eps(T),\n conlim::T=1/√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = minres(A, b, x0::AbstractVector; kwargs...)\n\nMINRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nSolve the shifted linear least-squares problem\n\nminimize ‖b - (A + λI)x‖₂²\n\nor the shifted linear system\n\n(A + λI) x = b\n\nof size n using the MINRES method, where λ ≥ 0 is a shift parameter, where A is Hermitian.\n\nMINRES is formally equivalent to applying CR to Ax=b when A is positive definite, but is typically more stable and also applies to the case where A is indefinite.\n\nMINRES produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂.\n\nInput arguments\n\nA: a linear operator that models a Hermitian matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nwindow: number of iterations used to accumulate a lower bound on the error;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\netol: stopping tolerance based on the lower bound on the error;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nC. C. Paige and M. A. Saunders, Solution of Sparse Indefinite Systems of Linear Equations, SIAM Journal on Numerical Analysis, 12(4), pp. 617–629, 1975.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#Krylov.minres!","page":"Hermitian indefinite linear systems","title":"Krylov.minres!","text":"solver = minres!(solver::MinresSolver, A, b; kwargs...)\nsolver = minres!(solver::MinresSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of minres.\n\nSee MinresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#MINRES-QLP","page":"Hermitian indefinite linear systems","title":"MINRES-QLP","text":"","category":"section"},{"location":"solvers/sid/","page":"Hermitian indefinite linear systems","title":"Hermitian indefinite linear systems","text":"minres_qlp\nminres_qlp!","category":"page"},{"location":"solvers/sid/#Krylov.minres_qlp","page":"Hermitian indefinite linear systems","title":"Krylov.minres_qlp","text":"(x, stats) = minres_qlp(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false, Artol::T=√eps(T),\n λ::T=zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = minres_qlp(A, b, x0::AbstractVector; kwargs...)\n\nMINRES-QLP can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nMINRES-QLP is the only method based on the Lanczos process that returns the minimum-norm solution on singular inconsistent systems (A + λI)x = b of size n, where λ is a shift parameter. It is significantly more complex but can be more reliable than MINRES when A is ill-conditioned.\n\nM also indicates the weighted norm in which residuals are measured.\n\nInput arguments\n\nA: a linear operator that models a Hermitian matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nArtol: relative stopping tolerance based on the Aᴴ-residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nS.-C. T. Choi, Iterative methods for singular linear equations and least-squares problems, Ph.D. thesis, ICME, Stanford University, 2006.\nS.-C. T. Choi, C. C. Paige and M. A. Saunders, MINRES-QLP: A Krylov subspace method for indefinite or singular symmetric systems, SIAM Journal on Scientific Computing, Vol. 33(4), pp. 1810–1836, 2011.\nS.-C. T. Choi and M. A. Saunders, Algorithm 937: MINRES-QLP for symmetric and Hermitian linear equations and least-squares problems, ACM Transactions on Mathematical Software, 40(2), pp. 1–12, 2014.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#Krylov.minres_qlp!","page":"Hermitian indefinite linear systems","title":"Krylov.minres_qlp!","text":"solver = minres_qlp!(solver::MinresQlpSolver, A, b; kwargs...)\nsolver = minres_qlp!(solver::MinresQlpSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of minres_qlp.\n\nSee MinresQlpSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#MINARES","page":"Hermitian indefinite linear systems","title":"MINARES","text":"","category":"section"},{"location":"solvers/sid/","page":"Hermitian indefinite linear systems","title":"Hermitian indefinite linear systems","text":"minares\nminares!","category":"page"},{"location":"solvers/sid/#Krylov.minares","page":"Hermitian indefinite linear systems","title":"Krylov.minares","text":"(x, stats) = minares(A, b::AbstractVector{FC};\n M=I, ldiv::Bool=false,\n λ::T = zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), Artol::T = √eps(T),\n itmax::Int=0, timemax::Float64=Inf,\n verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = minares(A, b, x0::AbstractVector; kwargs...)\n\nMINARES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nMINARES solves the Hermitian linear system Ax = b of size n. MINARES minimizes ‖Arₖ‖₂ when M = Iₙ and ‖AMrₖ‖M otherwise. The estimates computed every iteration are ‖Mrₖ‖₂ and ‖AMrₖ‖M.\n\nInput arguments\n\nA: a linear operator that models a Hermitian positive definite matrix of dimension n;\nb: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nArtol: relative stopping tolerance based on the Aᴴ-residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReference\n\nA. Montoison, D. Orban and M. A. Saunders, MinAres: An Iterative Solver for Symmetric Linear Systems, Cahier du GERAD G-2023-40, GERAD, Montréal, 2023.\n\n\n\n\n\n","category":"function"},{"location":"solvers/sid/#Krylov.minares!","page":"Hermitian indefinite linear systems","title":"Krylov.minares!","text":"solver = minares!(solver::MinaresSolver, A, b; kwargs...)\nsolver = minares!(solver::MinaresSolver, A, b, x0; kwargs...)\n\nwhere kwargs are keyword arguments of minares.\n\nSee MinaresSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"# Least-norm problems","category":"page"},{"location":"solvers/ln/#CGNE","page":"Least-norm problems","title":"CGNE","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"cgne\ncgne!","category":"page"},{"location":"solvers/ln/#Krylov.cgne","page":"Least-norm problems","title":"Krylov.cgne","text":"(x, stats) = cgne(A, b::AbstractVector{FC};\n N=I, ldiv::Bool=false,\n λ::T=zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the consistent linear system\n\nAx + √λs = b\n\nof size m × n using the Conjugate Gradient (CG) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CG to the normal equations of the second kind\n\n(AAᴴ + λI) y = b\n\nbut is more stable. When λ = 0, this method solves the minimum-norm problem\n\nmin ‖x‖₂ s.t. Ax = b.\n\nWhen λ > 0, it solves the problem\n\nmin ‖(x,s)‖₂ s.t. Ax + √λs = b.\n\nCGNE produces monotonic errors ‖x-x*‖₂ but not residuals ‖r‖₂. It is formally equivalent to CRAIG, though can be slightly less accurate, but simpler to implement. Only the x-part of the solution is returned.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nN: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nJ. E. Craig, The N-step iteration procedures, Journal of Mathematics and Physics, 34(1), pp. 64–73, 1955.\nJ. E. Craig, Iterations Procedures for Simultaneous Equations, Ph.D. Thesis, Department of Electrical Engineering, MIT, 1954.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.cgne!","page":"Least-norm problems","title":"Krylov.cgne!","text":"solver = cgne!(solver::CgneSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of cgne.\n\nSee CgneSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#CRMR","page":"Least-norm problems","title":"CRMR","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"crmr\ncrmr!","category":"page"},{"location":"solvers/ln/#Krylov.crmr","page":"Least-norm problems","title":"Krylov.crmr","text":"(x, stats) = crmr(A, b::AbstractVector{FC};\n N=I, ldiv::Bool=false,\n λ::T=zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the consistent linear system\n\nAx + √λs = b\n\nof size m × n using the Conjugate Residual (CR) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CR to the normal equations of the second kind\n\n(AAᴴ + λI) y = b\n\nbut is more stable. When λ = 0, this method solves the minimum-norm problem\n\nmin ‖x‖₂ s.t. x ∈ argmin ‖Ax - b‖₂.\n\nWhen λ > 0, this method solves the problem\n\nmin ‖(x,s)‖₂ s.t. Ax + √λs = b.\n\nCRMR produces monotonic residuals ‖r‖₂. It is formally equivalent to CRAIG-MR, though can be slightly less accurate, but simpler to implement. Only the x-part of the solution is returned.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nN: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;\nldiv: define whether the preconditioner uses ldiv! or mul!;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nD. Orban and M. Arioli, Iterative Solution of Symmetric Quasi-Definite Linear Systems, Volume 3 of Spotlights. SIAM, Philadelphia, PA, 2017.\nD. Orban, The Projected Golub-Kahan Process for Constrained Linear Least-Squares Problems. Cahier du GERAD G-2014-15, 2014.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.crmr!","page":"Least-norm problems","title":"Krylov.crmr!","text":"solver = crmr!(solver::CrmrSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of crmr.\n\nSee CrmrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#LNLQ","page":"Least-norm problems","title":"LNLQ","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"lnlq\nlnlq!","category":"page"},{"location":"solvers/ln/#Krylov.lnlq","page":"Least-norm problems","title":"Krylov.lnlq","text":"(x, y, stats) = lnlq(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n transfer_to_craig::Bool=true,\n sqd::Bool=false, λ::T=zero(T),\n σ::T=zero(T), utolx::T=√eps(T),\n utoly::T=√eps(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nFind the least-norm solution of the consistent linear system\n\nAx + λ²y = b\n\nof size m × n using the LNLQ method, where λ ≥ 0 is a regularization parameter.\n\nFor a system in the form Ax = b, LNLQ method is equivalent to applying SYMMLQ to AAᴴy = b and recovering x = Aᴴy but is more stable. Note that y are the Lagrange multipliers of the least-norm problem\n\nminimize ‖x‖ s.t. Ax = b.\n\nIf λ > 0, LNLQ solves the symmetric and quasi-definite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A λ²E ] [ y ] = [ b ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nmin ‖x‖²_F + λ²‖y‖²_E s.t. Ax + λ²Ey = b.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LNLQ is then equivalent to applying SYMMLQ to (AF⁻¹Aᴴ + λ²E)y = b with Fx = Aᴴy.\n\nIf λ = 0, LNLQ solves the symmetric and indefinite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A 0 ] [ y ] = [ b ].\n\nThe system above represents the optimality conditions of\n\nminimize ‖x‖²_F s.t. Ax = b.\n\nIn this case, M can still be specified and indicates the weighted norm in which residuals are measured.\n\nIn this implementation, both the x and y-parts of the solution are returned.\n\nutolx and utoly are tolerances on the upper bound of the distance to the solution ‖x-x‖ and ‖y-y‖, respectively. The bound is valid if λ>0 or σ>0 where σ should be strictly smaller than the smallest positive singular value. For instance σ:=(1-1e-7)σₘᵢₙ .\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\ntransfer_to_craig: transfer from the LNLQ point to the CRAIG point, when it exists. The transfer is based on the residual norm;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\nσ: strict lower bound on the smallest positive singular value σₘᵢₙ such as σ = (1-10⁻⁷)σₘᵢₙ;\nutolx: tolerance on the upper bound on the distance to the solution ‖x-x*‖;\nutoly: tolerance on the upper bound on the distance to the solution ‖y-y*‖;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\ny: a dense vector of length m;\nstats: statistics collected on the run in a LNLQStats structure.\n\nReference\n\nR. Estrin, D. Orban, M.A. Saunders, LNLQ: An Iterative Method for Least-Norm Problems with an Error Minimization Property, SIAM Journal on Matrix Analysis and Applications, 40(3), pp. 1102–1124, 2019.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.lnlq!","page":"Least-norm problems","title":"Krylov.lnlq!","text":"solver = lnlq!(solver::LnlqSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of lnlq.\n\nSee LnlqSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#CRAIG","page":"Least-norm problems","title":"CRAIG","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"craig\ncraig!","category":"page"},{"location":"solvers/ln/#Krylov.craig","page":"Least-norm problems","title":"Krylov.craig","text":"(x, y, stats) = craig(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n transfer_to_lsqr::Bool=false, sqd::Bool=false,\n λ::T=zero(T), btol::T=√eps(T),\n conlim::T=1/√eps(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nFind the least-norm solution of the consistent linear system\n\nAx + λ²y = b\n\nof size m × n using the Golub-Kahan implementation of Craig's method, where λ ≥ 0 is a regularization parameter. This method is equivalent to CGNE but is more stable.\n\nFor a system in the form Ax = b, Craig's method is equivalent to applying CG to AAᴴy = b and recovering x = Aᴴy. Note that y are the Lagrange multipliers of the least-norm problem\n\nminimize ‖x‖ s.t. Ax = b.\n\nIf λ > 0, CRAIG solves the symmetric and quasi-definite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A λ²E ] [ y ] = [ b ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nmin ‖x‖²_F + λ²‖y‖²_E s.t. Ax + λ²Ey = b.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. CRAIG is then equivalent to applying CG to (AF⁻¹Aᴴ + λ²E)y = b with Fx = Aᴴy.\n\nIf λ = 0, CRAIG solves the symmetric and indefinite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A 0 ] [ y ] = [ b ].\n\nThe system above represents the optimality conditions of\n\nminimize ‖x‖²_F s.t. Ax = b.\n\nIn this case, M can still be specified and indicates the weighted norm in which residuals are measured.\n\nIn this implementation, both the x and y-parts of the solution are returned.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\ntransfer_to_lsqr: transfer from the LSLQ point to the LSQR point, when it exists. The transfer is based on the residual norm;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\nbtol: stopping tolerance used to detect zero-residual problems;\nconlim: limit on the estimated condition number of A beyond which the solution will be abandoned;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\ny: a dense vector of length m;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nJ. E. Craig, The N-step iteration procedures, Journal of Mathematics and Physics, 34(1-4), pp. 64–73, 1955.\nC. C. Paige and M. A. Saunders, LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares, ACM Transactions on Mathematical Software, 8(1), pp. 43–71, 1982.\nM. A. Saunders, Solutions of Sparse Rectangular Systems Using LSQR and CRAIG, BIT Numerical Mathematics, 35(4), pp. 588–604, 1995.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.craig!","page":"Least-norm problems","title":"Krylov.craig!","text":"solver = craig!(solver::CraigSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of craig.\n\nSee CraigSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#CRAIGMR","page":"Least-norm problems","title":"CRAIGMR","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"craigmr\ncraigmr!","category":"page"},{"location":"solvers/ln/#Krylov.craigmr","page":"Least-norm problems","title":"Krylov.craigmr","text":"(x, y, stats) = craigmr(A, b::AbstractVector{FC};\n M=I, N=I, ldiv::Bool=false,\n sqd::Bool=false, λ::T=zero(T), atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\nSolve the consistent linear system\n\nAx + λ²y = b\n\nof size m × n using the CRAIGMR method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying the Conjugate Residuals method to the normal equations of the second kind\n\n(AAᴴ + λ²I) y = b\n\nbut is more stable. When λ = 0, this method solves the minimum-norm problem\n\nmin ‖x‖ s.t. x ∈ argmin ‖Ax - b‖.\n\nIf λ > 0, CRAIGMR solves the symmetric and quasi-definite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A λ²E ] [ y ] = [ b ],\n\nwhere E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.\n\nThe system above represents the optimality conditions of\n\nmin ‖x‖²_F + λ²‖y‖²_E s.t. Ax + λ²Ey = b.\n\nFor a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. CRAIGMR is then equivalent to applying MINRES to (AF⁻¹Aᴴ + λ²E)y = b with Fx = Aᴴy.\n\nIf λ = 0, CRAIGMR solves the symmetric and indefinite system\n\n[ -F Aᴴ ] [ x ] [ 0 ]\n[ A 0 ] [ y ] = [ b ].\n\nThe system above represents the optimality conditions of\n\nmin ‖x‖²_F s.t. Ax = b.\n\nIn this case, M can still be specified and indicates the weighted norm in which residuals are measured.\n\nCRAIGMR produces monotonic residuals ‖r‖₂. It is formally equivalent to CRMR, though can be slightly more accurate, and intricate to implement. Both the x- and y-parts of the solution are returned.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m.\n\nKeyword arguments\n\nM: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;\nN: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;\nldiv: define whether the preconditioners use ldiv! or mul!;\nsqd: if true, set λ=1 for Hermitian quasi-definite systems;\nλ: regularization parameter;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\ny: a dense vector of length m;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nD. Orban and M. Arioli. Iterative Solution of Symmetric Quasi-Definite Linear Systems, Volume 3 of Spotlights. SIAM, Philadelphia, PA, 2017.\nD. Orban, The Projected Golub-Kahan Process for Constrained, Linear Least-Squares Problems. Cahier du GERAD G-2014-15, 2014.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.craigmr!","page":"Least-norm problems","title":"Krylov.craigmr!","text":"solver = craigmr!(solver::CraigmrSolver, A, b; kwargs...)\n\nwhere kwargs are keyword arguments of craigmr.\n\nSee CraigmrSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#USYMLQ","page":"Least-norm problems","title":"USYMLQ","text":"","category":"section"},{"location":"solvers/ln/","page":"Least-norm problems","title":"Least-norm problems","text":"usymlq\nusymlq!","category":"page"},{"location":"solvers/ln/#Krylov.usymlq","page":"Least-norm problems","title":"Krylov.usymlq","text":"(x, stats) = usymlq(A, b::AbstractVector{FC}, c::AbstractVector{FC};\n transfer_to_usymcg::Bool=true, atol::T=√eps(T),\n rtol::T=√eps(T), itmax::Int=0,\n timemax::Float64=Inf, verbose::Int=0, history::Bool=false,\n callback=solver->false, iostream::IO=kstdout)\n\nT is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.\n\n(x, stats) = usymlq(A, b, c, x0::AbstractVector; kwargs...)\n\nUSYMLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.\n\nUSYMLQ determines the least-norm solution of the consistent linear system Ax = b of size m × n.\n\nUSYMLQ is based on the orthogonal tridiagonalization process and requires two initial nonzero vectors b and c. The vector c is only used to initialize the process and a default value can be b or Aᴴb depending on the shape of A. The error norm ‖x - x*‖ monotonously decreases in USYMLQ. When A is Hermitian and b = c, USYMLQ is equivalent to SYMMLQ. USYMLQ is considered as a generalization of SYMMLQ.\n\nIt can also be applied to under-determined and over-determined problems. In all cases, problems must be consistent.\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nc: a vector of length n.\n\nOptional argument\n\nx0: a vector of length n that represents an initial guess of the solution x.\n\nKeyword arguments\n\ntransfer_to_usymcg: transfer from the USYMLQ point to the USYMCG point, when it exists. The transfer is based on the residual norm;\natol: absolute stopping tolerance based on the residual norm;\nrtol: relative stopping tolerance based on the residual norm;\nitmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;\ntimemax: the time limit in seconds;\nverbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;\nhistory: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;\ncallback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;\niostream: stream to which output is logged.\n\nOutput arguments\n\nx: a dense vector of length n;\nstats: statistics collected on the run in a SimpleStats structure.\n\nReferences\n\nM. A. Saunders, H. D. Simon, and E. L. Yip, Two Conjugate-Gradient-Type Methods for Unsymmetric Linear Equations, SIAM Journal on Numerical Analysis, 25(4), pp. 927–940, 1988.\nA. Buttari, D. Orban, D. Ruiz and D. Titley-Peloquin, A tridiagonalization method for symmetric saddle-point and quasi-definite systems, SIAM Journal on Scientific Computing, 41(5), pp. 409–432, 2019.\nA. Montoison and D. Orban, BiLQ: An Iterative Method for Nonsymmetric Linear Systems with a Quasi-Minimum Error Property, SIAM Journal on Matrix Analysis and Applications, 41(3), pp. 1145–1166, 2020.\n\n\n\n\n\n","category":"function"},{"location":"solvers/ln/#Krylov.usymlq!","page":"Least-norm problems","title":"Krylov.usymlq!","text":"solver = usymlq!(solver::UsymlqSolver, A, b, c; kwargs...)\nsolver = usymlq!(solver::UsymlqSolver, A, b, c, x0; kwargs...)\n\nwhere kwargs are keyword arguments of usymlq.\n\nSee UsymlqSolver for more details about the solver.\n\n\n\n\n\n","category":"function"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"","category":"page"},{"location":"processes/#krylov-processes","page":"Krylov processes","title":"Krylov processes","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Krylov processes are the foundation of Krylov methods, they generate bases of Krylov subspaces. Depending on the Krylov subspaces generated, Krylov processes are more or less specialized for a subset of linear problems. The following table summarizes the most relevant processes for each linear problem.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Linear problems Processes\nHermitian linear systems Hermitian Lanczos\nSquare Non-Hermitian linear systems Non-Hermitian Lanczos – Arnoldi\nLeast-squares problems Golub-Kahan – Saunders-Simon-Yip\nLeast-norm problems Golub-Kahan – Saunders-Simon-Yip\nSaddle-point and Hermitian quasi-definite systems Golub-Kahan – Saunders-Simon-Yip\nGeneralized saddle-point and non-Hermitian partitioned systems Montoison-Orban","category":"page"},{"location":"processes/#Notation","page":"Krylov processes","title":"Notation","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"For a matrix A, A^H denotes the conjugate transpose of A. It coincides with A^T, the transpose of A, for real matrices. Define V_k = beginbmatrix v_1 ldots v_k endbmatrix enspace and enspace U_k = beginbmatrix u_1 ldots u_k endbmatrix.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"For a matrix C in mathbbC^n times n and a vector t in mathbbC^n, the k-th Krylov subspace generated by C and t is","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"mathcalK_k(C t) =\nleftsum_i=0^k-1 omega_i C^i t middle vert omega_i in mathbbC0 le i le k-1 right","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"For matrices C in mathbbC^n times n enspace and enspace T in mathbbC^n times p, the k-th block Krylov subspace generated by C and T is","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"mathcalK_k^square(C T) =\nleftsum_i=0^k-1 C^i T Omega_i middle vert Omega_i in mathbbC^p times p0 le i le k-1 right","category":"page"},{"location":"processes/#hermitian-lanczos","page":"Krylov processes","title":"Hermitian Lanczos","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: hermitian_lanczos)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the Hermitian Lanczos process, the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A V_k = V_k T_k + beta_k+1k v_k+1 e_k^T = V_k+1 T_k+1k \n V_k^H V_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where V_k is an orthonormal basis of the Krylov subspace mathcalK_k(Ab),","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"T_k =\nbeginbmatrix\n alpha_1 barbeta_2 \n beta_2 alpha_2 ddots \n ddots ddots barbeta_k \n beta_k alpha_k\nendbmatrix\n qquad\nT_k+1k =\nbeginbmatrix\n T_k \n beta_k+1 e_k^T\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Note that depending on how we normalize the vectors that compose V_k, T_k+1k can be a real tridiagonal matrix even if A is a complex matrix.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function hermitian_lanczos returns V_k+1, beta_1 and T_k+1k.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: SYMMLQ, CG, CR, CAR, MINRES, MINRES-QLP, MINARES, CGLS, CGLS-LANCZOS-SHIFT, CRLS, CGNE, CRMR, CG-LANCZOS and CG-LANCZOS-SHIFT.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"hermitian_lanczos(::Any, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.hermitian_lanczos-Union{Tuple{FC}, Tuple{Any, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.hermitian_lanczos","text":"V, β, T = hermitian_lanczos(A, b, k)\n\nInput arguments\n\nA: a linear operator that models an Hermitian matrix of dimension n;\nb: a vector of length n;\nk: the number of iterations of the Hermitian Lanczos process.\n\nOutput arguments\n\nV: a dense n × (k+1) matrix;\nβ: a coefficient such that βv₁ = b;\nT: a sparse (k+1) × k tridiagonal matrix.\n\nReference\n\nC. Lanczos, An Iteration Method for the Solution of the Eigenvalue Problem of Linear Differential and Integral Operators, Journal of Research of the National Bureau of Standards, 45(4), pp. 225–280, 1950.\n\n\n\n\n\n","category":"method"},{"location":"processes/#nonhermitian-lanczos","page":"Krylov processes","title":"Non-Hermitian Lanczos","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: nonhermitian_lanczos)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the non-Hermitian Lanczos process (also named the Lanczos biorthogonalization process), the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A V_k = V_k T_k + beta_k+1 v_k+1 e_k^T = V_k+1 T_k+1k \n A^H U_k = U_k T_k^H + bargamma_k+1 u_k+1 e_k^T = U_k+1 T_kk+1^H \n V_k^H U_k = U_k^H V_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where V_k and U_k are bases of the Krylov subspaces mathcalK_k (Ab) and mathcalK_k (A^Hc), respectively,","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"T_k = \nbeginbmatrix\n alpha_1 gamma_2 \n beta_2 alpha_2 ddots \n ddots ddots gamma_k \n beta_k alpha_k\nendbmatrix\n qquad\nT_k+1k =\nbeginbmatrix\n T_k \n beta_k+1 e_k^T\nendbmatrix\n qquad\nT_kk+1 =\nbeginbmatrix\n T_k gamma_k+1 e_k\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function nonhermitian_lanczos returns V_k+1, beta_1, T_k+1k, U_k+1, bargamma_1 and T_kk+1^H.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: BiLQ, QMR, BiLQR, CGS and BICGSTAB.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"note: Note\nThe scaling factors used in our implementation are beta_k = u_k^H v_k^tfrac12 and gamma_k = (u_k^H v_k) beta_k. With these scaling factors, the non-Hermitian Lanczos process coincides with the Hermitian Lanczos process when A = A^H and b = c.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"nonhermitian_lanczos(::Any, ::AbstractVector{FC}, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.nonhermitian_lanczos-Union{Tuple{FC}, Tuple{Any, AbstractVector{FC}, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.nonhermitian_lanczos","text":"V, β, T, U, γᴴ, Tᴴ = nonhermitian_lanczos(A, b, c, k)\n\nInput arguments\n\nA: a linear operator that models a square matrix of dimension n;\nb: a vector of length n;\nc: a vector of length n;\nk: the number of iterations of the non-Hermitian Lanczos process.\n\nOutput arguments\n\nV: a dense n × (k+1) matrix;\nβ: a coefficient such that βv₁ = b;\nT: a sparse (k+1) × k tridiagonal matrix;\nU: a dense n × (k+1) matrix;\nγᴴ: a coefficient such that γᴴu₁ = c;\nTᴴ: a sparse (k+1) × k tridiagonal matrix.\n\nReference\n\nC. Lanczos, An Iteration Method for the Solution of the Eigenvalue Problem of Linear Differential and Integral Operators, Journal of Research of the National Bureau of Standards, 45(4), pp. 225–280, 1950.\n\n\n\n\n\n","category":"method"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The non-Hermitian Lanczos process can be also implemented without A^H (transpose-free variant). To derive it, we can observe that beta_k+1 v_k+1 = P_k(A) b and bargamma_k+1 u_k+1 = Q_k(A^H) c where P_k and Q_k are polynomials of degree k. The polynomials are defined from the recursions:","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n P_0(A) = I_n \n P_1(A) = left(dfracA - alpha_1 I_nbeta_1right) P_0(A) \n P_k(A) = left(dfracA - alpha_k I_nbeta_kright) P_k-1(A) - dfracgamma_kbeta_k-1 P_k-2(A) quad k ge 2 \n \n Q_0(A^H) = I_n \n Q_1(A^H) = left(dfracA^H - baralpha_1 I_nbargamma_1right) Q_0(A^H) \n Q_k(A^H) = left(dfracA^H - baralpha_k I_nbargamma_kright) Q_k-1(A^H) - dfracbarbeta_kbargamma_k-1 Q_k-2(A^H) quad k ge 2\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Because alpha_k = u_k^H A v_k and (bargamma_k+1 u_k+1)^H (beta_k+1 v_k+1) = gamma_k+1 beta_k+1, we can determine the coefficients of T_k+1k and T_kk+1^H as follows:","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n alpha_k = dfrac1gamma_k beta_k langleQ_k-1(A^H) c A P_k-1(A) brangle \n = dfrac1gamma_k beta_k langlec barQ_k-1(A) A P_k-1(A) brangle \n \n beta_k+1 gamma_k+1 = langleQ_k(A^H) c P_k(A) brangle \n = langlec barQ_k(A) P_k(A) brangle\nendalign*","category":"page"},{"location":"processes/#arnoldi","page":"Krylov processes","title":"Arnoldi","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: arnoldi)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the Arnoldi process, the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A V_k = V_k H_k + h_k+1k v_k+1 e_k^T = V_k+1 H_k+1k \n V_k^H V_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where V_k is an orthonormal basis of the Krylov subspace mathcalK_k (Ab),","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"H_k =\nbeginbmatrix\n h_11 h_12 ldots h_1k \n h_21 ddots ddots vdots \n ddots ddots h_k-1k \n h_kk-1 h_kk\nendbmatrix\n qquad\nH_k+1k =\nbeginbmatrix\n H_k \n h_k+1k e_k^T\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function arnoldi returns V_k+1, beta and H_k+1k.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: DIOM, FOM, DQGMRES, GMRES and FGMRES.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"note: Note\nThe Arnoldi process coincides with the Hermitian Lanczos process when A is Hermitian.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"arnoldi(::Any, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.arnoldi-Union{Tuple{FC}, Tuple{Any, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.arnoldi","text":"V, β, H = arnoldi(A, b, k; reorthogonalization=false)\n\nInput arguments\n\nA: a linear operator that models a square matrix of dimension n;\nb: a vector of length n;\nk: the number of iterations of the Arnoldi process.\n\nKeyword argument\n\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors.\n\nOutput arguments\n\nV: a dense n × (k+1) matrix;\nβ: a coefficient such that βv₁ = b;\nH: a dense (k+1) × k upper Hessenberg matrix.\n\nReference\n\nW. E. Arnoldi, The principle of minimized iterations in the solution of the matrix eigenvalue problem, Quarterly of Applied Mathematics, 9, pp. 17–29, 1951.\n\n\n\n\n\n","category":"method"},{"location":"processes/#golub-kahan","page":"Krylov processes","title":"Golub-Kahan","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: golub_kahan)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the Golub-Kahan bidiagonalization process, the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A V_k = U_k+1 B_k \n A^H U_k+1 = V_k B_k^H + baralpha_k+1 v_k+1 e_k+1^T = V_k+1 L_k+1^H \n V_k^H V_k = U_k^H U_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where V_k and U_k are bases of the Krylov subspaces mathcalK_k (A^HAA^Hb) and mathcalK_k (AA^Hb), respectively,","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"L_k =\nbeginbmatrix\n alpha_1 \n beta_2 alpha_2 \n ddots ddots \n beta_k alpha_k\nendbmatrix\n qquad\nB_k =\nbeginbmatrix\n alpha_1 \n beta_2 alpha_2 \n ddots ddots \n beta_k alpha_k \n beta_k+1 \nendbmatrix\n=\nbeginbmatrix\n L_k \n beta_k+1 e_k^T\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Note that depending on how we normalize the vectors that compose V_k and U_k, L_k can be a real bidiagonal matrix even if A is a complex matrix.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function golub_kahan returns V_k+1, U_k+1, beta_1 and L_k+1.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: LNLQ, CRAIG, CRAIGMR, LSLQ, LSQR and LSMR.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"note: Note\nThe Golub-Kahan process coincides with the Hermitian Lanczos process applied to the normal equations A^HA x = A^Hb and AA^H x = b. It is also related to the Hermitian Lanczos process applied to beginbmatrix 0 A A^H 0 endbmatrix with initial vector beginbmatrix b 0 endbmatrix.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"golub_kahan(::Any, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.golub_kahan-Union{Tuple{FC}, Tuple{Any, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.golub_kahan","text":"V, U, β, L = golub_kahan(A, b, k)\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nk: the number of iterations of the Golub-Kahan process.\n\nOutput arguments\n\nV: a dense n × (k+1) matrix;\nU: a dense m × (k+1) matrix;\nβ: a coefficient such that βu₁ = b;\nL: a sparse (k+1) × (k+1) lower bidiagonal matrix.\n\nReferences\n\nG. H. Golub and W. Kahan, Calculating the Singular Values and Pseudo-Inverse of a Matrix, SIAM Journal on Numerical Analysis, 2(2), pp. 225–224, 1965.\nC. C. Paige, Bidiagonalization of Matrices and Solution of Linear Equations, SIAM Journal on Numerical Analysis, 11(1), pp. 197–209, 1974.\n\n\n\n\n\n","category":"method"},{"location":"processes/#saunders-simon-yip","page":"Krylov processes","title":"Saunders-Simon-Yip","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: saunders_simon_yip)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the Saunders-Simon-Yip process (also named the orthogonal tridiagonalization process), the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A U_k = V_k T_k + beta_k+1 v_k+1 e_k^T = V_k+1 T_k+1k \n A^H V_k = U_k T_k^H + bargamma_k+1 u_k+1 e_k^T = U_k+1 T_kk+1^H \n V_k^H V_k = U_k^H U_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where beginbmatrix V_k 0 0 U_k endbmatrix is an orthonormal basis of the block Krylov subspace mathcalK^square_k left(beginbmatrix 0 A A^H 0 endbmatrix beginbmatrix b 0 0 c endbmatrixright),","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"T_k = \nbeginbmatrix\n alpha_1 gamma_2 \n beta_2 alpha_2 ddots \n ddots ddots gamma_k \n beta_k alpha_k\nendbmatrix\n qquad\nT_k+1k =\nbeginbmatrix\n T_k \n beta_k+1 e_k^T\nendbmatrix\n qquad\nT_kk+1 =\nbeginbmatrix\n T_k gamma_k+1 e_k\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function saunders_simon_yip returns V_k+1, beta_1, T_k+1k, U_k+1, bargamma_1 and T_kk+1^H.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: USYMLQ, USYMQR, TriLQR, TriCG and TriMR.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"note: Note\nThe Saunders-Simon-Yip is equivalent to the block-Lanczos process applied to beginbmatrix 0 A A^H 0 endbmatrix with initial matrix beginbmatrix b 0 0 c endbmatrix.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"saunders_simon_yip(::Any, ::AbstractVector{FC}, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.saunders_simon_yip-Union{Tuple{FC}, Tuple{Any, AbstractVector{FC}, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.saunders_simon_yip","text":"V, β, T, U, γᴴ, Tᴴ = saunders_simon_yip(A, b, c, k)\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nb: a vector of length m;\nc: a vector of length n;\nk: the number of iterations of the Saunders-Simon-Yip process.\n\nOutput arguments\n\nV: a dense m × (k+1) matrix;\nβ: a coefficient such that βv₁ = b;\nT: a sparse (k+1) × k tridiagonal matrix;\nU: a dense n × (k+1) matrix;\nγᴴ: a coefficient such that γᴴu₁ = c;\nTᴴ: a sparse (k+1) × k tridiagonal matrix.\n\nReference\n\nM. A. Saunders, H. D. Simon, and E. L. Yip, Two Conjugate-Gradient-Type Methods for Unsymmetric Linear Equations, SIAM Journal on Numerical Analysis, 25(4), pp. 927–940, 1988.\n\n\n\n\n\n","category":"method"},{"location":"processes/#montoison-orban","page":"Krylov processes","title":"Montoison-Orban","text":"","category":"section"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"(Image: montoison_orban)","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"After k iterations of the Montoison-Orban process (also named the orthogonal Hessenberg reduction process), the situation may be summarized as","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"beginalign*\n A U_k = V_k H_k + h_k+1k v_k+1 e_k^T = V_k+1 H_k+1k \n B V_k = U_k F_k + f_k+1k u_k+1 e_k^T = U_k+1 F_k+1k \n V_k^H V_k = U_k^H U_k = I_k\nendalign*","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"where beginbmatrix V_k 0 0 U_k endbmatrix is an orthonormal basis of the block Krylov subspace mathcalK^square_k left(beginbmatrix 0 A B 0 endbmatrix beginbmatrix b 0 0 c endbmatrixright),","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"H_k =\nbeginbmatrix\n h_11 h_12 ldots h_1k \n h_21 ddots ddots vdots \n ddots ddots h_k-1k \n h_kk-1 h_kk\nendbmatrix\n qquad\nF_k =\nbeginbmatrix\n f_11 f_12 ldots f_1k \n f_21 ddots ddots vdots \n ddots ddots f_k-1k \n f_kk-1 f_kk\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"H_k+1k =\nbeginbmatrix\n H_k \n h_k+1k e_k^T\nendbmatrix\n qquad\nF_k+1k =\nbeginbmatrix\n F_k \n f_k+1k e_k^T\nendbmatrix","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"The function montoison_orban returns V_k+1, beta, H_k+1k, U_k+1, gamma and F_k+1k.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"Related methods: GPMR.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"note: Note\nThe Montoison-Orban is equivalent to the block-Arnoldi process applied to beginbmatrix 0 A B 0 endbmatrix with initial matrix beginbmatrix b 0 0 c endbmatrix. It also coincides with the Saunders-Simon-Yip process when B = A^H.","category":"page"},{"location":"processes/","page":"Krylov processes","title":"Krylov processes","text":"montoison_orban(::Any, ::Any, ::AbstractVector{FC}, ::AbstractVector{FC}, ::Int) where FC <: (Union{Complex{T}, T} where T <: AbstractFloat)","category":"page"},{"location":"processes/#Krylov.montoison_orban-Union{Tuple{FC}, Tuple{Any, Any, AbstractVector{FC}, AbstractVector{FC}, Int64}} where FC<:(Union{Complex{T}, T} where T<:AbstractFloat)","page":"Krylov processes","title":"Krylov.montoison_orban","text":"V, β, H, U, γ, F = montoison_orban(A, B, b, c, k; reorthogonalization=false)\n\nInput arguments\n\nA: a linear operator that models a matrix of dimension m × n;\nB: a linear operator that models a matrix of dimension n × m;\nb: a vector of length m;\nc: a vector of length n;\nk: the number of iterations of the Montoison-Orban process.\n\nKeyword argument\n\nreorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors.\n\nOutput arguments\n\nV: a dense m × (k+1) matrix;\nβ: a coefficient such that βv₁ = b;\nH: a dense (k+1) × k upper Hessenberg matrix;\nU: a dense n × (k+1) matrix;\nγ: a coefficient such that γu₁ = c;\nF: a dense (k+1) × k upper Hessenberg matrix.\n\nReference\n\nA. Montoison and D. Orban, GPMR: An Iterative Method for Unsymmetric Partitioned Linear Systems, SIAM Journal on Matrix Analysis and Applications, 44(1), pp. 293–311, 2023.\n\n\n\n\n\n","category":"method"},{"location":"examples/crls/","page":"CRLS","title":"CRLS","text":"using MatrixMarket, SuiteSparseMatrixCollection\nusing Krylov, LinearOperators\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"well1850\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1])_b.mtx\"))[:]\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\n# Define a regularization parameter.\nλ = 1.0e-3\n\n(x, stats) = crls(A, b, λ=λ)\nshow(stats)\nresid = norm(A' * (A * x - b) + λ * x) / norm(b)\n@printf(\"CRLS: Relative residual: %8.1e\\n\", resid)\n@printf(\"CRLS: ‖x‖: %8.1e\\n\", norm(x))","category":"page"},{"location":"callbacks/#callbacks","page":"Callbacks","title":"Callbacks","text":"","category":"section"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"Each Krylov method is able to call a callback function as callback(solver) at each iteration. The callback should return true if the main loop should terminate, and false otherwise. If the method terminated because of the callback, the output status will be \"user-requested exit\". For example, if the user defines minres_callback(solver::MinresSolver), it can be passed to the solver using","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"(x, stats) = minres(A, b, callback = minres_callback)","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"If you need to write a callback that uses variables that are not in a KrylovSolver, use a closure:","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"function custom_stopping_condition(solver::KrylovSolver, A, b, r, tol)\n mul!(r, A, solver.x)\n r .-= b # r := b - Ax\n bool = norm(r) ≤ tol # tolerance based on the 2-norm of the residual\n return bool\nend\n\ncg_callback(solver) = custom_stopping_condition(solver, A, b, r, tol)\n(x, stats) = cg(A, b, callback = cg_callback)","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"Alternatively, use a structure and make it callable:","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"mutable struct CallbackWorkspace{T}\n A::Matrix{T}\n b::Vector{T}\n r::Vector{T}\n tol::T\nend\n\nfunction (workspace::CallbackWorkspace)(solver::KrylovSolver)\n mul!(workspace.r, workspace.A, solver.x)\n workspace.r .-= workspace.b\n bool = norm(workspace.r) ≤ workspace.tol\n return bool\nend\n\nbicgstab_callback = CallbackWorkspace(A, b, r, tol)\n(x, stats) = bicgstab(A, b, callback = bicgstab_callback)","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"Although the main goal of a callback is to add new stopping conditions, it can also retrieve information from the workspace of a Krylov method along the iterations. We now illustrate how to store all iterates x_k of the GMRES method.","category":"page"},{"location":"callbacks/","page":"Callbacks","title":"Callbacks","text":"S = Krylov.ktypeof(b)\nglobal X = S[] # Storage for GMRES iterates\n\nfunction gmres_callback(solver)\n z = solver.z\n k = solver.inner_iter\n nr = sum(1:k)\n V = solver.V\n R = solver.R\n y = copy(z)\n\n # Solve Rk * yk = zk\n for i = k : -1 : 1\n pos = nr + i - k\n for j = k : -1 : i+1\n y[i] = y[i] - R[pos] * y[j]\n pos = pos - j + 1\n end\n y[i] = y[i] / R[pos]\n end\n\n # xk = Vk * yk\n xk = sum(V[i] * y[i] for i = 1:k)\n push!(X, xk)\n\n return false # We don't want to add new stopping conditions\nend\n\n(x, stats) = gmres(A, b, callback = gmres_callback)","category":"page"},{"location":"gpu/#gpu","page":"GPU support","title":"GPU support","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"Krylov methods are well suited for GPU computations because they only require matrix-vector products (u leftarrow Av, u leftarrow A^Hw) and vector operations (v, u^H v, v leftarrow alpha u + beta v), which are highly parallelizable.","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"The implementations in Krylov.jl are generic so as to take advantage of the multiple dispatch and broadcast features of Julia. Those allow the implementations to be specialized automatically by the compiler for both CPU and GPU. Thus, Krylov.jl works with GPU backends that build on GPUArrays.jl, such as CUDA.jl, AMDGPU.jl, oneAPI.jl or Metal.jl.","category":"page"},{"location":"gpu/#Nvidia-GPUs","page":"GPU support","title":"Nvidia GPUs","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"All solvers in Krylov.jl can be used with CUDA.jl and allow computations on Nvidia GPUs. Problems stored in CPU format (Matrix and Vector) must first be converted to the related GPU format (CuMatrix and CuVector).","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using CUDA, Krylov\n\nif CUDA.functional()\n # CPU Arrays\n A_cpu = rand(20, 20)\n b_cpu = rand(20)\n\n # GPU Arrays\n A_gpu = CuMatrix(A_cpu)\n b_gpu = CuVector(b_cpu)\n\n # Solve a square and dense system on an Nivida GPU\n x, stats = bilq(A_gpu, b_gpu)\nend","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"Sparse matrices have a specific storage on Nvidia GPUs (CuSparseMatrixCSC, CuSparseMatrixCSR or CuSparseMatrixCOO):","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using CUDA, Krylov\nusing CUDA.CUSPARSE, SparseArrays\n\nif CUDA.functional()\n # CPU Arrays\n A_cpu = sprand(200, 100, 0.3)\n b_cpu = rand(200)\n\n # GPU Arrays\n A_csc_gpu = CuSparseMatrixCSC(A_cpu)\n A_csr_gpu = CuSparseMatrixCSR(A_cpu)\n A_coo_gpu = CuSparseMatrixCOO(A_cpu)\n b_gpu = CuVector(b_cpu)\n\n # Solve a rectangular and sparse system on an Nvidia GPU\n x_csc, stats_csc = lslq(A_csc_gpu, b_gpu)\n x_csr, stats_csr = lsqr(A_csr_gpu, b_gpu)\n x_coo, stats_coo = lsmr(A_coo_gpu, b_gpu)\nend","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"If you use a Krylov method that only requires A * v products (see here), the most efficient format is CuSparseMatrixCSR. Optimized operator-vector and operator-matrix products that exploit GPU features can be also used by means of linear operators.","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"For instance, when executing sparse matrix products on NVIDIA GPUs, the mul! function utilized by Krylov.jl internally calls three routines from CUSPARSE. The first one conducts an analysis of the sparse matrix's structure, the second one determines the size of the buffer that needs to be allocated, and the last one handles the product using the allocated buffer. To circumvent the need for repeated analysis computations and buffer allocation/deallocation with each product, we introduce a KrylovOperator in KrylovPreconditioners.jl. This operator performs the analysis and allocates the buffer only once.","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using Krylov, KrylovPreconditioners\n\n# A_gpu can be a sparse COO, CSC or CSR matrix\nopA_gpu = KrylovOperator(A_gpu)\nx_gpu, stats = gmres(opA_gpu, b_gpu)","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"Preconditioners, especially incomplete Cholesky or incomplete LU factorizations that involve sparse triangular solves, can be applied directly on GPU thanks to efficient operators (like TriangularOperator) that take advantage of CUSPARSE routines.","category":"page"},{"location":"gpu/#Example-with-a-symmetric-positive-definite-system","page":"GPU support","title":"Example with a symmetric positive-definite system","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using SparseArrays, Krylov, LinearOperators\nusing CUDA, CUDA.CUSPARSE\n\nif CUDA.functional()\n # Transfer the linear system from the CPU to the GPU\n A_gpu = CuSparseMatrixCSR(A_cpu) # A_gpu = CuSparseMatrixCSC(A_cpu)\n b_gpu = CuVector(b_cpu)\n\n # IC(0) decomposition LLᴴ ≈ A for CuSparseMatrixCSC or CuSparseMatrixCSR matrices\n P = ic02(A_gpu)\n\n # Additional vector required for solving triangular systems\n n = length(b_gpu)\n T = eltype(b_gpu)\n z = CUDA.zeros(T, n)\n\n # Solve Py = x\n function ldiv_ic0!(P::CuSparseMatrixCSR, x, y, z)\n ldiv!(z, LowerTriangular(P), x) # Forward substitution with L\n ldiv!(y, LowerTriangular(P)', z) # Backward substitution with Lᴴ\n return y\n end\n\n function ldiv_ic0!(P::CuSparseMatrixCSC, x, y, z)\n ldiv!(z, UpperTriangular(P)', x) # Forward substitution with L\n ldiv!(y, UpperTriangular(P), z) # Backward substitution with Lᴴ\n return y\n end\n\n # Operator that model P⁻¹\n symmetric = hermitian = true\n opM = LinearOperator(T, n, n, symmetric, hermitian, (y, x) -> ldiv_ic0!(P, x, y, z))\n\n # Solve an Hermitian positive definite system with an IC(0) preconditioner on GPU\n x, stats = cg(A_gpu, b_gpu, M=opM)\nend","category":"page"},{"location":"gpu/#Example-with-a-general-square-system","page":"GPU support","title":"Example with a general square system","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using SparseArrays, Krylov, LinearOperators\nusing CUDA, CUDA.CUSPARSE, CUDA.CUSOLVER\n\nif CUDA.functional()\n # Optional -- Compute a permutation vector p such that A[:,p] has no zero diagonal\n p = zfd(A_cpu)\n p .+= 1\n A_cpu = A_cpu[:,p]\n\n # Transfer the linear system from the CPU to the GPU\n A_gpu = CuSparseMatrixCSR(A_cpu) # A_gpu = CuSparseMatrixCSC(A_cpu)\n b_gpu = CuVector(b_cpu)\n\n # ILU(0) decomposition LU ≈ A for CuSparseMatrixCSC or CuSparseMatrixCSR matrices\n P = ilu02(A_gpu)\n\n # Additional vector required for solving triangular systems\n n = length(b_gpu)\n T = eltype(b_gpu)\n z = CUDA.zeros(T, n)\n\n # Solve Py = x\n function ldiv_ilu0!(P::CuSparseMatrixCSR, x, y, z)\n ldiv!(z, UnitLowerTriangular(P), x) # Forward substitution with L\n ldiv!(y, UpperTriangular(P), z) # Backward substitution with U\n return y\n end\n\n function ldiv_ilu0!(P::CuSparseMatrixCSC, x, y, z)\n ldiv!(z, LowerTriangular(P), x) # Forward substitution with L\n ldiv!(y, UnitUpperTriangular(P), z) # Backward substitution with U\n return y\n end\n\n # Operator that model P⁻¹\n symmetric = hermitian = false\n opM = LinearOperator(T, n, n, symmetric, hermitian, (y, x) -> ldiv_ilu0!(P, x, y, z))\n\n # Solve a non-Hermitian system with an ILU(0) preconditioner on GPU\n x̄, stats = bicgstab(A_gpu, b_gpu, M=opM)\n\n # Recover the solution of Ax = b with the solution of A[:,p]x̄ = b\n invp = invperm(p)\n x = x̄[invp]\nend","category":"page"},{"location":"gpu/#AMD-GPUs","page":"GPU support","title":"AMD GPUs","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"All solvers in Krylov.jl can be used with AMDGPU.jl and allow computations on AMD GPUs. Problems stored in CPU format (Matrix and Vector) must first be converted to the related GPU format (ROCMatrix and ROCVector).","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using Krylov, AMDGPU\n\nif AMDGPU.functional()\n # CPU Arrays\n A_cpu = rand(ComplexF64, 20, 20)\n A_cpu = A_cpu + A_cpu'\n b_cpu = rand(ComplexF64, 20)\n\n A_gpu = ROCMatrix(A_cpu)\n b_gpu = ROCVector(b_cpu)\n\n # Solve a dense Hermitian system on an AMD GPU\n x, stats = minres(A_gpu, b_gpu)\nend","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"Sparse matrices have a specific storage on AMD GPUs (ROCSparseMatrixCSC, ROCSparseMatrixCSR or ROCSparseMatrixCOO):","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using AMDGPU, Krylov\nusing AMDGPU.rocSPARSE, SparseArrays\n\nif AMDGPU.functional()\n # CPU Arrays\n A_cpu = sprand(100, 200, 0.3)\n b_cpu = rand(100)\n\n # GPU Arrays\n A_csc_gpu = ROCSparseMatrixCSC(A_cpu)\n A_csr_gpu = ROCSparseMatrixCSR(A_cpu)\n A_coo_gpu = ROCSparseMatrixCOO(A_cpu)\n b_gpu = ROCVector(b_cpu)\n\n # Solve a rectangular and sparse system on an AMD GPU\n x_csc, y_csc, stats_csc = lnlq(A_csc_gpu, b_gpu)\n x_csr, y_csr, stats_csr = craig(A_csr_gpu, b_gpu)\n x_coo, y_coo, stats_coo = craigmr(A_coo_gpu, b_gpu)\nend","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"When executing sparse products on AMD GPUs, the mul! function utilized by Krylov.jl internally calls three routines from rocSPARSE. The first one conducts an analysis of the sparse matrix's structure, the second one determines the size of the buffer that needs to be allocated, and the last one handles the product using the allocated buffer. To circumvent the need for repeated analysis computations and buffer allocation/deallocation with each product, we introduce a KrylovOperator in KrylovPreconditioners.jl. This operator performs the analysis and allocates the buffer only once.","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using Krylov, KrylovPreconditioners\n\n# A_gpu can be a sparse COO, CSC or CSR matrix\nopA_gpu = KrylovOperator(A_gpu)\nx_gpu, stats = bicgstab(opA_gpu, b_gpu)","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"Preconditioners, especially incomplete Cholesky or incomplete LU factorizations that involve sparse triangular solves, can be applied directly on GPU thanks to efficient operators (like TriangularOperator) that take advantage of rocSPARSE routines.","category":"page"},{"location":"gpu/#Intel-GPUs","page":"GPU support","title":"Intel GPUs","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"All solvers in Krylov.jl can be used with oneAPI.jl and allow computations on Intel GPUs. Problems stored in CPU format (Matrix and Vector) must first be converted to the related GPU format (oneMatrix and oneVector).","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using Krylov, oneAPI\n\nif oneAPI.functional()\n T = Float32 # oneAPI.jl also works with ComplexF32\n m = 20\n n = 10\n\n # CPU Arrays\n A_cpu = rand(T, m, n)\n b_cpu = rand(T, m)\n\n # GPU Arrays\n A_gpu = oneMatrix(A_cpu)\n b_gpu = oneVector(b_cpu)\n\n # Solve a dense least-squares problem on an Intel GPU\n x, stats = lsqr(A_gpu, b_gpu)\nend","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"note: Note\nThe library oneMKL is interfaced in oneAPI.jl and accelerates linear algebra operations on Intel GPUs. Only dense linear systems are supported for the time being because sparse linear algebra routines are not interfaced yet.","category":"page"},{"location":"gpu/#Apple-M1-GPUs","page":"GPU support","title":"Apple M1 GPUs","text":"","category":"section"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"All solvers in Krylov.jl can be used with Metal.jl and allow computations on Apple M1 GPUs. Problems stored in CPU format (Matrix and Vector) must first be converted to the related GPU format (MtlMatrix and MtlVector).","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"using Krylov, Metal\n\nT = Float32 # Metal.jl also works with ComplexF32\nn = 10\nm = 20\n\n# CPU Arrays\nA_cpu = rand(T, n, m)\nb_cpu = rand(T, n)\n\n# GPU Arrays\nA_gpu = MtlMatrix(A_cpu)\nb_gpu = MtlVector(b_cpu)\n\n# Solve a dense least-norm problem on an Apple M1 GPU\nx, stats = craig(A_gpu, b_gpu)","category":"page"},{"location":"gpu/","page":"GPU support","title":"GPU support","text":"warning: Warning\nMetal.jl is under heavy development and is considered experimental for now.","category":"page"},{"location":"#Home","page":"Home","title":"Krylov.jl documentation","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"This package provides implementations of certain of the most useful Krylov method for a variety of problems:","category":"page"},{"location":"","page":"Home","title":"Home","text":"1 - Square or rectangular full-rank systems","category":"page"},{"location":"","page":"Home","title":"Home","text":" Ax = b","category":"page"},{"location":"","page":"Home","title":"Home","text":"should be solved when b lies in the range space of A. This situation occurs when","category":"page"},{"location":"","page":"Home","title":"Home","text":"A is square and nonsingular,\nA is tall and has full column rank and b lies in the range of A.","category":"page"},{"location":"","page":"Home","title":"Home","text":"2 - Linear least-squares problems","category":"page"},{"location":"","page":"Home","title":"Home","text":" min b - Ax","category":"page"},{"location":"","page":"Home","title":"Home","text":"should be solved when b is not in the range of A (inconsistent systems), regardless of the shape and rank of A. This situation mainly occurs when","category":"page"},{"location":"","page":"Home","title":"Home","text":"A is square and singular,\nA is tall and thin.","category":"page"},{"location":"","page":"Home","title":"Home","text":"Underdetermined systems are less common but also occur.","category":"page"},{"location":"","page":"Home","title":"Home","text":"If there are infinitely many such x (because A is column rank-deficient), one with minimum norm is identified","category":"page"},{"location":"","page":"Home","title":"Home","text":" min x quad textsubject to quad x in argmin b - Ax","category":"page"},{"location":"","page":"Home","title":"Home","text":"3 - Linear least-norm problems","category":"page"},{"location":"","page":"Home","title":"Home","text":" min x quad textsubject to quad Ax = b","category":"page"},{"location":"","page":"Home","title":"Home","text":"should be solved when A is column rank-deficient but b is in the range of A (consistent systems), regardless of the shape of A. This situation mainly occurs when","category":"page"},{"location":"","page":"Home","title":"Home","text":"A is square and singular,\nA is short and wide.","category":"page"},{"location":"","page":"Home","title":"Home","text":"Overdetermined systems are less common but also occur.","category":"page"},{"location":"","page":"Home","title":"Home","text":"4 - Adjoint systems","category":"page"},{"location":"","page":"Home","title":"Home","text":" Ax = b quad textand quad A^H y = c","category":"page"},{"location":"","page":"Home","title":"Home","text":"where A can have any shape.","category":"page"},{"location":"","page":"Home","title":"Home","text":"5 - Saddle-point and Hermitian quasi-definite systems","category":"page"},{"location":"","page":"Home","title":"Home","text":" beginbmatrix M phantom-A A^H -N endbmatrix beginbmatrix x y endbmatrix = left(beginbmatrix b 0 endbmatrixbeginbmatrix 0 c endbmatrixbeginbmatrix b c endbmatrixright)","category":"page"},{"location":"","page":"Home","title":"Home","text":"where A can have any shape.","category":"page"},{"location":"","page":"Home","title":"Home","text":"6 - Generalized saddle-point and non-Hermitian partitioned systems","category":"page"},{"location":"","page":"Home","title":"Home","text":" beginbmatrix M A B N endbmatrix beginbmatrix x y endbmatrix = beginbmatrix b c endbmatrix","category":"page"},{"location":"","page":"Home","title":"Home","text":"where A can have any shape and B has the shape of Aᴴ. A, B, b and c must be all nonzero.","category":"page"},{"location":"","page":"Home","title":"Home","text":"Krylov solvers are particularly appropriate in situations where such problems must be solved but a factorization is not possible, either because:","category":"page"},{"location":"","page":"Home","title":"Home","text":"A is not available explicitly,\nA would be dense or would consume an excessive amount of memory if it were materialized,\nfactors would consume an excessive amount of memory.","category":"page"},{"location":"","page":"Home","title":"Home","text":"Iterative methods are recommended in either of the following situations:","category":"page"},{"location":"","page":"Home","title":"Home","text":"the problem is sufficiently large that a factorization is not feasible or would be slow,\nan effective preconditioner is known in cases where the problem has unfavorable spectral structure,\nthe operator can be represented efficiently as a sparse matrix,\nthe operator is fast, i.e., can be applied with better complexity than if it were materialized as a matrix. Certain fast operators would materialize as dense matrices.","category":"page"},{"location":"#Features","page":"Home","title":"Features","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"All solvers in Krylov.jl have in-place version, are compatible with GPU and work in any floating-point data type.","category":"page"},{"location":"#How-to-Install","page":"Home","title":"How to Install","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Krylov can be installed and tested through the Julia package manager:","category":"page"},{"location":"","page":"Home","title":"Home","text":"julia> ]\npkg> add Krylov\npkg> test Krylov","category":"page"},{"location":"#Bug-reports-and-discussions","page":"Home","title":"Bug reports and discussions","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"If you think you found a bug, feel free to open an issue. Focused suggestions and requests can also be opened as issues. Before opening a pull request, start an issue or a discussion on the topic, please.","category":"page"},{"location":"","page":"Home","title":"Home","text":"If you want to ask a question not suited for a bug report, feel free to start a discussion here. This forum is for general discussion about this repository and the JuliaSmoothOptimizers organization, so questions about any of our packages are welcome.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"","category":"page"},{"location":"matrix_free/#matrix-free","page":"Matrix-free operators","title":"Matrix-free operators","text":"","category":"section"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"All methods are matrix-free, which means that you only need to provide operator-vector products.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"The A or B input arguments of Krylov.jl solvers can be any object that represents a linear operator. That object must implement mul!, for multiplication with a vector, size() and eltype(). For certain methods it must also implement adjoint().","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"Some methods only require A * v products, whereas other ones also require A' * u products. In the latter case, adjoint(A) must also be implemented.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"A * v A * v and A' * u\nCG, CR, CAR CGLS, CRLS, CGNE, CRMR\nSYMMLQ, CG-LANCZOS, MINRES, MINRES-QLP, MINARES LSLQ, LSQR, LSMR, LNLQ, CRAIG, CRAIGMR\nDIOM, FOM, DQGMRES, GMRES, FGMRES, BLOCK-GMRES BiLQ, QMR, BiLQR, USYMLQ, USYMQR, TriLQR\nCGS, BICGSTAB TriCG, TriMR\nCG-LANCZOS-SHIFT CGLS-LANCZOS-SHIFT","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"info: Info\nGPMR is the only method that requires A * v and B * w products.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"Preconditioners M, N, C, D, E or F can be also linear operators and must implement mul! or ldiv!.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"We strongly recommend LinearOperators.jl to model matrix-free operators, but other packages such as LinearMaps.jl, DiffEqOperators.jl or your own operator can be used as well.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"With LinearOperators.jl, operators are defined as","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"A = LinearOperator(type, nrows, ncols, symmetric, hermitian, prod, tprod, ctprod)","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"where","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"type is the operator element type;\nnrow and ncol are its dimensions;\nsymmetric and hermitian should be set to true or false;\nprod(y, v), tprod(y, w) and ctprod(u, w) are called when writing mul!(y, A, v), mul!(y, transpose(A), w), and mul!(y, A', u), respectively.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"See the tutorial and the detailed documentation for more information on LinearOperators.jl.","category":"page"},{"location":"matrix_free/#Examples-with-automatic-differentiation","page":"Matrix-free operators","title":"Examples with automatic differentiation","text":"","category":"section"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"In the field of nonlinear optimization, finding critical points of a continuous function frequently involves linear systems with a Hessian or Jacobian as coefficient. Materializing such operators as matrices is expensive in terms of operations and memory consumption and is unreasonable for high-dimensional problems. However, it is often possible to implement efficient Hessian-vector and Jacobian-vector products, for example with the help of automatic differentiation tools, and used within Krylov solvers. We now illustrate variants with explicit matrices and with matrix-free operators for two well-known optimization methods.","category":"page"},{"location":"matrix_free/#Example-1:-Newton's-Method-for-convex-optimization","page":"Matrix-free operators","title":"Example 1: Newton's Method for convex optimization","text":"","category":"section"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"At each iteration of Newton's method applied to a mathcalC^2 strictly convex function f mathbbR^n rightarrow mathbbR, a descent direction direction is determined by minimizing the quadratic Taylor model of f:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"min_d in mathbbR^nf(x_k) + nabla f(x_k)^T d + tfrac12d^T nabla^2 f(x_k) d","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"which is equivalent to solving the symmetric and positive-definite system","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"nabla^2 f(x_k) d = -nabla f(x_k)","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"The system above can be solved with the conjugate gradient method as follows, using the explicit Hessian:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"using ForwardDiff, Krylov\n\nxk = -ones(4)\n\nf(x) = (x[1] - 1)^2 + (x[2] - 2)^2 + (x[3] - 3)^2 + (x[4] - 4)^2\n\ng(x) = ForwardDiff.gradient(f, x)\n\nH(x) = ForwardDiff.hessian(f, x)\n\nd, stats = cg(H(xk), -g(xk))","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"The explicit Hessian can be replaced by a linear operator that only computes Hessian-vector products:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"using ForwardDiff, LinearOperators, Krylov\n\nxk = -ones(4)\n\nf(x) = (x[1] - 1)^2 + (x[2] - 2)^2 + (x[3] - 3)^2 + (x[4] - 4)^2\n\ng(x) = ForwardDiff.gradient(f, x)\n\nH(y, v) = ForwardDiff.derivative!(y, t -> g(xk + t * v), 0)\nopH = LinearOperator(Float64, 4, 4, true, true, (y, v) -> H(y, v))\n\ncg(opH, -g(xk))","category":"page"},{"location":"matrix_free/#Example-2:-The-Gauss-Newton-Method-for-Nonlinear-Least-Squares","page":"Matrix-free operators","title":"Example 2: The Gauss-Newton Method for Nonlinear Least Squares","text":"","category":"section"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"At each iteration of the Gauss-Newton method applied to a nonlinear least-squares objective f(x) = tfrac12 F(x)^2 where F mathbbR^n rightarrow mathbbR^m is mathcalC^1, we solve the subproblem:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"min_d in mathbbR^ntfrac12J(x_k) d + F(x_k)^2","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"where J(x) is the Jacobian of F at x.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"An appropriate iterative method to solve the above linear least-squares problems is LSMR. We could pass the explicit Jacobian to LSMR as follows:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"using ForwardDiff, Krylov\n\nxk = ones(2)\n\nF(x) = [x[1]^4 - 3; exp(x[2]) - 2; log(x[1]) - x[2]^2]\n\nJ(x) = ForwardDiff.jacobian(F, x)\n\nd, stats = lsmr(J(xk), -F(xk))","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"However, the explicit Jacobian can be replaced by a linear operator that only computes Jacobian-vector and transposed Jacobian-vector products:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"using LinearAlgebra, ForwardDiff, LinearOperators, Krylov\n\nxk = ones(2)\n\nF(x) = [x[1]^4 - 3; exp(x[2]) - 2; log(x[1]) - x[2]^2]\n\nJ(y, v) = ForwardDiff.derivative!(y, t -> F(xk + t * v), 0)\nJᵀ(y, u) = ForwardDiff.gradient!(y, x -> dot(F(x), u), xk)\nopJ = LinearOperator(Float64, 3, 2, false, false, (y, v) -> J(y, v),\n (y, w) -> Jᵀ(y, w),\n (y, u) -> Jᵀ(y, u))\n\nlsmr(opJ, -F(xk))","category":"page"},{"location":"matrix_free/#Example-with-FFT-and-IFFT","page":"Matrix-free operators","title":"Example with FFT and IFFT","text":"","category":"section"},{"location":"matrix_free/#Example-3:-Solving-the-Poisson-equation-with-FFT-and-IFFT","page":"Matrix-free operators","title":"Example 3: Solving the Poisson equation with FFT and IFFT","text":"","category":"section"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"In applications related to partial differential equations (PDEs), linear systems can arise from discretizing differential operators. Storing such operators as explicit matrices is computationally expensive and unnecessary when matrix-free methods can be used, particularly with structured grids.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"The FFT is an algorithm that computes the discrete Fourier transform (DFT) of a sequence, transforming data from the spatial domain to the frequency domain. In the context of solving PDEs, it simplifies the application of differential operators like the Laplacian by converting derivatives into algebraic operations.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"For a function u(x) discretized on a periodic grid with n points, the FFT of u is:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"hatu_k = sum_j=0^n-1 u_j e^-i k x_j","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"where hatu_k represents the Fourier coefficients for the frequency k, and u_j is the value of u at the grid point x_j defined as x_j = frac2 pi jL with period L. The inverse FFT (IFFT) reconstructs u from its Fourier coefficients:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"u_j = frac1n sum_k=0^n-1 hatu_k e^i k x_j","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"In Fourier space, the Laplacian operator fracd^2dx^2 becomes a simple multiplication by -k^2, where k is the wavenumber derived from the grid size. This transforms the Poisson equation fracd^2 u(x)dx^2 = f(x) into an algebraic equation in the frequency domain:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"-k^2 hatu_k = hatf_k","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"By solving for hatu_k and applying the IFFT, we can recover the solution u(x) efficiently.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"The inverse FFT is used to convert data from the frequency domain back to the spatial domain. Once the solution in frequency space is obtained by dividing the Fourier coefficients hatf_k by -k^2 for k neq 0, the IFFT is applied to transform the result back to the original grid points in the spatial domain. At k = 0, the equation -k^2 hatu_0 = hatf_0 becomes indeterminate since k^2 = 0. This situation corresponds to the zero-frequency component hatf_0, which represents the mean of f(x). In such cases, hatu_0 is treated separately. It is typically set to 0 to remove the constant mode, or adjusted based on boundary conditions or other constraints.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"In some cases, even though the FFT provides an efficient way to apply differential operators (such as the Laplacian) in the frequency domain, a direct solution may not be feasible due to complex boundary conditions, variable coefficients, or grid irregularities. In these situations, the FFT must be coupled with a Krylov method to iteratively solve the problem.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"This example consists of solving the 1D Poisson equation on a periodic domain 0 4pi:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"fracd^2 u(x)dx^2 = f(x)","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"where u(x) is the unknown solution, and f(x) is the given source term. We solve this equation using FFTW.jl to compute the matrix-free action of the Laplacian within the conjugate gradient solver.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"Note that while a direct FFT-based approach can be used here due to the simplicity of the periodic boundary conditions, this example illustrates how a Krylov method can be employed to solve more challenging problems.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"using FFTW, Krylov, LinearAlgebra\n\n# Define the problem size and domain\nn = 32768 # Number of grid points (2^15)\nL = 4π # Length of the domain\nx = LinRange(0, L, n+1)[1:end-1] # Periodic grid (excluding the last point)\n\n# Define the source term f(x)\nf = sin.(x)\n\n# Define a matrix-free operator using FFT and IFFT\nstruct FFTPoissonOperator\n n::Int\n L::Float64\n complex::Bool\n k::Vector{Float64} # Store Fourier wave numbers\nend\n\nfunction FFTPoissonOperator(n::Int, L::Float64, complex::Bool)\n if complex\n k = Vector{Float64}(undef, n)\n else\n k = Vector{Float64}(undef, n÷2 + 1)\n end\n k[1] = sum(f) / n # average value of f(x) over the domain\n for j in 1:(n÷2)\n k[j+1] = 2 * π * j / L # Positive wave numbers\n end\n if complex\n for j in 1:(n÷2 - 1)\n k[n-j+1] = -2 * π * j / L # Negative wave numbers\n end\n end\n return FFTPoissonOperator(n, L, complex, k)\nend\n\nBase.size(A::FFTPoissonOperator) = (n, n)\n\nfunction Base.eltype(A::FFTPoissonOperator)\n type = A.complex ? ComplexF64 : Float64\n return type\nend\n\nfunction LinearAlgebra.mul!(y::Vector, A::FFTPoissonOperator, u::Vector)\n # Transform the input vector `u` to the frequency domain using `fft` or `rfft`.\n # If the operator is complex, use the full FFT; otherwise, use the real FFT.\n if A.complex\n u_hat = fft(u)\n else\n u_hat = rfft(u)\n end\n\n # In Fourier space, solve the system by multiplying with -k^2 (corresponding to the second derivative).\n # This step applies the Laplacian operator in the frequency domain.\n u_hat .= -u_hat .* (A.k .^ 2)\n\n # Transform the result back to the spatial domain using `ifft` or `irfft`.\n # If the operator is complex, use the full inverse FFT; otherwise, use the inverse real FFT.\n if A.complex\n y .= ifft(u_hat)\n else\n y .= irfft(u_hat, A.n)\n end\n\n return y\nend\n\n# Create the matrix-free operator for the Poisson equation\ncomplex = false\nA = FFTPoissonOperator(n, L, complex)\n\n# Solve the linear system using CG\nu_sol, stats = cg(A, f, atol=1e-10, rtol=0.0, verbose=1)\n\n# The exact solution is u(x) = -sin(x)\nu_star = -sin.(x)\nu_star ≈ u_sol","category":"page"},{"location":"matrix_free/#Example-with-discretized-PDE","page":"Matrix-free operators","title":"Example with discretized PDE","text":"","category":"section"},{"location":"matrix_free/#Example-4:-Solving-the-3D-Helmholtz-equation","page":"Matrix-free operators","title":"Example 4: Solving the 3D Helmholtz equation","text":"","category":"section"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"The Helmholtz equation in 3D is a fundamental equation used in various fields like acoustics, electromagnetism, and quantum mechanics to model stationary wave phenomena. It is given by:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"nabla^2 u(xyz) + k^2 u(xyz) = f(xyz)","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"where:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"u(x y z)\nis the unknown function representing, for example, a pressure field in acoustics, a scalar potential in electromagnetism, or a wave function in quantum mechanics,\nnabla^2\nis the Laplacian operator in three dimensions,\nk\nis the wave number, related to the frequency of the wave by k = frac2pilambda, where lambda is the wavelength,\nf(xyz)\nis a source term that drives the wave phenomena, such as a forcing function or external influence.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"To discretize the Helmholtz equation, we use finite differences on a uniform 3D grid with grid spacings Delta x, Delta y, and Delta z. For a grid point (i j k), the second derivatives are approximated as follows:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"In the x-direction:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"fracpartial^2 upartial x^2 approx fracu_i+1jk - 2u_ijk + u_i-1jkDelta x^2","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"In the y-direction:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"fracpartial^2 upartial y^2 approx fracu_ij+1k - 2u_ijk + u_ij-1kDelta y^2","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"In the z-direction:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"fracpartial^2 upartial z^2 approx fracu_ijk+1 - 2u_ijk + u_ijk-1Delta z^2","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"Combining these, the discretized Helmholtz equation becomes:","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"fracu_i+1jk - 2u_ijk + u_i-1jkDelta x^2 + fracu_ij+1k - 2u_ijk + u_ij-1kDelta y^2 + fracu_ijk+1 - 2u_ijk + u_ijk-1Delta z^2 + k^2 u_ijk = f_ijk","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"This discretization results in an equation at each grid point, resulting in a large and sparse linear system when assembled across the entire 3D grid. To simplify the example, we impose Dirichlet boundary conditions with the solution u(x y z) = 0 on the boundary of the cubic domain.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"Explicitly constructing this large sparse matrix is often impractical and unnecessary. Instead, we can define a function that directly applies the Helmholtz operator to the 3D grid, avoiding the need to form the matrix explicitly.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"Krylov.jl operates on vectors, so we must vectorize both the solution and the computational domain. However, we can still maintain the structure of the original 3D operator by using reshape and vec. This approach enables a simpler and efficient application of the operator in 3D while leveraging the vectorized framework for linear algebra operations.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"using Krylov, LinearAlgebra\n\n# Parameters\nL = 1.0 # Length of the cubic domain\nNx = 200 # Number of interior grid points in x\nNy = 200 # Number of interior grid points in y\nNz = 200 # Number of interior grid points in z\nΔx = L / (Nx + 1) # Grid spacing in x\nΔy = L / (Ny + 1) # Grid spacing in y\nΔz = L / (Nz + 1) # Grid spacing in z\nwavelength = 0.5 # Wavelength of the wave\nk = 2 * π / wavelength # Wave number\n\n# Create the grid points\nx = 0:Δx:L # Points in x dimension (Nx + 2)\ny = 0:Δy:L # Points in y dimension (Ny + 2)\nz = 0:Δz:L # Points in z dimension (Nz + 2)\n\n# Define a matrix-free Helmholtz operator\nstruct HelmholtzOperator\n Nx::Int\n Ny::Int\n Nz::Int\n Δx::Float64\n Δy::Float64\n Δz::Float64\n k::Float64\nend\n\nBase.size(A::HelmholtzOperator) = (A.Nx * A.Ny * A.Nz, A.Nx * A.Ny * A.Nz)\n\nBase.eltype(A::HelmholtzOperator) = Float64\n\nfunction LinearAlgebra.mul!(y::Vector, A::HelmholtzOperator, u::Vector)\n # Reshape vectors y and u into 3D arrays\n U = reshape(u, A.Nx, A.Ny, A.Nz)\n Y = reshape(y, A.Nx, A.Ny, A.Nz)\n\n # Apply the discrete Laplacian in 3D with k^2 * u\n for i in 1:A.Nx\n for j in 1:A.Ny\n for k in 1:A.Nz\n if i == 1\n dx2 = (U[i+1,j,k] -2 * U[i,j,k]) / (A.Δx)^2\n elseif i == A.Nx\n dx2 = (-2 * U[i,j,k] + U[i-1,j,k]) / (A.Δx)^2\n else\n dx2 = (U[i+1,j,k] -2 * U[i,j,k] + U[i-1,j,k]) / (A.Δx)^2\n end\n\n if j == 1\n dy2 = (U[i,j+1,k] -2 * U[i,j,k]) / (A.Δy)^2\n elseif j == A.Ny\n dy2 = (-2 * U[i,j,k] + U[i,j-1,k]) / (A.Δy)^2\n else\n dy2 = (U[i,j+1,k] -2 * U[i,j,k] + U[i,j-1,k]) / (A.Δy)^2\n end\n\n if k == 1\n dz2 = (U[i,j,k+1] -2 * U[i,j,k]) / (A.Δz)^2\n elseif k == A.Nz\n dz2 = (-2 * U[i,j,k] + U[i,j,k-1]) / (A.Δz)^2\n else\n dz2 = (U[i,j,k+1] -2 * U[i,j,k] + U[i,j,k-1]) / (A.Δz)^2\n end\n\n Y[i,j,k] = dx2 + dy2 + dz2 + (A.k)^2 * U[i,j,k]\n end\n end\n end\n\n return y\nend\n\n# Create the matrix-free operator for the Helmholtz equation\nA = HelmholtzOperator(Nx, Ny, Nz, Δx, Δy, Δz, k)\n\n# Source term f(x, y, z) = -2k² * sin(kx) * sin(ky) * sin(kz)\nF = [-2 * k^2 * sin(k * x[ii+1]) * sin(k * y[jj+1]) * sin(k * z[kk+1]) for ii in 1:Nx, jj in 1:Ny, kk in 1:Nz]\nf = vec(F)\n\n# Solve the linear system using MinAres\nu_sol, stats = minares(A, f, atol=1e-10, rtol=0.0, verbose=1)\n\n# Solution as 3D array\nU_sol = reshape(u_sol, Nx, Ny, Nz)\n\n# Exact solution u(x,y,z) = sin(kx) * sin(ky) * sin(kz)\nU_star = [sin(k * x[ii+1]) * sin(k * y[jj+1]) * sin(k * z[kk+1]) for ii in 1:Nx, jj in 1:Ny, kk in 1:Nz]\n\n# Compute the maximum error between the numerical solution U_sol and the exact solution U_star\nnorm(U_sol - U_star, Inf)","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"Note that preconditioners can be also implemented as abstract operators. For instance, we could compute the Cholesky factorization of M and N and create linear operators that perform the forward and backsolves.","category":"page"},{"location":"matrix_free/","page":"Matrix-free operators","title":"Matrix-free operators","text":"Krylov methods combined with factorization free operators allow to reduce computation time and memory requirements considerably by avoiding building and storing the system matrix. In the field of partial differential equations, the implementation of high-performance factorization free operators and assembly free preconditioning is a subject of active research.","category":"page"},{"location":"examples/cgls/","page":"CGLS","title":"CGLS","text":"using MatrixMarket, SuiteSparseMatrixCollection\nusing Krylov, LinearOperators\nusing LinearAlgebra, Printf\n\nssmc = ssmc_db(verbose=false)\nmatrix = ssmc_matrices(ssmc, \"HB\", \"well1033\")\npath = fetch_ssmc(matrix, format=\"MM\")\n\nA = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1]).mtx\"))\nb = MatrixMarket.mmread(joinpath(path[1], \"$(matrix.name[1])_b.mtx\"))[:]\n(m, n) = size(A)\n@printf(\"System size: %d rows and %d columns\\n\", m, n)\n\n# Define a regularization parameter.\nλ = 1.0e-3\n\n(x, stats) = cgls(A, b, λ=λ)\nshow(stats)\nresid = norm(A' * (A * x - b) + λ * x) / norm(b)\n@printf(\"CGLS: Relative residual: %8.1e\\n\", resid)\n@printf(\"CGLS: ‖x‖: %8.1e\\n\", norm(x))","category":"page"}] } diff --git a/previews/PR911/solvers/as/index.html b/previews/PR911/solvers/as/index.html index c66c9e1de..f92686a91 100644 --- a/previews/PR911/solvers/as/index.html +++ b/previews/PR911/solvers/as/index.html @@ -4,11 +4,11 @@ rtol::T=√eps(T), itmax::Int=0, timemax::Float64=Inf, verbose::Int=0, history::Bool=false, callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, y, stats) = bilqr(A, b, c, x0::AbstractVector, y0::AbstractVector; kwargs...)

BiLQR can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.

Combine BiLQ and QMR to solve adjoint systems.

[0  A] [y] = [b]
-[Aᴴ 0] [x]   [c]

The relation bᴴc ≠ 0 must be satisfied. BiLQ is used for solving primal system Ax = b of size n. QMR is used for solving dual system Aᴴy = c of size n.

Input arguments

  • A: a linear operator that models a matrix of dimension n;
  • b: a vector of length n;
  • c: a vector of length n.

Optional arguments

  • x0: a vector of length n that represents an initial guess of the solution x;
  • y0: a vector of length n that represents an initial guess of the solution y.

Keyword arguments

  • transfer_to_bicg: transfer from the BiLQ point to the BiCG point, when it exists. The transfer is based on the residual norm;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • y: a dense vector of length n;
  • stats: statistics collected on the run in an AdjointStats structure.

Reference

source
Krylov.bilqr! — Function
solver = bilqr!(solver::BilqrSolver, A, b, c; kwargs...)
-solver = bilqr!(solver::BilqrSolver, A, b, c, x0, y0; kwargs...)

where kwargs are keyword arguments of bilqr.

See BilqrSolver for more details about the solver.

source

TriLQR

Krylov.trilqr — Function
(x, y, stats) = trilqr(A, b::AbstractVector{FC}, c::AbstractVector{FC};
+[Aᴴ 0] [x]   [c]

The relation bᴴc ≠ 0 must be satisfied. BiLQ is used for solving primal system Ax = b of size n. QMR is used for solving dual system Aᴴy = c of size n.

Input arguments

  • A: a linear operator that models a matrix of dimension n;
  • b: a vector of length n;
  • c: a vector of length n.

Optional arguments

  • x0: a vector of length n that represents an initial guess of the solution x;
  • y0: a vector of length n that represents an initial guess of the solution y.

Keyword arguments

  • transfer_to_bicg: transfer from the BiLQ point to the BiCG point, when it exists. The transfer is based on the residual norm;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • y: a dense vector of length n;
  • stats: statistics collected on the run in an AdjointStats structure.

Reference

source
Krylov.bilqr! — Function
solver = bilqr!(solver::BilqrSolver, A, b, c; kwargs...)
+solver = bilqr!(solver::BilqrSolver, A, b, c, x0, y0; kwargs...)

where kwargs are keyword arguments of bilqr.

See BilqrSolver for more details about the solver.

source

TriLQR

Krylov.trilqr — Function
(x, y, stats) = trilqr(A, b::AbstractVector{FC}, c::AbstractVector{FC};
                        transfer_to_usymcg::Bool=true, atol::T=√eps(T),
                        rtol::T=√eps(T), itmax::Int=0,
                        timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
                        callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, y, stats) = trilqr(A, b, c, x0::AbstractVector, y0::AbstractVector; kwargs...)

TriLQR can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.

Combine USYMLQ and USYMQR to solve adjoint systems.

[0  A] [y] = [b]
-[Aᴴ 0] [x]   [c]

USYMLQ is used for solving primal system Ax = b of size m × n. USYMQR is used for solving dual system Aᴴy = c of size n × m.

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m;
  • c: a vector of length n.

Optional arguments

  • x0: a vector of length n that represents an initial guess of the solution x;
  • y0: a vector of length m that represents an initial guess of the solution y.

Keyword arguments

  • transfer_to_usymcg: transfer from the USYMLQ point to the USYMCG point, when it exists. The transfer is based on the residual norm;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • y: a dense vector of length m;
  • stats: statistics collected on the run in an AdjointStats structure.

Reference

source
Krylov.trilqr! — Function
solver = trilqr!(solver::TrilqrSolver, A, b, c; kwargs...)
-solver = trilqr!(solver::TrilqrSolver, A, b, c, x0, y0; kwargs...)

where kwargs are keyword arguments of trilqr.

See TrilqrSolver for more details about the solver.

source
+[Aᴴ 0] [x] [c]

USYMLQ is used for solving primal system Ax = b of size m × n. USYMQR is used for solving dual system Aᴴy = c of size n × m.

Input arguments

Optional arguments

Keyword arguments

Output arguments

Reference

source
Krylov.trilqr! — Function
solver = trilqr!(solver::TrilqrSolver, A, b, c; kwargs...)
+solver = trilqr!(solver::TrilqrSolver, A, b, c, x0, y0; kwargs...)

where kwargs are keyword arguments of trilqr.

See TrilqrSolver for more details about the solver.

source
diff --git a/previews/PR911/solvers/gsp/index.html b/previews/PR911/solvers/gsp/index.html index fece771e6..e92c6739a 100644 --- a/previews/PR911/solvers/gsp/index.html +++ b/previews/PR911/solvers/gsp/index.html @@ -9,5 +9,5 @@ callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, y, stats) = gpmr(A, B, b, c, x0::AbstractVector, y0::AbstractVector; kwargs...)

GPMR can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.

Given matrices A of dimension m × n and B of dimension n × m, GPMR solves the non-Hermitian partitioned linear system

[ λIₘ   A  ] [ x ] = [ b ]
 [  B   μIₙ ] [ y ]   [ c ],

of size (n+m) × (n+m) where λ and μ are real or complex numbers. A can have any shape and B has the shape of Aᴴ. A, B, b and c must be all nonzero.

This implementation allows left and right block diagonal preconditioners

[ C    ] [ λM   A ] [ E    ] [ E⁻¹x ] = [ Cb ]
 [    D ] [  B  μN ] [    F ] [ F⁻¹y ]   [ Dc ],

and can solve

[ λM   A ] [ x ] = [ b ]
-[  B  μN ] [ y ]   [ c ]

when CE = M⁻¹ and DF = N⁻¹.

By default, GPMR solves unsymmetric linear systems with λ = 1 and μ = 1.

GPMR is based on the orthogonal Hessenberg reduction process and its relations with the block-Arnoldi process. The residual norm ‖rₖ‖ is monotonically decreasing in GPMR.

GPMR stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖r₀‖ * rtol. atol is an absolute tolerance and rtol is a relative tolerance.

Input arguments

Optional arguments

Keyword arguments

Output arguments

Reference

source
Krylov.gpmr! — Function
solver = gpmr!(solver::GpmrSolver, A, B, b, c; kwargs...)
-solver = gpmr!(solver::GpmrSolver, A, B, b, c, x0, y0; kwargs...)

where kwargs are keyword arguments of gpmr.

Note that the memory keyword argument is the only exception. It's required to create a GpmrSolver and can't be changed later.

See GpmrSolver for more details about the solver.

source
+[ B μN ] [ y ] [ c ]

when CE = M⁻¹ and DF = N⁻¹.

By default, GPMR solves unsymmetric linear systems with λ = 1 and μ = 1.

GPMR is based on the orthogonal Hessenberg reduction process and its relations with the block-Arnoldi process. The residual norm ‖rₖ‖ is monotonically decreasing in GPMR.

GPMR stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖r₀‖ * rtol. atol is an absolute tolerance and rtol is a relative tolerance.

Input arguments

Optional arguments

Keyword arguments

Output arguments

Reference

source
Krylov.gpmr! — Function
solver = gpmr!(solver::GpmrSolver, A, B, b, c; kwargs...)
+solver = gpmr!(solver::GpmrSolver, A, B, b, c, x0, y0; kwargs...)

where kwargs are keyword arguments of gpmr.

Note that the memory keyword argument is the only exception. It's required to create a GpmrSolver and can't be changed later.

See GpmrSolver for more details about the solver.

source
diff --git a/previews/PR911/solvers/ln/index.html b/previews/PR911/solvers/ln/index.html index 13a72ac5e..3f361603b 100644 --- a/previews/PR911/solvers/ln/index.html +++ b/previews/PR911/solvers/ln/index.html @@ -4,12 +4,12 @@ λ::T=zero(T), atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0, timemax::Float64=Inf, verbose::Int=0, history::Bool=false, - callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

Solve the consistent linear system

Ax + √λs = b

of size m × n using the Conjugate Gradient (CG) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CG to the normal equations of the second kind

(AAᴴ + λI) y = b

but is more stable. When λ = 0, this method solves the minimum-norm problem

min ‖x‖₂ s.t. Ax = b.

When λ > 0, it solves the problem

min ‖(x,s)‖₂  s.t. Ax + √λs = b.

CGNE produces monotonic errors ‖x-x*‖₂ but not residuals ‖r‖₂. It is formally equivalent to CRAIG, though can be slightly less accurate, but simpler to implement. Only the x-part of the solution is returned.

Input arguments

Keyword arguments

Output arguments

References

source
Krylov.cgne! — Function
solver = cgne!(solver::CgneSolver, A, b; kwargs...)

where kwargs are keyword arguments of cgne.

See CgneSolver for more details about the solver.

source

CRMR

Krylov.crmr — Function
(x, stats) = crmr(A, b::AbstractVector{FC};
+                  callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

Solve the consistent linear system

Ax + √λs = b

of size m × n using the Conjugate Gradient (CG) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CG to the normal equations of the second kind

(AAᴴ + λI) y = b

but is more stable. When λ = 0, this method solves the minimum-norm problem

min ‖x‖₂ s.t. Ax = b.

When λ > 0, it solves the problem

min ‖(x,s)‖₂  s.t. Ax + √λs = b.

CGNE produces monotonic errors ‖x-x*‖₂ but not residuals ‖r‖₂. It is formally equivalent to CRAIG, though can be slightly less accurate, but simpler to implement. Only the x-part of the solution is returned.

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m.

Keyword arguments

  • N: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • λ: regularization parameter;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

References

  • J. E. Craig, The N-step iteration procedures, Journal of Mathematics and Physics, 34(1), pp. 64–73, 1955.
  • J. E. Craig, Iterations Procedures for Simultaneous Equations, Ph.D. Thesis, Department of Electrical Engineering, MIT, 1954.
source
Krylov.cgne! — Function
solver = cgne!(solver::CgneSolver, A, b; kwargs...)

where kwargs are keyword arguments of cgne.

See CgneSolver for more details about the solver.

source

CRMR

Krylov.crmr — Function
(x, stats) = crmr(A, b::AbstractVector{FC};
                   N=I, ldiv::Bool=false,
                   λ::T=zero(T), atol::T=√eps(T),
                   rtol::T=√eps(T), itmax::Int=0,
                   timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
-                  callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

Solve the consistent linear system

Ax + √λs = b

of size m × n using the Conjugate Residual (CR) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CR to the normal equations of the second kind

(AAᴴ + λI) y = b

but is more stable. When λ = 0, this method solves the minimum-norm problem

min ‖x‖₂  s.t.  x ∈ argmin ‖Ax - b‖₂.

When λ > 0, this method solves the problem

min ‖(x,s)‖₂  s.t. Ax + √λs = b.

CRMR produces monotonic residuals ‖r‖₂. It is formally equivalent to CRAIG-MR, though can be slightly less accurate, but simpler to implement. Only the x-part of the solution is returned.

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m.

Keyword arguments

  • N: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • λ: regularization parameter;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

References

source
Krylov.crmr! — Function
solver = crmr!(solver::CrmrSolver, A, b; kwargs...)

where kwargs are keyword arguments of crmr.

See CrmrSolver for more details about the solver.

source

LNLQ

Krylov.lnlq — Function
(x, y, stats) = lnlq(A, b::AbstractVector{FC};
+                  callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

Solve the consistent linear system

Ax + √λs = b

of size m × n using the Conjugate Residual (CR) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CR to the normal equations of the second kind

(AAᴴ + λI) y = b

but is more stable. When λ = 0, this method solves the minimum-norm problem

min ‖x‖₂  s.t.  x ∈ argmin ‖Ax - b‖₂.

When λ > 0, this method solves the problem

min ‖(x,s)‖₂  s.t. Ax + √λs = b.

CRMR produces monotonic residuals ‖r‖₂. It is formally equivalent to CRAIG-MR, though can be slightly less accurate, but simpler to implement. Only the x-part of the solution is returned.

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m.

Keyword arguments

  • N: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • λ: regularization parameter;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

References

source
Krylov.crmr! — Function
solver = crmr!(solver::CrmrSolver, A, b; kwargs...)

where kwargs are keyword arguments of crmr.

See CrmrSolver for more details about the solver.

source

LNLQ

Krylov.lnlq — Function
(x, y, stats) = lnlq(A, b::AbstractVector{FC};
                      M=I, N=I, ldiv::Bool=false,
                      transfer_to_craig::Bool=true,
                      sqd::Bool=false, λ::T=zero(T),
@@ -19,7 +19,7 @@
                      timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
                      callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

Find the least-norm solution of the consistent linear system

Ax + λ²y = b

of size m × n using the LNLQ method, where λ ≥ 0 is a regularization parameter.

For a system in the form Ax = b, LNLQ method is equivalent to applying SYMMLQ to AAᴴy = b and recovering x = Aᴴy but is more stable. Note that y are the Lagrange multipliers of the least-norm problem

minimize ‖x‖  s.t.  Ax = b.

If λ > 0, LNLQ solves the symmetric and quasi-definite system

[ -F    Aᴴ ] [ x ]   [ 0 ]
 [  A  λ²E  ] [ y ] = [ b ],

where E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.

The system above represents the optimality conditions of

min ‖x‖²_F + λ²‖y‖²_E  s.t.  Ax + λ²Ey = b.

For a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LNLQ is then equivalent to applying SYMMLQ to (AF⁻¹Aᴴ + λ²E)y = b with Fx = Aᴴy.

If λ = 0, LNLQ solves the symmetric and indefinite system

[ -F   Aᴴ ] [ x ]   [ 0 ]
-[  A   0  ] [ y ] = [ b ].

The system above represents the optimality conditions of

minimize ‖x‖²_F  s.t.  Ax = b.

In this case, M can still be specified and indicates the weighted norm in which residuals are measured.

In this implementation, both the x and y-parts of the solution are returned.

utolx and utoly are tolerances on the upper bound of the distance to the solution ‖x-x‖ and ‖y-y‖, respectively. The bound is valid if λ>0 or σ>0 where σ should be strictly smaller than the smallest positive singular value. For instance σ:=(1-1e-7)σₘᵢₙ .

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
  • N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • transfer_to_craig: transfer from the LNLQ point to the CRAIG point, when it exists. The transfer is based on the residual norm;
  • sqd: if true, set λ=1 for Hermitian quasi-definite systems;
  • λ: regularization parameter;
  • σ: strict lower bound on the smallest positive singular value σₘᵢₙ such as σ = (1-10⁻⁷)σₘᵢₙ;
  • utolx: tolerance on the upper bound on the distance to the solution ‖x-x*‖;
  • utoly: tolerance on the upper bound on the distance to the solution ‖y-y*‖;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • y: a dense vector of length m;
  • stats: statistics collected on the run in a LNLQStats structure.

Reference

source
Krylov.lnlq! — Function
solver = lnlq!(solver::LnlqSolver, A, b; kwargs...)

where kwargs are keyword arguments of lnlq.

See LnlqSolver for more details about the solver.

source

CRAIG

Krylov.craig — Function
(x, y, stats) = craig(A, b::AbstractVector{FC};
+[  A   0  ] [ y ] = [ b ].

The system above represents the optimality conditions of

minimize ‖x‖²_F  s.t.  Ax = b.

In this case, M can still be specified and indicates the weighted norm in which residuals are measured.

In this implementation, both the x and y-parts of the solution are returned.

utolx and utoly are tolerances on the upper bound of the distance to the solution ‖x-x‖ and ‖y-y‖, respectively. The bound is valid if λ>0 or σ>0 where σ should be strictly smaller than the smallest positive singular value. For instance σ:=(1-1e-7)σₘᵢₙ .

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
  • N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • transfer_to_craig: transfer from the LNLQ point to the CRAIG point, when it exists. The transfer is based on the residual norm;
  • sqd: if true, set λ=1 for Hermitian quasi-definite systems;
  • λ: regularization parameter;
  • σ: strict lower bound on the smallest positive singular value σₘᵢₙ such as σ = (1-10⁻⁷)σₘᵢₙ;
  • utolx: tolerance on the upper bound on the distance to the solution ‖x-x*‖;
  • utoly: tolerance on the upper bound on the distance to the solution ‖y-y*‖;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • y: a dense vector of length m;
  • stats: statistics collected on the run in a LNLQStats structure.

Reference

source
Krylov.lnlq! — Function
solver = lnlq!(solver::LnlqSolver, A, b; kwargs...)

where kwargs are keyword arguments of lnlq.

See LnlqSolver for more details about the solver.

source

CRAIG

Krylov.craig — Function
(x, y, stats) = craig(A, b::AbstractVector{FC};
                       M=I, N=I, ldiv::Bool=false,
                       transfer_to_lsqr::Bool=false, sqd::Bool=false,
                       λ::T=zero(T), btol::T=√eps(T),
@@ -28,16 +28,16 @@
                       timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
                       callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

Find the least-norm solution of the consistent linear system

Ax + λ²y = b

of size m × n using the Golub-Kahan implementation of Craig's method, where λ ≥ 0 is a regularization parameter. This method is equivalent to CGNE but is more stable.

For a system in the form Ax = b, Craig's method is equivalent to applying CG to AAᴴy = b and recovering x = Aᴴy. Note that y are the Lagrange multipliers of the least-norm problem

minimize ‖x‖  s.t.  Ax = b.

If λ > 0, CRAIG solves the symmetric and quasi-definite system

[ -F     Aᴴ ] [ x ]   [ 0 ]
 [  A   λ²E  ] [ y ] = [ b ],

where E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.

The system above represents the optimality conditions of

min ‖x‖²_F + λ²‖y‖²_E  s.t.  Ax + λ²Ey = b.

For a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. CRAIG is then equivalent to applying CG to (AF⁻¹Aᴴ + λ²E)y = b with Fx = Aᴴy.

If λ = 0, CRAIG solves the symmetric and indefinite system

[ -F   Aᴴ ] [ x ]   [ 0 ]
-[  A   0  ] [ y ] = [ b ].

The system above represents the optimality conditions of

minimize ‖x‖²_F  s.t.  Ax = b.

In this case, M can still be specified and indicates the weighted norm in which residuals are measured.

In this implementation, both the x and y-parts of the solution are returned.

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
  • N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • transfer_to_lsqr: transfer from the LSLQ point to the LSQR point, when it exists. The transfer is based on the residual norm;
  • sqd: if true, set λ=1 for Hermitian quasi-definite systems;
  • λ: regularization parameter;
  • btol: stopping tolerance used to detect zero-residual problems;
  • conlim: limit on the estimated condition number of A beyond which the solution will be abandoned;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • y: a dense vector of length m;
  • stats: statistics collected on the run in a SimpleStats structure.

References

source
Krylov.craig! — Function
solver = craig!(solver::CraigSolver, A, b; kwargs...)

where kwargs are keyword arguments of craig.

See CraigSolver for more details about the solver.

source

CRAIGMR

Krylov.craigmr — Function
(x, y, stats) = craigmr(A, b::AbstractVector{FC};
+[  A   0  ] [ y ] = [ b ].

The system above represents the optimality conditions of

minimize ‖x‖²_F  s.t.  Ax = b.

In this case, M can still be specified and indicates the weighted norm in which residuals are measured.

In this implementation, both the x and y-parts of the solution are returned.

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
  • N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • transfer_to_lsqr: transfer from the LSLQ point to the LSQR point, when it exists. The transfer is based on the residual norm;
  • sqd: if true, set λ=1 for Hermitian quasi-definite systems;
  • λ: regularization parameter;
  • btol: stopping tolerance used to detect zero-residual problems;
  • conlim: limit on the estimated condition number of A beyond which the solution will be abandoned;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • y: a dense vector of length m;
  • stats: statistics collected on the run in a SimpleStats structure.

References

source
Krylov.craig! — Function
solver = craig!(solver::CraigSolver, A, b; kwargs...)

where kwargs are keyword arguments of craig.

See CraigSolver for more details about the solver.

source

CRAIGMR

Krylov.craigmr — Function
(x, y, stats) = craigmr(A, b::AbstractVector{FC};
                         M=I, N=I, ldiv::Bool=false,
                         sqd::Bool=false, λ::T=zero(T), atol::T=√eps(T),
                         rtol::T=√eps(T), itmax::Int=0,
                         timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
                         callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

Solve the consistent linear system

Ax + λ²y = b

of size m × n using the CRAIGMR method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying the Conjugate Residuals method to the normal equations of the second kind

(AAᴴ + λ²I) y = b

but is more stable. When λ = 0, this method solves the minimum-norm problem

min ‖x‖  s.t.  x ∈ argmin ‖Ax - b‖.

If λ > 0, CRAIGMR solves the symmetric and quasi-definite system

[ -F    Aᴴ ] [ x ]   [ 0 ]
 [  A  λ²E  ] [ y ] = [ b ],

where E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.

The system above represents the optimality conditions of

min ‖x‖²_F + λ²‖y‖²_E  s.t.  Ax + λ²Ey = b.

For a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. CRAIGMR is then equivalent to applying MINRES to (AF⁻¹Aᴴ + λ²E)y = b with Fx = Aᴴy.

If λ = 0, CRAIGMR solves the symmetric and indefinite system

[ -F   Aᴴ ] [ x ]   [ 0 ]
-[  A   0  ] [ y ] = [ b ].

The system above represents the optimality conditions of

min ‖x‖²_F  s.t.  Ax = b.

In this case, M can still be specified and indicates the weighted norm in which residuals are measured.

CRAIGMR produces monotonic residuals ‖r‖₂. It is formally equivalent to CRMR, though can be slightly more accurate, and intricate to implement. Both the x- and y-parts of the solution are returned.

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
  • N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • sqd: if true, set λ=1 for Hermitian quasi-definite systems;
  • λ: regularization parameter;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • y: a dense vector of length m;
  • stats: statistics collected on the run in a SimpleStats structure.

References

source
Krylov.craigmr! — Function
solver = craigmr!(solver::CraigmrSolver, A, b; kwargs...)

where kwargs are keyword arguments of craigmr.

See CraigmrSolver for more details about the solver.

source

USYMLQ

Krylov.usymlq — Function
(x, stats) = usymlq(A, b::AbstractVector{FC}, c::AbstractVector{FC};
+[  A   0  ] [ y ] = [ b ].

The system above represents the optimality conditions of

min ‖x‖²_F  s.t.  Ax = b.

In this case, M can still be specified and indicates the weighted norm in which residuals are measured.

CRAIGMR produces monotonic residuals ‖r‖₂. It is formally equivalent to CRMR, though can be slightly more accurate, and intricate to implement. Both the x- and y-parts of the solution are returned.

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
  • N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • sqd: if true, set λ=1 for Hermitian quasi-definite systems;
  • λ: regularization parameter;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • y: a dense vector of length m;
  • stats: statistics collected on the run in a SimpleStats structure.

References

source
Krylov.craigmr! — Function
solver = craigmr!(solver::CraigmrSolver, A, b; kwargs...)

where kwargs are keyword arguments of craigmr.

See CraigmrSolver for more details about the solver.

source

USYMLQ

Krylov.usymlq — Function
(x, stats) = usymlq(A, b::AbstractVector{FC}, c::AbstractVector{FC};
                     transfer_to_usymcg::Bool=true, atol::T=√eps(T),
                     rtol::T=√eps(T), itmax::Int=0,
                     timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
-                    callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = usymlq(A, b, c, x0::AbstractVector; kwargs...)

USYMLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

USYMLQ determines the least-norm solution of the consistent linear system Ax = b of size m × n.

USYMLQ is based on the orthogonal tridiagonalization process and requires two initial nonzero vectors b and c. The vector c is only used to initialize the process and a default value can be b or Aᴴb depending on the shape of A. The error norm ‖x - x*‖ monotonously decreases in USYMLQ. When A is Hermitian and b = c, USYMLQ is equivalent to SYMMLQ. USYMLQ is considered as a generalization of SYMMLQ.

It can also be applied to under-determined and over-determined problems. In all cases, problems must be consistent.

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m;
  • c: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • transfer_to_usymcg: transfer from the USYMLQ point to the USYMCG point, when it exists. The transfer is based on the residual norm;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

References

source
Krylov.usymlq! — Function
solver = usymlq!(solver::UsymlqSolver, A, b, c; kwargs...)
-solver = usymlq!(solver::UsymlqSolver, A, b, c, x0; kwargs...)

where kwargs are keyword arguments of usymlq.

See UsymlqSolver for more details about the solver.

source
+ callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = usymlq(A, b, c, x0::AbstractVector; kwargs...)

USYMLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

USYMLQ determines the least-norm solution of the consistent linear system Ax = b of size m × n.

USYMLQ is based on the orthogonal tridiagonalization process and requires two initial nonzero vectors b and c. The vector c is only used to initialize the process and a default value can be b or Aᴴb depending on the shape of A. The error norm ‖x - x*‖ monotonously decreases in USYMLQ. When A is Hermitian and b = c, USYMLQ is equivalent to SYMMLQ. USYMLQ is considered as a generalization of SYMMLQ.

It can also be applied to under-determined and over-determined problems. In all cases, problems must be consistent.

Input arguments

Optional argument

Keyword arguments

Output arguments

References

source
Krylov.usymlq! — Function
solver = usymlq!(solver::UsymlqSolver, A, b, c; kwargs...)
+solver = usymlq!(solver::UsymlqSolver, A, b, c, x0; kwargs...)

where kwargs are keyword arguments of usymlq.

See UsymlqSolver for more details about the solver.

source
diff --git a/previews/PR911/solvers/ls/index.html b/previews/PR911/solvers/ls/index.html index 76c94705b..2c490abc2 100644 --- a/previews/PR911/solvers/ls/index.html +++ b/previews/PR911/solvers/ls/index.html @@ -4,15 +4,15 @@ λ::T=zero(T), atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0, timemax::Float64=Inf, verbose::Int=0, history::Bool=false, - callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

Solve the regularized linear least-squares problem

minimize ‖b - Ax‖₂² + λ‖x‖₂²

of size m × n using the Conjugate Gradient (CG) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CG to the normal equations

(AᴴA + λI) x = Aᴴb

but is more stable.

CGLS produces monotonic residuals ‖r‖₂ but not optimality residuals ‖Aᴴr‖₂. It is formally equivalent to LSQR, though can be slightly less accurate, but simpler to implement.

Input arguments

Keyword arguments

Output arguments

References

source
Krylov.cgls! — Function
solver = cgls!(solver::CglsSolver, A, b; kwargs...)

where kwargs are keyword arguments of cgls.

See CglsSolver for more details about the solver.

source

CGLS-LANCZOS-SHIFT

Krylov.cgls_lanczos_shift — Function
(x, stats) = cgls_lanczos_shift(A, b::AbstractVector{FC};
+                  callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

Solve the regularized linear least-squares problem

minimize ‖b - Ax‖₂² + λ‖x‖₂²

of size m × n using the Conjugate Gradient (CG) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CG to the normal equations

(AᴴA + λI) x = Aᴴb

but is more stable.

CGLS produces monotonic residuals ‖r‖₂ but not optimality residuals ‖Aᴴr‖₂. It is formally equivalent to LSQR, though can be slightly less accurate, but simpler to implement.

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
  • λ: regularization parameter;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

References

source
Krylov.cgls! — Function
solver = cgls!(solver::CglsSolver, A, b; kwargs...)

where kwargs are keyword arguments of cgls.

See CglsSolver for more details about the solver.

source

CGLS-LANCZOS-SHIFT

Krylov.cgls_lanczos_shift — Function
(x, stats) = cgls_lanczos_shift(A, b::AbstractVector{FC};
                   M=I, λ::T=zero(T), atol::T=√eps(T), rtol::T=√eps(T),
                   radius::T=zero(T), itmax::Int=0, verbose::Int=0, history::Bool=false,
-                  callback=solver->false)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

Solve the regularized linear least-squares problem

minimize ‖b - Ax‖₂² + λ‖x‖₂²

using the Conjugate Gradient (CG) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CG to the normal equations

(AᴴA + λI) x = Aᴴb

but is more stable.

CGLS produces monotonic residuals ‖r‖₂ but not optimality residuals ‖Aᴴr‖₂. It is formally equivalent to LSQR, though can be slightly less accurate, but simpler to implement.

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m;
  • shifts: a vector of length p.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a vector of p dense vectors, each one of length n;
  • stats: statistics collected on the run in a LanczosShiftStats structure.

References

source
Krylov.cgls_lanczos_shift! — Function
solver = cgls_lanczos_shift!(solver::CglsLanczosShiftSolver, A, b, shifts; kwargs...)

where kwargs are keyword arguments of cgls_lanczos_shift.

See CglsLanczosShiftSolver for more details about the solver.

source

CRLS

Krylov.crls — Function
(x, stats) = crls(A, b::AbstractVector{FC};
+                  callback=solver->false)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

Solve the regularized linear least-squares problem

minimize ‖b - Ax‖₂² + λ‖x‖₂²

using the Conjugate Gradient (CG) method, where λ ≥ 0 is a regularization parameter. This method is equivalent to applying CG to the normal equations

(AᴴA + λI) x = Aᴴb

but is more stable.

CGLS produces monotonic residuals ‖r‖₂ but not optimality residuals ‖Aᴴr‖₂. It is formally equivalent to LSQR, though can be slightly less accurate, but simpler to implement.

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m;
  • shifts: a vector of length p.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a vector of p dense vectors, each one of length n;
  • stats: statistics collected on the run in a LanczosShiftStats structure.

References

source
Krylov.cgls_lanczos_shift! — Function
solver = cgls_lanczos_shift!(solver::CglsLanczosShiftSolver, A, b, shifts; kwargs...)

where kwargs are keyword arguments of cgls_lanczos_shift.

See CglsLanczosShiftSolver for more details about the solver.

source

CRLS

Krylov.crls — Function
(x, stats) = crls(A, b::AbstractVector{FC};
                   M=I, ldiv::Bool=false, radius::T=zero(T),
                   λ::T=zero(T), atol::T=√eps(T), rtol::T=√eps(T),
                   itmax::Int=0, timemax::Float64=Inf,
                   verbose::Int=0, history::Bool=false,
-                  callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

Solve the linear least-squares problem

minimize ‖b - Ax‖₂² + λ‖x‖₂²

of size m × n using the Conjugate Residuals (CR) method. This method is equivalent to applying MINRES to the normal equations

(AᴴA + λI) x = Aᴴb.

This implementation recurs the residual r := b - Ax.

CRLS produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂. It is formally equivalent to LSMR, though can be substantially less accurate, but simpler to implement.

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
  • λ: regularization parameter;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

  • D. C.-L. Fong, Minimum-Residual Methods for Sparse, Least-Squares using Golubg-Kahan Bidiagonalization, Ph.D. Thesis, Stanford University, 2011.
source
Krylov.crls! — Function
solver = crls!(solver::CrlsSolver, A, b; kwargs...)

where kwargs are keyword arguments of crls.

See CrlsSolver for more details about the solver.

source

LSLQ

Krylov.lslq — Function
(x, stats) = lslq(A, b::AbstractVector{FC};
+                  callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

Solve the linear least-squares problem

minimize ‖b - Ax‖₂² + λ‖x‖₂²

of size m × n using the Conjugate Residuals (CR) method. This method is equivalent to applying MINRES to the normal equations

(AᴴA + λI) x = Aᴴb.

This implementation recurs the residual r := b - Ax.

CRLS produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂. It is formally equivalent to LSMR, though can be substantially less accurate, but simpler to implement.

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size n used for preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
  • λ: regularization parameter;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

  • D. C.-L. Fong, Minimum-Residual Methods for Sparse, Least-Squares using Golubg-Kahan Bidiagonalization, Ph.D. Thesis, Stanford University, 2011.
source
Krylov.crls! — Function
solver = crls!(solver::CrlsSolver, A, b; kwargs...)

where kwargs are keyword arguments of crls.

See CrlsSolver for more details about the solver.

source

LSLQ

Krylov.lslq — Function
(x, stats) = lslq(A, b::AbstractVector{FC};
                   M=I, N=I, ldiv::Bool=false,
                   window::Int=5, transfer_to_lsqr::Bool=false,
                   sqd::Bool=false, λ::T=zero(T),
@@ -23,7 +23,7 @@
                   timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
                   callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

Solve the regularized linear least-squares problem

minimize ‖b - Ax‖₂² + λ²‖x‖₂²

of size m × n using the LSLQ method, where λ ≥ 0 is a regularization parameter. LSLQ is formally equivalent to applying SYMMLQ to the normal equations

(AᴴA + λ²I) x = Aᴴb

but is more stable.

If λ > 0, we solve the symmetric and quasi-definite system

[ E      A ] [ r ]   [ b ]
 [ Aᴴ  -λ²F ] [ x ] = [ 0 ],

where E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.

The system above represents the optimality conditions of

minimize ‖b - Ax‖²_E⁻¹ + λ²‖x‖²_F.

For a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LSLQ is then equivalent to applying SYMMLQ to (AᴴE⁻¹A + λ²F)x = AᴴE⁻¹b with r = E⁻¹(b - Ax).

If λ = 0, we solve the symmetric and indefinite system

[ E    A ] [ r ]   [ b ]
-[ Aᴴ   0 ] [ x ] = [ 0 ].

The system above represents the optimality conditions of

minimize ‖b - Ax‖²_E⁻¹.

In this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).

Main features

  • the solution estimate is updated along orthogonal directions
  • the norm of the solution estimate ‖xᴸₖ‖₂ is increasing
  • the error ‖eₖ‖₂ := ‖xᴸₖ - x*‖₂ is decreasing
  • it is possible to transition cheaply from the LSLQ iterate to the LSQR iterate if there is an advantage (there always is in terms of error)
  • if A is rank deficient, identify the minimum least-squares solution

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
  • N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • window: number of iterations used to accumulate a lower bound on the error;
  • transfer_to_lsqr: transfer from the LSLQ point to the LSQR point, when it exists. The transfer is based on the residual norm;
  • sqd: if true, set λ=1 for Hermitian quasi-definite systems;
  • λ: regularization parameter;
  • σ: strict lower bound on the smallest positive singular value σₘᵢₙ such as σ = (1-10⁻⁷)σₘᵢₙ;
  • etol: stopping tolerance based on the lower bound on the error;
  • utol: stopping tolerance based on the upper bound on the error;
  • btol: stopping tolerance used to detect zero-residual problems;
  • conlim: limit on the estimated condition number of A beyond which the solution will be abandoned;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;

  • stats: statistics collected on the run in a LSLQStats structure.

  • stats.err_lbnds is a vector of lower bounds on the LQ error–-the vector is empty if window is set to zero

  • stats.err_ubnds_lq is a vector of upper bounds on the LQ error–-the vector is empty if σ == 0 is left at zero

  • stats.err_ubnds_cg is a vector of upper bounds on the CG error–-the vector is empty if σ == 0 is left at zero

  • stats.error_with_bnd is a boolean indicating whether there was an error in the upper bounds computation (cancellation errors, too large σ ...)

Stopping conditions

The iterations stop as soon as one of the following conditions holds true:

  • the optimality residual is sufficiently small (stats.status = "found approximate minimum least-squares solution") in the sense that either
    • ‖Aᴴr‖ / (‖A‖ ‖r‖) ≤ atol, or
    • 1 + ‖Aᴴr‖ / (‖A‖ ‖r‖) ≤ 1
  • an approximate zero-residual solution has been found (stats.status = "found approximate zero-residual solution") in the sense that either
    • ‖r‖ / ‖b‖ ≤ btol + atol ‖A‖ * ‖xᴸ‖ / ‖b‖, or
    • 1 + ‖r‖ / ‖b‖ ≤ 1
  • the estimated condition number of A is too large in the sense that either
    • 1/cond(A) ≤ 1/conlim (stats.status = "condition number exceeds tolerance"), or
    • 1 + 1/cond(A) ≤ 1 (stats.status = "condition number seems too large for this machine")
  • the lower bound on the LQ forward error is less than etol * ‖xᴸ‖
  • the upper bound on the CG forward error is less than utol * ‖xᶜ‖

References

source
Krylov.lslq! — Function
solver = lslq!(solver::LslqSolver, A, b; kwargs...)

where kwargs are keyword arguments of lslq.

See LslqSolver for more details about the solver.

source

LSQR

Krylov.lsqr — Function
(x, stats) = lsqr(A, b::AbstractVector{FC};
+[ Aᴴ   0 ] [ x ] = [ 0 ].

The system above represents the optimality conditions of

minimize ‖b - Ax‖²_E⁻¹.

In this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).

Main features

  • the solution estimate is updated along orthogonal directions
  • the norm of the solution estimate ‖xᴸₖ‖₂ is increasing
  • the error ‖eₖ‖₂ := ‖xᴸₖ - x*‖₂ is decreasing
  • it is possible to transition cheaply from the LSLQ iterate to the LSQR iterate if there is an advantage (there always is in terms of error)
  • if A is rank deficient, identify the minimum least-squares solution

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
  • N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • window: number of iterations used to accumulate a lower bound on the error;
  • transfer_to_lsqr: transfer from the LSLQ point to the LSQR point, when it exists. The transfer is based on the residual norm;
  • sqd: if true, set λ=1 for Hermitian quasi-definite systems;
  • λ: regularization parameter;
  • σ: strict lower bound on the smallest positive singular value σₘᵢₙ such as σ = (1-10⁻⁷)σₘᵢₙ;
  • etol: stopping tolerance based on the lower bound on the error;
  • utol: stopping tolerance based on the upper bound on the error;
  • btol: stopping tolerance used to detect zero-residual problems;
  • conlim: limit on the estimated condition number of A beyond which the solution will be abandoned;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;

  • stats: statistics collected on the run in a LSLQStats structure.

  • stats.err_lbnds is a vector of lower bounds on the LQ error–-the vector is empty if window is set to zero

  • stats.err_ubnds_lq is a vector of upper bounds on the LQ error–-the vector is empty if σ == 0 is left at zero

  • stats.err_ubnds_cg is a vector of upper bounds on the CG error–-the vector is empty if σ == 0 is left at zero

  • stats.error_with_bnd is a boolean indicating whether there was an error in the upper bounds computation (cancellation errors, too large σ ...)

Stopping conditions

The iterations stop as soon as one of the following conditions holds true:

  • the optimality residual is sufficiently small (stats.status = "found approximate minimum least-squares solution") in the sense that either
    • ‖Aᴴr‖ / (‖A‖ ‖r‖) ≤ atol, or
    • 1 + ‖Aᴴr‖ / (‖A‖ ‖r‖) ≤ 1
  • an approximate zero-residual solution has been found (stats.status = "found approximate zero-residual solution") in the sense that either
    • ‖r‖ / ‖b‖ ≤ btol + atol ‖A‖ * ‖xᴸ‖ / ‖b‖, or
    • 1 + ‖r‖ / ‖b‖ ≤ 1
  • the estimated condition number of A is too large in the sense that either
    • 1/cond(A) ≤ 1/conlim (stats.status = "condition number exceeds tolerance"), or
    • 1 + 1/cond(A) ≤ 1 (stats.status = "condition number seems too large for this machine")
  • the lower bound on the LQ forward error is less than etol * ‖xᴸ‖
  • the upper bound on the CG forward error is less than utol * ‖xᶜ‖

References

source
Krylov.lslq! — Function
solver = lslq!(solver::LslqSolver, A, b; kwargs...)

where kwargs are keyword arguments of lslq.

See LslqSolver for more details about the solver.

source

LSQR

Krylov.lsqr — Function
(x, stats) = lsqr(A, b::AbstractVector{FC};
                   M=I, N=I, ldiv::Bool=false,
                   window::Int=5, sqd::Bool=false, λ::T=zero(T),
                   radius::T=zero(T), etol::T=√eps(T),
@@ -33,7 +33,7 @@
                   timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
                   callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

Solve the regularized linear least-squares problem

minimize ‖b - Ax‖₂² + λ²‖x‖₂²

of size m × n using the LSQR method, where λ ≥ 0 is a regularization parameter. LSQR is formally equivalent to applying CG to the normal equations

(AᴴA + λ²I) x = Aᴴb

(and therefore to CGLS) but is more stable.

LSQR produces monotonic residuals ‖r‖₂ but not optimality residuals ‖Aᴴr‖₂. It is formally equivalent to CGLS, though can be slightly more accurate.

If λ > 0, LSQR solves the symmetric and quasi-definite system

[ E      A ] [ r ]   [ b ]
 [ Aᴴ  -λ²F ] [ x ] = [ 0 ],

where E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.

The system above represents the optimality conditions of

minimize ‖b - Ax‖²_E⁻¹ + λ²‖x‖²_F.

For a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LSQR is then equivalent to applying CG to (AᴴE⁻¹A + λ²F)x = AᴴE⁻¹b with r = E⁻¹(b - Ax).

If λ = 0, we solve the symmetric and indefinite system

[ E    A ] [ r ]   [ b ]
-[ Aᴴ   0 ] [ x ] = [ 0 ].

The system above represents the optimality conditions of

minimize ‖b - Ax‖²_E⁻¹.

In this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
  • N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • window: number of iterations used to accumulate a lower bound on the error;
  • sqd: if true, set λ=1 for Hermitian quasi-definite systems;
  • λ: regularization parameter;
  • radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
  • etol: stopping tolerance based on the lower bound on the error;
  • axtol: tolerance on the backward error;
  • btol: stopping tolerance used to detect zero-residual problems;
  • conlim: limit on the estimated condition number of A beyond which the solution will be abandoned;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.lsqr! — Function
solver = lsqr!(solver::LsqrSolver, A, b; kwargs...)

where kwargs are keyword arguments of lsqr.

See LsqrSolver for more details about the solver.

source

LSMR

Krylov.lsmr — Function
(x, stats) = lsmr(A, b::AbstractVector{FC};
+[ Aᴴ   0 ] [ x ] = [ 0 ].

The system above represents the optimality conditions of

minimize ‖b - Ax‖²_E⁻¹.

In this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
  • N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • window: number of iterations used to accumulate a lower bound on the error;
  • sqd: if true, set λ=1 for Hermitian quasi-definite systems;
  • λ: regularization parameter;
  • radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
  • etol: stopping tolerance based on the lower bound on the error;
  • axtol: tolerance on the backward error;
  • btol: stopping tolerance used to detect zero-residual problems;
  • conlim: limit on the estimated condition number of A beyond which the solution will be abandoned;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.lsqr! — Function
solver = lsqr!(solver::LsqrSolver, A, b; kwargs...)

where kwargs are keyword arguments of lsqr.

See LsqrSolver for more details about the solver.

source

LSMR

Krylov.lsmr — Function
(x, stats) = lsmr(A, b::AbstractVector{FC};
                   M=I, N=I, ldiv::Bool=false,
                   window::Int=5, sqd::Bool=false, λ::T=zero(T),
                   radius::T=zero(T), etol::T=√eps(T),
@@ -43,8 +43,8 @@
                   timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
                   callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

Solve the regularized linear least-squares problem

minimize ‖b - Ax‖₂² + λ²‖x‖₂²

of size m × n using the LSMR method, where λ ≥ 0 is a regularization parameter. LSMR is formally equivalent to applying MINRES to the normal equations

(AᴴA + λ²I) x = Aᴴb

(and therefore to CRLS) but is more stable.

LSMR produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂. It is formally equivalent to CRLS, though can be substantially more accurate.

LSMR can be also used to find a null vector of a singular matrix A by solving the problem min ‖Aᴴx - b‖ with any nonzero vector b. At a minimizer, the residual vector r = b - Aᴴx will satisfy Ar = 0.

If λ > 0, we solve the symmetric and quasi-definite system

[ E      A ] [ r ]   [ b ]
 [ Aᴴ  -λ²F ] [ x ] = [ 0 ],

where E and F are symmetric and positive definite. Preconditioners M = E⁻¹ ≻ 0 and N = F⁻¹ ≻ 0 may be provided in the form of linear operators. If sqd=true, λ is set to the common value 1.

The system above represents the optimality conditions of

minimize ‖b - Ax‖²_E⁻¹ + λ²‖x‖²_F.

For a symmetric and positive definite matrix K, the K-norm of a vector x is ‖x‖²_K = xᴴKx. LSMR is then equivalent to applying MINRES to (AᴴE⁻¹A + λ²F)x = AᴴE⁻¹b with r = E⁻¹(b - Ax).

If λ = 0, we solve the symmetric and indefinite system

[ E    A ] [ r ]   [ b ]
-[ Aᴴ   0 ] [ x ] = [ 0 ].

The system above represents the optimality conditions of

minimize ‖b - Ax‖²_E⁻¹.

In this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
  • N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • window: number of iterations used to accumulate a lower bound on the error;
  • sqd: if true, set λ=1 for Hermitian quasi-definite systems;
  • λ: regularization parameter;
  • radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
  • etol: stopping tolerance based on the lower bound on the error;
  • axtol: tolerance on the backward error;
  • btol: stopping tolerance used to detect zero-residual problems;
  • conlim: limit on the estimated condition number of A beyond which the solution will be abandoned;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a LsmrStats structure.

Reference

source
Krylov.lsmr! — Function
solver = lsmr!(solver::LsmrSolver, A, b; kwargs...)

where kwargs are keyword arguments of lsmr.

See LsmrSolver for more details about the solver.

source

USYMQR

Krylov.usymqr — Function
(x, stats) = usymqr(A, b::AbstractVector{FC}, c::AbstractVector{FC};
+[ Aᴴ   0 ] [ x ] = [ 0 ].

The system above represents the optimality conditions of

minimize ‖b - Ax‖²_E⁻¹.

In this case, N can still be specified and indicates the weighted norm in which x and Aᴴr should be measured. r can be recovered by computing E⁻¹(b - Ax).

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the augmented system;
  • N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the augmented system;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • window: number of iterations used to accumulate a lower bound on the error;
  • sqd: if true, set λ=1 for Hermitian quasi-definite systems;
  • λ: regularization parameter;
  • radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
  • etol: stopping tolerance based on the lower bound on the error;
  • axtol: tolerance on the backward error;
  • btol: stopping tolerance used to detect zero-residual problems;
  • conlim: limit on the estimated condition number of A beyond which the solution will be abandoned;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a LsmrStats structure.

Reference

source
Krylov.lsmr! — Function
solver = lsmr!(solver::LsmrSolver, A, b; kwargs...)

where kwargs are keyword arguments of lsmr.

See LsmrSolver for more details about the solver.

source

USYMQR

Krylov.usymqr — Function
(x, stats) = usymqr(A, b::AbstractVector{FC}, c::AbstractVector{FC};
                     atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,
                     timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
-                    callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = usymqr(A, b, c, x0::AbstractVector; kwargs...)

USYMQR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

USYMQR solves the linear least-squares problem min ‖b - Ax‖² of size m × n. USYMQR solves Ax = b if it is consistent.

USYMQR is based on the orthogonal tridiagonalization process and requires two initial nonzero vectors b and c. The vector c is only used to initialize the process and a default value can be b or Aᴴb depending on the shape of A. The residual norm ‖b - Ax‖ monotonously decreases in USYMQR. When A is Hermitian and b = c, QMR is equivalent to MINRES. USYMQR is considered as a generalization of MINRES.

It can also be applied to under-determined and over-determined problems. USYMQR finds the minimum-norm solution if problems are inconsistent.

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m;
  • c: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

References

source
Krylov.usymqr! — Function
solver = usymqr!(solver::UsymqrSolver, A, b, c; kwargs...)
-solver = usymqr!(solver::UsymqrSolver, A, b, c, x0; kwargs...)

where kwargs are keyword arguments of usymqr.

See UsymqrSolver for more details about the solver.

source
+ callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = usymqr(A, b, c, x0::AbstractVector; kwargs...)

USYMQR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

USYMQR solves the linear least-squares problem min ‖b - Ax‖² of size m × n. USYMQR solves Ax = b if it is consistent.

USYMQR is based on the orthogonal tridiagonalization process and requires two initial nonzero vectors b and c. The vector c is only used to initialize the process and a default value can be b or Aᴴb depending on the shape of A. The residual norm ‖b - Ax‖ monotonously decreases in USYMQR. When A is Hermitian and b = c, QMR is equivalent to MINRES. USYMQR is considered as a generalization of MINRES.

It can also be applied to under-determined and over-determined problems. USYMQR finds the minimum-norm solution if problems are inconsistent.

Input arguments

Optional argument

Keyword arguments

Output arguments

References

source
Krylov.usymqr! — Function
solver = usymqr!(solver::UsymqrSolver, A, b, c; kwargs...)
+solver = usymqr!(solver::UsymqrSolver, A, b, c, x0; kwargs...)

where kwargs are keyword arguments of usymqr.

See UsymqrSolver for more details about the solver.

source
diff --git a/previews/PR911/solvers/sid/index.html b/previews/PR911/solvers/sid/index.html index f77fa37b2..a0a6d0ca3 100644 --- a/previews/PR911/solvers/sid/index.html +++ b/previews/PR911/solvers/sid/index.html @@ -6,25 +6,25 @@ conlim::T=1/√eps(T), atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0, timemax::Float64=Inf, verbose::Int=0, history::Bool=false, - callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = symmlq(A, b, x0::AbstractVector; kwargs...)

SYMMLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above

Solve the shifted linear system

(A + λI) x = b

of size n using the SYMMLQ method, where λ is a shift parameter, and A is Hermitian.

SYMMLQ produces monotonic errors ‖x* - x‖₂.

Input arguments

Optional argument

Keyword arguments

Output arguments

Reference

source
Krylov.symmlq! — Function
solver = symmlq!(solver::SymmlqSolver, A, b; kwargs...)
-solver = symmlq!(solver::SymmlqSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of symmlq.

See SymmlqSolver for more details about the solver.

source

MINRES

Krylov.minres — Function
(x, stats) = minres(A, b::AbstractVector{FC};
+                    callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = symmlq(A, b, x0::AbstractVector; kwargs...)

SYMMLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above

Solve the shifted linear system

(A + λI) x = b

of size n using the SYMMLQ method, where λ is a shift parameter, and A is Hermitian.

SYMMLQ produces monotonic errors ‖x* - x‖₂.

Input arguments

  • A: a linear operator that models a Hermitian matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • window: number of iterations used to accumulate a lower bound on the error;
  • transfer_to_cg: transfer from the SYMMLQ point to the CG point, when it exists. The transfer is based on the residual norm;
  • λ: regularization parameter;
  • λest: positive strict lower bound on the smallest eigenvalue λₘᵢₙ when solving a positive-definite system, such as λest = (1-10⁻⁷)λₘᵢₙ;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • etol: stopping tolerance based on the lower bound on the error;
  • conlim: limit on the estimated condition number of A beyond which the solution will be abandoned;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SymmlqStats structure.

Reference

source
Krylov.symmlq! — Function
solver = symmlq!(solver::SymmlqSolver, A, b; kwargs...)
+solver = symmlq!(solver::SymmlqSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of symmlq.

See SymmlqSolver for more details about the solver.

source

MINRES

Krylov.minres — Function
(x, stats) = minres(A, b::AbstractVector{FC};
                     M=I, ldiv::Bool=false, window::Int=5,
                     λ::T=zero(T), atol::T=√eps(T),
                     rtol::T=√eps(T), etol::T=√eps(T),
                     conlim::T=1/√eps(T), itmax::Int=0,
                     timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
-                    callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = minres(A, b, x0::AbstractVector; kwargs...)

MINRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

Solve the shifted linear least-squares problem

minimize ‖b - (A + λI)x‖₂²

or the shifted linear system

(A + λI) x = b

of size n using the MINRES method, where λ ≥ 0 is a shift parameter, where A is Hermitian.

MINRES is formally equivalent to applying CR to Ax=b when A is positive definite, but is typically more stable and also applies to the case where A is indefinite.

MINRES produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂.

Input arguments

  • A: a linear operator that models a Hermitian matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • window: number of iterations used to accumulate a lower bound on the error;
  • λ: regularization parameter;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • etol: stopping tolerance based on the lower bound on the error;
  • conlim: limit on the estimated condition number of A beyond which the solution will be abandoned;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.minres! — Function
solver = minres!(solver::MinresSolver, A, b; kwargs...)
-solver = minres!(solver::MinresSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of minres.

See MinresSolver for more details about the solver.

source

MINRES-QLP

Krylov.minres_qlp — Function
(x, stats) = minres_qlp(A, b::AbstractVector{FC};
+                    callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = minres(A, b, x0::AbstractVector; kwargs...)

MINRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

Solve the shifted linear least-squares problem

minimize ‖b - (A + λI)x‖₂²

or the shifted linear system

(A + λI) x = b

of size n using the MINRES method, where λ ≥ 0 is a shift parameter, where A is Hermitian.

MINRES is formally equivalent to applying CR to Ax=b when A is positive definite, but is typically more stable and also applies to the case where A is indefinite.

MINRES produces monotonic residuals ‖r‖₂ and optimality residuals ‖Aᴴr‖₂.

Input arguments

  • A: a linear operator that models a Hermitian matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • window: number of iterations used to accumulate a lower bound on the error;
  • λ: regularization parameter;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • etol: stopping tolerance based on the lower bound on the error;
  • conlim: limit on the estimated condition number of A beyond which the solution will be abandoned;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.minres! — Function
solver = minres!(solver::MinresSolver, A, b; kwargs...)
+solver = minres!(solver::MinresSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of minres.

See MinresSolver for more details about the solver.

source

MINRES-QLP

Krylov.minres_qlp — Function
(x, stats) = minres_qlp(A, b::AbstractVector{FC};
                         M=I, ldiv::Bool=false, Artol::T=√eps(T),
                         λ::T=zero(T), atol::T=√eps(T),
                         rtol::T=√eps(T), itmax::Int=0,
                         timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
-                        callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = minres_qlp(A, b, x0::AbstractVector; kwargs...)

MINRES-QLP can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

MINRES-QLP is the only method based on the Lanczos process that returns the minimum-norm solution on singular inconsistent systems (A + λI)x = b of size n, where λ is a shift parameter. It is significantly more complex but can be more reliable than MINRES when A is ill-conditioned.

M also indicates the weighted norm in which residuals are measured.

Input arguments

  • A: a linear operator that models a Hermitian matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • λ: regularization parameter;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • Artol: relative stopping tolerance based on the Aᴴ-residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

References

source
Krylov.minres_qlp! — Function
solver = minres_qlp!(solver::MinresQlpSolver, A, b; kwargs...)
-solver = minres_qlp!(solver::MinresQlpSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of minres_qlp.

See MinresQlpSolver for more details about the solver.

source

MINARES

Krylov.minares — Function
(x, stats) = minares(A, b::AbstractVector{FC};
+                        callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = minres_qlp(A, b, x0::AbstractVector; kwargs...)

MINRES-QLP can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

MINRES-QLP is the only method based on the Lanczos process that returns the minimum-norm solution on singular inconsistent systems (A + λI)x = b of size n, where λ is a shift parameter. It is significantly more complex but can be more reliable than MINRES when A is ill-conditioned.

M also indicates the weighted norm in which residuals are measured.

Input arguments

  • A: a linear operator that models a Hermitian matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • λ: regularization parameter;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • Artol: relative stopping tolerance based on the Aᴴ-residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

References

source
Krylov.minres_qlp! — Function
solver = minres_qlp!(solver::MinresQlpSolver, A, b; kwargs...)
+solver = minres_qlp!(solver::MinresQlpSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of minres_qlp.

See MinresQlpSolver for more details about the solver.

source

MINARES

Krylov.minares — Function
(x, stats) = minares(A, b::AbstractVector{FC};
                      M=I, ldiv::Bool=false,
                      λ::T = zero(T), atol::T=√eps(T),
                      rtol::T=√eps(T), Artol::T = √eps(T),
                      itmax::Int=0, timemax::Float64=Inf,
                      verbose::Int=0, history::Bool=false,
-                     callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = minares(A, b, x0::AbstractVector; kwargs...)

MINARES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

MINARES solves the Hermitian linear system Ax = b of size n. MINARES minimizes ‖Arₖ‖₂ when M = Iₙ and ‖AMrₖ‖M otherwise. The estimates computed every iteration are ‖Mrₖ‖₂ and ‖AMrₖ‖M.

Input arguments

  • A: a linear operator that models a Hermitian positive definite matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • λ: regularization parameter;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • Artol: relative stopping tolerance based on the Aᴴ-residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.minares! — Function
solver = minares!(solver::MinaresSolver, A, b; kwargs...)
-solver = minares!(solver::MinaresSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of minares.

See MinaresSolver for more details about the solver.

source
+ callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = minares(A, b, x0::AbstractVector; kwargs...)

MINARES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

MINARES solves the Hermitian linear system Ax = b of size n. MINARES minimizes ‖Arₖ‖₂ when M = Iₙ and ‖AMrₖ‖M otherwise. The estimates computed every iteration are ‖Mrₖ‖₂ and ‖AMrₖ‖M.

Input arguments

Optional argument

Keyword arguments

Output arguments

Reference

source
Krylov.minares! — Function
solver = minares!(solver::MinaresSolver, A, b; kwargs...)
+solver = minares!(solver::MinaresSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of minares.

See MinaresSolver for more details about the solver.

source
diff --git a/previews/PR911/solvers/sp_sqd/index.html b/previews/PR911/solvers/sp_sqd/index.html index 89d096ccf..63f5a69d1 100644 --- a/previews/PR911/solvers/sp_sqd/index.html +++ b/previews/PR911/solvers/sp_sqd/index.html @@ -8,8 +8,8 @@ timemax::Float64=Inf, verbose::Int=0, history::Bool=false, callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, y, stats) = tricg(A, b, c, x0::AbstractVector, y0::AbstractVector; kwargs...)

TriCG can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.

Given a matrix A of dimension m × n, TriCG solves the Hermitian linear system

[ τE    A ] [ x ] = [ b ]
 [  Aᴴ  νF ] [ y ]   [ c ],

of size (n+m) × (n+m) where τ and ν are real numbers, E = M⁻¹ ≻ 0 and F = N⁻¹ ≻ 0. b and c must both be nonzero. TriCG could breakdown if τ = 0 or ν = 0. It's recommended to use TriMR in these cases.

By default, TriCG solves Hermitian and quasi-definite linear systems with τ = 1 and ν = -1.

TriCG is based on the preconditioned orthogonal tridiagonalization process and its relation with the preconditioned block-Lanczos process.

[ M   0 ]
-[ 0   N ]

indicates the weighted norm in which residuals are measured. It's the Euclidean norm when M and N are identity operators.

TriCG stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖r₀‖ * rtol. atol is an absolute tolerance and rtol is a relative tolerance.

Input arguments

Optional arguments

Keyword arguments

Output arguments

Reference

source
Krylov.tricg! — Function
solver = tricg!(solver::TricgSolver, A, b, c; kwargs...)
-solver = tricg!(solver::TricgSolver, A, b, c, x0, y0; kwargs...)

where kwargs are keyword arguments of tricg.

See TricgSolver for more details about the solver.

source

TriMR

Krylov.trimr — Function
(x, y, stats) = trimr(A, b::AbstractVector{FC}, c::AbstractVector{FC};
+[ 0   N ]

indicates the weighted norm in which residuals are measured. It's the Euclidean norm when M and N are identity operators.

TriCG stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖r₀‖ * rtol. atol is an absolute tolerance and rtol is a relative tolerance.

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m;
  • c: a vector of length n.

Optional arguments

  • x0: a vector of length m that represents an initial guess of the solution x;
  • y0: a vector of length n that represents an initial guess of the solution y.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the partitioned system;
  • N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the partitioned system;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • spd: if true, set τ = 1 and ν = 1 for Hermitian and positive-definite linear system;
  • snd: if true, set τ = -1 and ν = -1 for Hermitian and negative-definite linear systems;
  • flip: if true, set τ = -1 and ν = 1 for another known variant of Hermitian quasi-definite systems;
  • τ and ν: diagonal scaling factors of the partitioned Hermitian linear system;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length m;
  • y: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.tricg! — Function
solver = tricg!(solver::TricgSolver, A, b, c; kwargs...)
+solver = tricg!(solver::TricgSolver, A, b, c, x0, y0; kwargs...)

where kwargs are keyword arguments of tricg.

See TricgSolver for more details about the solver.

source

TriMR

Krylov.trimr — Function
(x, y, stats) = trimr(A, b::AbstractVector{FC}, c::AbstractVector{FC};
                       M=I, N=I, ldiv::Bool=false,
                       spd::Bool=false, snd::Bool=false,
                       flip::Bool=false, sp::Bool=false,
@@ -18,5 +18,5 @@
                       timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
                       callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, y, stats) = trimr(A, b, c, x0::AbstractVector, y0::AbstractVector; kwargs...)

TriMR can be warm-started from initial guesses x0 and y0 where kwargs are the same keyword arguments as above.

Given a matrix A of dimension m × n, TriMR solves the symmetric linear system

[ τE    A ] [ x ] = [ b ]
 [  Aᴴ  νF ] [ y ]   [ c ],

of size (n+m) × (n+m) where τ and ν are real numbers, E = M⁻¹ ≻ 0, F = N⁻¹ ≻ 0. b and c must both be nonzero. TriMR handles saddle-point systems (τ = 0 or ν = 0) and adjoint systems (τ = 0 and ν = 0) without any risk of breakdown.

By default, TriMR solves symmetric and quasi-definite linear systems with τ = 1 and ν = -1.

TriMR is based on the preconditioned orthogonal tridiagonalization process and its relation with the preconditioned block-Lanczos process.

[ M   0 ]
-[ 0   N ]

indicates the weighted norm in which residuals are measured. It's the Euclidean norm when M and N are identity operators.

TriMR stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖r₀‖ * rtol. atol is an absolute tolerance and rtol is a relative tolerance.

Input arguments

  • A: a linear operator that models a matrix of dimension m × n;
  • b: a vector of length m;
  • c: a vector of length n.

Optional arguments

  • x0: a vector of length m that represents an initial guess of the solution x;
  • y0: a vector of length n that represents an initial guess of the solution y.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size m used for centered preconditioning of the partitioned system;
  • N: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning of the partitioned system;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • spd: if true, set τ = 1 and ν = 1 for Hermitian and positive-definite linear system;
  • snd: if true, set τ = -1 and ν = -1 for Hermitian and negative-definite linear systems;
  • flip: if true, set τ = -1 and ν = 1 for another known variant of Hermitian quasi-definite systems;
  • sp: if true, set τ = 1 and ν = 0 for saddle-point systems;
  • τ and ν: diagonal scaling factors of the partitioned Hermitian linear system;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to m+n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length m;
  • y: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.trimr! — Function
solver = trimr!(solver::TrimrSolver, A, b, c; kwargs...)
-solver = trimr!(solver::TrimrSolver, A, b, c, x0, y0; kwargs...)

where kwargs are keyword arguments of trimr.

See TrimrSolver for more details about the solver.

source
+[ 0 N ]

indicates the weighted norm in which residuals are measured. It's the Euclidean norm when M and N are identity operators.

TriMR stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖r₀‖ * rtol. atol is an absolute tolerance and rtol is a relative tolerance.

Input arguments

Optional arguments

Keyword arguments

Output arguments

Reference

source
Krylov.trimr! — Function
solver = trimr!(solver::TrimrSolver, A, b, c; kwargs...)
+solver = trimr!(solver::TrimrSolver, A, b, c, x0, y0; kwargs...)

where kwargs are keyword arguments of trimr.

See TrimrSolver for more details about the solver.

source
diff --git a/previews/PR911/solvers/spd/index.html b/previews/PR911/solvers/spd/index.html index 1d6c6ea0e..e4ad9a78c 100644 --- a/previews/PR911/solvers/spd/index.html +++ b/previews/PR911/solvers/spd/index.html @@ -4,28 +4,28 @@ linesearch::Bool=false, atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0, timemax::Float64=Inf, verbose::Int=0, history::Bool=false, - callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = cg(A, b, x0::AbstractVector; kwargs...)

CG can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

The conjugate gradient method to solve the Hermitian linear system Ax = b of size n.

The method does not abort if A is not definite. M also indicates the weighted norm in which residuals are measured.

Input arguments

Optional argument

Keyword arguments

Output arguments

Reference

source
Krylov.cg! — Function
solver = cg!(solver::CgSolver, A, b; kwargs...)
-solver = cg!(solver::CgSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of cg.

See CgSolver for more details about the solver.

source

CR

Krylov.cr — Function
(x, stats) = cr(A, b::AbstractVector{FC};
+                callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = cg(A, b, x0::AbstractVector; kwargs...)

CG can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

The conjugate gradient method to solve the Hermitian linear system Ax = b of size n.

The method does not abort if A is not definite. M also indicates the weighted norm in which residuals are measured.

Input arguments

  • A: a linear operator that models a Hermitian positive definite matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
  • linesearch: if true, indicate that the solution is to be used in an inexact Newton method with linesearch. If negative curvature is detected at iteration k > 0, the solution of iteration k-1 is returned. If negative curvature is detected at iteration 0, the right-hand side is returned (i.e., the negative gradient);
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.cg! — Function
solver = cg!(solver::CgSolver, A, b; kwargs...)
+solver = cg!(solver::CgSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of cg.

See CgSolver for more details about the solver.

source

CR

Krylov.cr — Function
(x, stats) = cr(A, b::AbstractVector{FC};
                 M=I, ldiv::Bool=false, radius::T=zero(T),
                 linesearch::Bool=false, γ::T=√eps(T),
                 atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,
                 timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
-                callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = cr(A, b, x0::AbstractVector; kwargs...)

CR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

A truncated version of Stiefel’s Conjugate Residual method to solve the Hermitian linear system Ax = b of size n or the least-squares problem min ‖b - Ax‖ if A is singular. The matrix A must be Hermitian semi-definite. M also indicates the weighted norm in which residuals are measured.

Input arguments

  • A: a linear operator that models a Hermitian positive definite matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
  • linesearch: if true, indicate that the solution is to be used in an inexact Newton method with linesearch. If negative curvature is detected at iteration k > 0, the solution of iteration k-1 is returned. If negative curvature is detected at iteration 0, the right-hand side is returned (i.e., the negative gradient);
  • γ: tolerance to determine that the curvature of the quadratic model is nonpositive;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

References

source
Krylov.cr! — Function
solver = cr!(solver::CrSolver, A, b; kwargs...)
-solver = cr!(solver::CrSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of cr.

See CrSolver for more details about the solver.

source

CAR

Krylov.car — Function
(x, stats) = car(A, b::AbstractVector{FC};
+                callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = cr(A, b, x0::AbstractVector; kwargs...)

CR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

A truncated version of Stiefel’s Conjugate Residual method to solve the Hermitian linear system Ax = b of size n or the least-squares problem min ‖b - Ax‖ if A is singular. The matrix A must be Hermitian semi-definite. M also indicates the weighted norm in which residuals are measured.

Input arguments

  • A: a linear operator that models a Hermitian positive definite matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • radius: add the trust-region constraint ‖x‖ ≤ radius if radius > 0. Useful to compute a step in a trust-region method for optimization;
  • linesearch: if true, indicate that the solution is to be used in an inexact Newton method with linesearch. If negative curvature is detected at iteration k > 0, the solution of iteration k-1 is returned. If negative curvature is detected at iteration 0, the right-hand side is returned (i.e., the negative gradient);
  • γ: tolerance to determine that the curvature of the quadratic model is nonpositive;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

References

source
Krylov.cr! — Function
solver = cr!(solver::CrSolver, A, b; kwargs...)
+solver = cr!(solver::CrSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of cr.

See CrSolver for more details about the solver.

source

CAR

Krylov.car — Function
(x, stats) = car(A, b::AbstractVector{FC};
                  M=I, ldiv::Bool=false,
                  atol::T=√eps(T), rtol::T=√eps(T),
                  itmax::Int=0, timemax::Float64=Inf,
                  verbose::Int=0, history::Bool=false,
-                 callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = car(A, b, x0::AbstractVector; kwargs...)

CAR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

CAR solves the Hermitian and positive definite linear system Ax = b of size n. CAR minimizes ‖Arₖ‖₂ when M = Iₙ and ‖AMrₖ‖M otherwise. The estimates computed every iteration are ‖Mrₖ‖₂ and ‖AMrₖ‖M.

Input arguments

  • A: a linear operator that models a Hermitian positive definite matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.car! — Function
solver = car!(solver::CarSolver, A, b; kwargs...)
-solver = car!(solver::CarSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of car.

See CarSolver for more details about the solver.

source

CG-LANCZOS

Krylov.cg_lanczos — Function
(x, stats) = cg_lanczos(A, b::AbstractVector{FC};
+                 callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = car(A, b, x0::AbstractVector; kwargs...)

CAR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

CAR solves the Hermitian and positive definite linear system Ax = b of size n. CAR minimizes ‖Arₖ‖₂ when M = Iₙ and ‖AMrₖ‖M otherwise. The estimates computed every iteration are ‖Mrₖ‖₂ and ‖AMrₖ‖M.

Input arguments

  • A: a linear operator that models a Hermitian positive definite matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.car! — Function
solver = car!(solver::CarSolver, A, b; kwargs...)
+solver = car!(solver::CarSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of car.

See CarSolver for more details about the solver.

source

CG-LANCZOS

Krylov.cg_lanczos — Function
(x, stats) = cg_lanczos(A, b::AbstractVector{FC};
                         M=I, ldiv::Bool=false,
                         check_curvature::Bool=false, atol::T=√eps(T),
                         rtol::T=√eps(T), itmax::Int=0,
                         timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
-                        callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = cg_lanczos(A, b, x0::AbstractVector; kwargs...)

CG-LANCZOS can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

The Lanczos version of the conjugate gradient method to solve the Hermitian linear system Ax = b of size n.

The method does not abort if A is not definite.

Input arguments

  • A: a linear operator that models a Hermitian matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • check_curvature: if true, check that the curvature of the quadratic along the search direction is positive, and abort if not, unless linesearch is also true;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a LanczosStats structure.

References

source
Krylov.cg_lanczos! — Function
solver = cg_lanczos!(solver::CgLanczosSolver, A, b; kwargs...)
-solver = cg_lanczos!(solver::CgLanczosSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of cg_lanczos.

See CgLanczosSolver for more details about the solver.

source

CG-LANCZOS-SHIFT

Krylov.cg_lanczos_shift — Function
(x, stats) = cg_lanczos_shift(A, b::AbstractVector{FC}, shifts::AbstractVector{T};
+                        callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = cg_lanczos(A, b, x0::AbstractVector; kwargs...)

CG-LANCZOS can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

The Lanczos version of the conjugate gradient method to solve the Hermitian linear system Ax = b of size n.

The method does not abort if A is not definite.

Input arguments

  • A: a linear operator that models a Hermitian matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • check_curvature: if true, check that the curvature of the quadratic along the search direction is positive, and abort if not, unless linesearch is also true;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a LanczosStats structure.

References

source
Krylov.cg_lanczos! — Function
solver = cg_lanczos!(solver::CgLanczosSolver, A, b; kwargs...)
+solver = cg_lanczos!(solver::CgLanczosSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of cg_lanczos.

See CgLanczosSolver for more details about the solver.

source

CG-LANCZOS-SHIFT

Krylov.cg_lanczos_shift — Function
(x, stats) = cg_lanczos_shift(A, b::AbstractVector{FC}, shifts::AbstractVector{T};
                               M=I, ldiv::Bool=false,
                               check_curvature::Bool=false, atol::T=√eps(T),
                               rtol::T=√eps(T), itmax::Int=0,
                               timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
-                              callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

The Lanczos version of the conjugate gradient method to solve a family of shifted systems

(A + αI) x = b  (α = α₁, ..., αₚ)

of size n. The method does not abort if A + αI is not definite.

Input arguments

  • A: a linear operator that models a Hermitian matrix of dimension n;
  • b: a vector of length n;
  • shifts: a vector of length p.

Keyword arguments

  • M: linear operator that models a Hermitian positive-definite matrix of size n used for centered preconditioning;
  • ldiv: define whether the preconditioner uses ldiv! or mul!;
  • check_curvature: if true, check that the curvature of the quadratic along the search direction is positive, and abort if not, unless linesearch is also true;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a vector of p dense vectors, each one of length n;
  • stats: statistics collected on the run in a LanczosShiftStats structure.

References

source
Krylov.cg_lanczos_shift! — Function
solver = cg_lanczos_shift!(solver::CgLanczosShiftSolver, A, b, shifts; kwargs...)

where kwargs are keyword arguments of cg_lanczos_shift.

See CgLanczosShiftSolver for more details about the solver.

source
+ callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

The Lanczos version of the conjugate gradient method to solve a family of shifted systems

(A + αI) x = b  (α = α₁, ..., αₚ)

of size n. The method does not abort if A + αI is not definite.

Input arguments

Keyword arguments

Output arguments

References

source
Krylov.cg_lanczos_shift! — Function
solver = cg_lanczos_shift!(solver::CgLanczosShiftSolver, A, b, shifts; kwargs...)

where kwargs are keyword arguments of cg_lanczos_shift.

See CgLanczosShiftSolver for more details about the solver.

source
diff --git a/previews/PR911/solvers/unsymmetric/index.html b/previews/PR911/solvers/unsymmetric/index.html index b60e9de77..cc191c678 100644 --- a/previews/PR911/solvers/unsymmetric/index.html +++ b/previews/PR911/solvers/unsymmetric/index.html @@ -4,51 +4,51 @@ M=I, N=I, ldiv::Bool=false, atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0, timemax::Float64=Inf, verbose::Int=0, history::Bool=false, - callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = bilq(A, b, x0::AbstractVector; kwargs...)

BiLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

Solve the square linear system Ax = b of size n using BiLQ. BiLQ is based on the Lanczos biorthogonalization process and requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b. When A is Hermitian and b = c, BiLQ is equivalent to SYMMLQ. BiLQ requires support for adjoint(M) and adjoint(N) if preconditioners are provided.

Input arguments

Optional argument

Keyword arguments

Output arguments

References

source
Krylov.bilq! — Function
solver = bilq!(solver::BilqSolver, A, b; kwargs...)
-solver = bilq!(solver::BilqSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of bilq.

See BilqSolver for more details about the solver.

source

QMR

Krylov.qmr — Function
(x, stats) = qmr(A, b::AbstractVector{FC};
+                  callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = bilq(A, b, x0::AbstractVector; kwargs...)

BiLQ can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

Solve the square linear system Ax = b of size n using BiLQ. BiLQ is based on the Lanczos biorthogonalization process and requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b. When A is Hermitian and b = c, BiLQ is equivalent to SYMMLQ. BiLQ requires support for adjoint(M) and adjoint(N) if preconditioners are provided.

Input arguments

  • A: a linear operator that models a matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • c: the second initial vector of length n required by the Lanczos biorthogonalization process;
  • transfer_to_bicg: transfer from the BiLQ point to the BiCG point, when it exists. The transfer is based on the residual norm;
  • M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
  • N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

References

source
Krylov.bilq! — Function
solver = bilq!(solver::BilqSolver, A, b; kwargs...)
+solver = bilq!(solver::BilqSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of bilq.

See BilqSolver for more details about the solver.

source

QMR

Krylov.qmr — Function
(x, stats) = qmr(A, b::AbstractVector{FC};
                  c::AbstractVector{FC}=b, M=I, N=I, ldiv::Bool=false, atol::T=√eps(T),
                  rtol::T=√eps(T), itmax::Int=0, timemax::Float64=Inf, verbose::Int=0,
-                 history::Bool=false, callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = qmr(A, b, x0::AbstractVector; kwargs...)

QMR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

Solve the square linear system Ax = b of size n using QMR.

QMR is based on the Lanczos biorthogonalization process and requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b. When A is Hermitian and b = c, QMR is equivalent to MINRES. QMR requires support for adjoint(M) and adjoint(N) if preconditioners are provided.

Input arguments

  • A: a linear operator that models a matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • c: the second initial vector of length n required by the Lanczos biorthogonalization process;
  • M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
  • N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

References

source
Krylov.qmr! — Function
solver = qmr!(solver::QmrSolver, A, b; kwargs...)
-solver = qmr!(solver::QmrSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of qmr.

See QmrSolver for more details about the solver.

source

CGS

Krylov.cgs — Function
(x, stats) = cgs(A, b::AbstractVector{FC};
+                 history::Bool=false, callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = qmr(A, b, x0::AbstractVector; kwargs...)

QMR can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

Solve the square linear system Ax = b of size n using QMR.

QMR is based on the Lanczos biorthogonalization process and requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b. When A is Hermitian and b = c, QMR is equivalent to MINRES. QMR requires support for adjoint(M) and adjoint(N) if preconditioners are provided.

Input arguments

  • A: a linear operator that models a matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • c: the second initial vector of length n required by the Lanczos biorthogonalization process;
  • M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
  • N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

References

source
Krylov.qmr! — Function
solver = qmr!(solver::QmrSolver, A, b; kwargs...)
+solver = qmr!(solver::QmrSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of qmr.

See QmrSolver for more details about the solver.

source

CGS

Krylov.cgs — Function
(x, stats) = cgs(A, b::AbstractVector{FC};
                  c::AbstractVector{FC}=b, M=I, N=I,
                  ldiv::Bool=false, atol::T=√eps(T),
                  rtol::T=√eps(T), itmax::Int=0,
                  timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
-                 callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = cgs(A, b, x0::AbstractVector; kwargs...)

CGS can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

Solve the consistent linear system Ax = b of size n using CGS. CGS requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b.

From "Iterative Methods for Sparse Linear Systems (Y. Saad)" :

«The method is based on a polynomial variant of the conjugate gradients algorithm. Although related to the so-called bi-conjugate gradients (BCG) algorithm, it does not involve adjoint matrix-vector multiplications, and the expected convergence rate is about twice that of the BCG algorithm.

The Conjugate Gradient Squared algorithm works quite well in many cases. However, one difficulty is that, since the polynomials are squared, rounding errors tend to be more damaging than in the standard BCG algorithm. In particular, very high variations of the residual vectors often cause the residual norms computed to become inaccurate.

TFQMR and BICGSTAB were developed to remedy this difficulty.»

Input arguments

  • A: a linear operator that models a matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • c: the second initial vector of length n required by the Lanczos biorthogonalization process;
  • M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
  • N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.cgs! — Function
solver = cgs!(solver::CgsSolver, A, b; kwargs...)
-solver = cgs!(solver::CgsSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of cgs.

See CgsSolver for more details about the solver.

source

BiCGSTAB

Krylov.bicgstab — Function
(x, stats) = bicgstab(A, b::AbstractVector{FC};
+                 callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = cgs(A, b, x0::AbstractVector; kwargs...)

CGS can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

Solve the consistent linear system Ax = b of size n using CGS. CGS requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b.

From "Iterative Methods for Sparse Linear Systems (Y. Saad)" :

«The method is based on a polynomial variant of the conjugate gradients algorithm. Although related to the so-called bi-conjugate gradients (BCG) algorithm, it does not involve adjoint matrix-vector multiplications, and the expected convergence rate is about twice that of the BCG algorithm.

The Conjugate Gradient Squared algorithm works quite well in many cases. However, one difficulty is that, since the polynomials are squared, rounding errors tend to be more damaging than in the standard BCG algorithm. In particular, very high variations of the residual vectors often cause the residual norms computed to become inaccurate.

TFQMR and BICGSTAB were developed to remedy this difficulty.»

Input arguments

  • A: a linear operator that models a matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • c: the second initial vector of length n required by the Lanczos biorthogonalization process;
  • M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
  • N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.cgs! — Function
solver = cgs!(solver::CgsSolver, A, b; kwargs...)
+solver = cgs!(solver::CgsSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of cgs.

See CgsSolver for more details about the solver.

source

BiCGSTAB

Krylov.bicgstab — Function
(x, stats) = bicgstab(A, b::AbstractVector{FC};
                       c::AbstractVector{FC}=b, M=I, N=I,
                       ldiv::Bool=false, atol::T=√eps(T),
                       rtol::T=√eps(T), itmax::Int=0,
                       timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
-                      callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = bicgstab(A, b, x0::AbstractVector; kwargs...)

BICGSTAB can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

Solve the square linear system Ax = b of size n using BICGSTAB. BICGSTAB requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b.

The Biconjugate Gradient Stabilized method is a variant of BiCG, like CGS, but using different updates for the Aᴴ-sequence in order to obtain smoother convergence than CGS.

If BICGSTAB stagnates, we recommend DQGMRES and BiLQ as alternative methods for unsymmetric square systems.

BICGSTAB stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖b‖ * rtol.

Input arguments

  • A: a linear operator that models a matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • c: the second initial vector of length n required by the Lanczos biorthogonalization process;
  • M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
  • N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

References

source
Krylov.bicgstab! — Function
solver = bicgstab!(solver::BicgstabSolver, A, b; kwargs...)
-solver = bicgstab!(solver::BicgstabSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of bicgstab.

See BicgstabSolver for more details about the solver.

source

DIOM

Krylov.diom — Function
(x, stats) = diom(A, b::AbstractVector{FC};
+                      callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = bicgstab(A, b, x0::AbstractVector; kwargs...)

BICGSTAB can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

Solve the square linear system Ax = b of size n using BICGSTAB. BICGSTAB requires two initial vectors b and c. The relation bᴴc ≠ 0 must be satisfied and by default c = b.

The Biconjugate Gradient Stabilized method is a variant of BiCG, like CGS, but using different updates for the Aᴴ-sequence in order to obtain smoother convergence than CGS.

If BICGSTAB stagnates, we recommend DQGMRES and BiLQ as alternative methods for unsymmetric square systems.

BICGSTAB stops when itmax iterations are reached or when ‖rₖ‖ ≤ atol + ‖b‖ * rtol.

Input arguments

  • A: a linear operator that models a matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • c: the second initial vector of length n required by the Lanczos biorthogonalization process;
  • M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
  • N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

References

source
Krylov.bicgstab! — Function
solver = bicgstab!(solver::BicgstabSolver, A, b; kwargs...)
+solver = bicgstab!(solver::BicgstabSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of bicgstab.

See BicgstabSolver for more details about the solver.

source

DIOM

Krylov.diom — Function
(x, stats) = diom(A, b::AbstractVector{FC};
                   memory::Int=20, M=I, N=I, ldiv::Bool=false,
                   reorthogonalization::Bool=false, atol::T=√eps(T),
                   rtol::T=√eps(T), itmax::Int=0,
                   timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
-                  callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = diom(A, b, x0::AbstractVector; kwargs...)

DIOM can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

Solve the consistent linear system Ax = b of size n using DIOM.

DIOM only orthogonalizes the new vectors of the Krylov basis against the memory most recent vectors. If CG is well defined on Ax = b and memory = 2, DIOM is theoretically equivalent to CG. If k ≤ memory where k is the number of iterations, DIOM is theoretically equivalent to FOM. Otherwise, DIOM interpolates between CG and FOM and is similar to CG with partial reorthogonalization.

An advantage of DIOM is that non-Hermitian or Hermitian indefinite or both non-Hermitian and indefinite systems of linear equations can be handled by this single algorithm.

Input arguments

  • A: a linear operator that models a matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • memory: the number of most recent vectors of the Krylov basis against which to orthogonalize a new vector;
  • M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
  • N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • reorthogonalization: reorthogonalize the new vectors of the Krylov basis against the memory most recent vectors;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.diom! — Function
solver = diom!(solver::DiomSolver, A, b; kwargs...)
-solver = diom!(solver::DiomSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of diom.

Note that the memory keyword argument is the only exception. It's required to create a DiomSolver and can't be changed later.

See DiomSolver for more details about the solver.

source

FOM

Krylov.fom — Function
(x, stats) = fom(A, b::AbstractVector{FC};
+                  callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = diom(A, b, x0::AbstractVector; kwargs...)

DIOM can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

Solve the consistent linear system Ax = b of size n using DIOM.

DIOM only orthogonalizes the new vectors of the Krylov basis against the memory most recent vectors. If CG is well defined on Ax = b and memory = 2, DIOM is theoretically equivalent to CG. If k ≤ memory where k is the number of iterations, DIOM is theoretically equivalent to FOM. Otherwise, DIOM interpolates between CG and FOM and is similar to CG with partial reorthogonalization.

An advantage of DIOM is that non-Hermitian or Hermitian indefinite or both non-Hermitian and indefinite systems of linear equations can be handled by this single algorithm.

Input arguments

  • A: a linear operator that models a matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • memory: the number of most recent vectors of the Krylov basis against which to orthogonalize a new vector;
  • M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
  • N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • reorthogonalization: reorthogonalize the new vectors of the Krylov basis against the memory most recent vectors;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.diom! — Function
solver = diom!(solver::DiomSolver, A, b; kwargs...)
+solver = diom!(solver::DiomSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of diom.

Note that the memory keyword argument is the only exception. It's required to create a DiomSolver and can't be changed later.

See DiomSolver for more details about the solver.

source

FOM

Krylov.fom — Function
(x, stats) = fom(A, b::AbstractVector{FC};
                  memory::Int=20, M=I, N=I, ldiv::Bool=false,
                  restart::Bool=false, reorthogonalization::Bool=false,
                  atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,
                  timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
-                 callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = fom(A, b, x0::AbstractVector; kwargs...)

FOM can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

Solve the linear system Ax = b of size n using FOM.

FOM algorithm is based on the Arnoldi process and a Galerkin condition.

Input arguments

  • A: a linear operator that models a matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • memory: if restart = true, the restarted version FOM(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;
  • M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
  • N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • restart: restart the method after memory iterations;
  • reorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.fom! — Function
solver = fom!(solver::FomSolver, A, b; kwargs...)
-solver = fom!(solver::FomSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of fom.

Note that the memory keyword argument is the only exception. It's required to create a FomSolver and can't be changed later.

See FomSolver for more details about the solver.

source

DQGMRES

Krylov.dqgmres — Function
(x, stats) = dqgmres(A, b::AbstractVector{FC};
+                 callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = fom(A, b, x0::AbstractVector; kwargs...)

FOM can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

Solve the linear system Ax = b of size n using FOM.

FOM algorithm is based on the Arnoldi process and a Galerkin condition.

Input arguments

  • A: a linear operator that models a matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • memory: if restart = true, the restarted version FOM(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;
  • M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
  • N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • restart: restart the method after memory iterations;
  • reorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.fom! — Function
solver = fom!(solver::FomSolver, A, b; kwargs...)
+solver = fom!(solver::FomSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of fom.

Note that the memory keyword argument is the only exception. It's required to create a FomSolver and can't be changed later.

See FomSolver for more details about the solver.

source

DQGMRES

Krylov.dqgmres — Function
(x, stats) = dqgmres(A, b::AbstractVector{FC};
                      memory::Int=20, M=I, N=I, ldiv::Bool=false,
                      reorthogonalization::Bool=false, atol::T=√eps(T),
                      rtol::T=√eps(T), itmax::Int=0,
                      timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
-                     callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = dqgmres(A, b, x0::AbstractVector; kwargs...)

DQGMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

Solve the consistent linear system Ax = b of size n using DQGMRES.

DQGMRES algorithm is based on the incomplete Arnoldi orthogonalization process and computes a sequence of approximate solutions with the quasi-minimal residual property.

DQGMRES only orthogonalizes the new vectors of the Krylov basis against the memory most recent vectors. If MINRES is well defined on Ax = b and memory = 2, DQGMRES is theoretically equivalent to MINRES. If k ≤ memory where k is the number of iterations, DQGMRES is theoretically equivalent to GMRES. Otherwise, DQGMRES interpolates between MINRES and GMRES and is similar to MINRES with partial reorthogonalization.

Input arguments

  • A: a linear operator that models a matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • memory: the number of most recent vectors of the Krylov basis against which to orthogonalize a new vector;
  • M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
  • N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
  • reorthogonalization: reorthogonalize the new vectors of the Krylov basis against the memory most recent vectors;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.dqgmres! — Function
solver = dqgmres!(solver::DqgmresSolver, A, b; kwargs...)
-solver = dqgmres!(solver::DqgmresSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of dqgmres.

Note that the memory keyword argument is the only exception. It's required to create a DqgmresSolver and can't be changed later.

See DqgmresSolver for more details about the solver.

source

GMRES

Krylov.gmres — Function
(x, stats) = gmres(A, b::AbstractVector{FC};
+                     callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = dqgmres(A, b, x0::AbstractVector; kwargs...)

DQGMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

Solve the consistent linear system Ax = b of size n using DQGMRES.

DQGMRES algorithm is based on the incomplete Arnoldi orthogonalization process and computes a sequence of approximate solutions with the quasi-minimal residual property.

DQGMRES only orthogonalizes the new vectors of the Krylov basis against the memory most recent vectors. If MINRES is well defined on Ax = b and memory = 2, DQGMRES is theoretically equivalent to MINRES. If k ≤ memory where k is the number of iterations, DQGMRES is theoretically equivalent to GMRES. Otherwise, DQGMRES interpolates between MINRES and GMRES and is similar to MINRES with partial reorthogonalization.

Input arguments

  • A: a linear operator that models a matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • memory: the number of most recent vectors of the Krylov basis against which to orthogonalize a new vector;
  • M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
  • N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
  • reorthogonalization: reorthogonalize the new vectors of the Krylov basis against the memory most recent vectors;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.dqgmres! — Function
solver = dqgmres!(solver::DqgmresSolver, A, b; kwargs...)
+solver = dqgmres!(solver::DqgmresSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of dqgmres.

Note that the memory keyword argument is the only exception. It's required to create a DqgmresSolver and can't be changed later.

See DqgmresSolver for more details about the solver.

source

GMRES

Krylov.gmres — Function
(x, stats) = gmres(A, b::AbstractVector{FC};
                    memory::Int=20, M=I, N=I, ldiv::Bool=false,
                    restart::Bool=false, reorthogonalization::Bool=false,
                    atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,
                    timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
-                   callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = gmres(A, b, x0::AbstractVector; kwargs...)

GMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

Solve the linear system Ax = b of size n using GMRES.

GMRES algorithm is based on the Arnoldi process and computes a sequence of approximate solutions with the minimum residual.

Input arguments

  • A: a linear operator that models a matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • memory: if restart = true, the restarted version GMRES(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;
  • M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
  • N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • restart: restart the method after memory iterations;
  • reorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.gmres! — Function
solver = gmres!(solver::GmresSolver, A, b; kwargs...)
-solver = gmres!(solver::GmresSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of gmres.

Note that the memory keyword argument is the only exception. It's required to create a GmresSolver and can't be changed later.

See GmresSolver for more details about the solver.

source

FGMRES

Krylov.fgmres — Function
(x, stats) = fgmres(A, b::AbstractVector{FC};
+                   callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = gmres(A, b, x0::AbstractVector; kwargs...)

GMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

Solve the linear system Ax = b of size n using GMRES.

GMRES algorithm is based on the Arnoldi process and computes a sequence of approximate solutions with the minimum residual.

Input arguments

  • A: a linear operator that models a matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • memory: if restart = true, the restarted version GMRES(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;
  • M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
  • N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • restart: restart the method after memory iterations;
  • reorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.gmres! — Function
solver = gmres!(solver::GmresSolver, A, b; kwargs...)
+solver = gmres!(solver::GmresSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of gmres.

Note that the memory keyword argument is the only exception. It's required to create a GmresSolver and can't be changed later.

See GmresSolver for more details about the solver.

source

FGMRES

Krylov.fgmres — Function
(x, stats) = fgmres(A, b::AbstractVector{FC};
                     memory::Int=20, M=I, N=I, ldiv::Bool=false,
                     restart::Bool=false, reorthogonalization::Bool=false,
                     atol::T=√eps(T), rtol::T=√eps(T), itmax::Int=0,
                     timemax::Float64=Inf, verbose::Int=0, history::Bool=false,
-                    callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = fgmres(A, b, x0::AbstractVector; kwargs...)

FGMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

Solve the linear system Ax = b of size n using FGMRES.

FGMRES computes a sequence of approximate solutions with minimum residual. FGMRES is a variant of GMRES that allows changes in the right preconditioner at each iteration.

This implementation allows a left preconditioner M and a flexible right preconditioner N. A situation in which the preconditioner is "not constant" is when a relaxation-type method, a Chebyshev iteration or another Krylov subspace method is used as a preconditioner. Compared to GMRES, there is no additional cost incurred in the arithmetic but the memory requirement almost doubles. Thus, GMRES is recommended if the right preconditioner N is constant.

Input arguments

  • A: a linear operator that models a matrix of dimension n;
  • b: a vector of length n.

Optional argument

  • x0: a vector of length n that represents an initial guess of the solution x.

Keyword arguments

  • memory: if restart = true, the restarted version FGMRES(k) is used with k = memory. If restart = false, the parameter memory should be used as a hint of the number of iterations to limit dynamic memory allocations. Additional storage will be allocated if the number of iterations exceeds memory;
  • M: linear operator that models a nonsingular matrix of size n used for left preconditioning;
  • N: linear operator that models a nonsingular matrix of size n used for right preconditioning;
  • ldiv: define whether the preconditioners use ldiv! or mul!;
  • restart: restart the method after memory iterations;
  • reorthogonalization: reorthogonalize the new vectors of the Krylov basis against all previous vectors;
  • atol: absolute stopping tolerance based on the residual norm;
  • rtol: relative stopping tolerance based on the residual norm;
  • itmax: the maximum number of iterations. If itmax=0, the default number of iterations is set to 2n;
  • timemax: the time limit in seconds;
  • verbose: additional details can be displayed if verbose mode is enabled (verbose > 0). Information will be displayed every verbose iterations;
  • history: collect additional statistics on the run such as residual norms, or Aᴴ-residual norms;
  • callback: function or functor called as callback(solver) that returns true if the Krylov method should terminate, and false otherwise;
  • iostream: stream to which output is logged.

Output arguments

  • x: a dense vector of length n;
  • stats: statistics collected on the run in a SimpleStats structure.

Reference

source
Krylov.fgmres! — Function
solver = fgmres!(solver::FgmresSolver, A, b; kwargs...)
-solver = fgmres!(solver::FgmresSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of fgmres.

Note that the memory keyword argument is the only exception. It's required to create a FgmresSolver and can't be changed later.

See FgmresSolver for more details about the solver.

source
+ callback=solver->false, iostream::IO=kstdout)

T is an AbstractFloat such as Float32, Float64 or BigFloat. FC is T or Complex{T}.

(x, stats) = fgmres(A, b, x0::AbstractVector; kwargs...)

FGMRES can be warm-started from an initial guess x0 where kwargs are the same keyword arguments as above.

Solve the linear system Ax = b of size n using FGMRES.

FGMRES computes a sequence of approximate solutions with minimum residual. FGMRES is a variant of GMRES that allows changes in the right preconditioner at each iteration.

This implementation allows a left preconditioner M and a flexible right preconditioner N. A situation in which the preconditioner is "not constant" is when a relaxation-type method, a Chebyshev iteration or another Krylov subspace method is used as a preconditioner. Compared to GMRES, there is no additional cost incurred in the arithmetic but the memory requirement almost doubles. Thus, GMRES is recommended if the right preconditioner N is constant.

Input arguments

Optional argument

Keyword arguments

Output arguments

Reference

source
Krylov.fgmres! — Function
solver = fgmres!(solver::FgmresSolver, A, b; kwargs...)
+solver = fgmres!(solver::FgmresSolver, A, b, x0; kwargs...)

where kwargs are keyword arguments of fgmres.

Note that the memory keyword argument is the only exception. It's required to create a FgmresSolver and can't be changed later.

See FgmresSolver for more details about the solver.

source
diff --git a/previews/PR911/storage/index.html b/previews/PR911/storage/index.html index a808d5004..e8fb2bf0e 100644 --- a/previews/PR911/storage/index.html +++ b/previews/PR911/storage/index.html @@ -52,4 +52,4 @@ └──────────────────┴──────────────────┴────────────────────┘

If we want the total number of bytes used by the solver, we can call nbytes = sizeof(solver).

nbytes = sizeof(solver)
560121

Thereafter, we can use Base.format_bytes(nbytes) to recover what is displayed in the REPL.

Base.format_bytes(nbytes)
"546.993 KiB"

To verify that we match the theoretical results, we just need to multiply the storage requirement of a method by the number of bytes associated to the precision of the linear problem. For instance, we need 4 bytes for the precision Float32, 8 bytes for precisions Float64 and ComplexF32, and 16 bytes for the precision ComplexF64.

FC = Float64                            # precision of the least-squares problem
 ncoefs_lsmr = 5*n + 2*m                 # number of coefficients
 nbytes_lsmr = sizeof(FC) * ncoefs_lsmr  # number of bytes
560000

Therefore, you can check that you have enough memory in RAM to allocate a KrylovSolver.

free_nbytes = Sys.free_memory()
-Base.format_bytes(free_nbytes)  # Total free memory in RAM in bytes.
"13.207 GiB"
Note
+Base.format_bytes(free_nbytes) # Total free memory in RAM in bytes.
"13.174 GiB"
Note
diff --git a/previews/PR911/tips/index.html b/previews/PR911/tips/index.html index fe87d2eb9..09c3926c0 100644 --- a/previews/PR911/tips/index.html +++ b/previews/PR911/tips/index.html @@ -23,4 +23,4 @@ julia -t N # alternative: --threads N JULIA_NUM_THREADS=N julia

Thereafter, you can verify the number of threads usable by Julia

using Base.Threads
-nthreads()

The following benchmarks illustrate the time required in seconds to compute 1000 sparse matrix-vector products with symmetric matrices of the SuiteSparse Matrix Collection. The computer used for the benchmarks has 2 physical cores and Julia was launched with JULIA_NUM_THREADS=2.

benchmarks

+nthreads()

The following benchmarks illustrate the time required in seconds to compute 1000 sparse matrix-vector products with symmetric matrices of the SuiteSparse Matrix Collection. The computer used for the benchmarks has 2 physical cores and Julia was launched with JULIA_NUM_THREADS=2.

benchmarks

diff --git a/previews/PR911/warm-start/index.html b/previews/PR911/warm-start/index.html index 01fe2a24a..ee9b7d6a4 100644 --- a/previews/PR911/warm-start/index.html +++ b/previews/PR911/warm-start/index.html @@ -23,4 +23,4 @@ Δx, Δy, stats = trimr(A, b₀, c₀, M=M, N=N) x = x₀ + Δx -y = y₀ + Δy +y = y₀ + Δy