Absolute and relative tolerance for linear solvers

[ranocha]

This PR adds keyword arguments abstol and reltol to iterative solvers for linear system and deprecates the former keyword tol (which is now reltol).

• BiCGStab(l)
• CG
• Chebyshev
• GMRES
• IDR(s)
• MINRES
• QMR
• LSMR doesn't seem to fit to this common framework
• LSQR doesn't seem to fit to this common framework

Since this is my first contribution to IterativeSolvers.jl, I would like to get some feedback on this work to improve this PR. Thanks!

If this approach is acceptable, I will continue to work on the other iterative solvers and adapt them accordingly.

By the way, I noticed that CG doesn't respect the stopping criterion described in the docs, see also #244. I think it would be good to fix this (as described in my comment in cg.jl).

Closes #207 by adding the keyword argument abstol.

Closes #244 by fixing the convergence criterion for CG (reltol should be relative to the initial residual instead of the RHS b).

[ranocha]

I made all the changes to the iterative solvers for "simple" linear problems. Since the solvers for least-squares problems use a different setup of tolerances, I didn't adapt them. It would be nice to have another round of reviews to improve this PR 🙂

[mschauer]

Can one express the logic/stopping criteria with reltol and abstol using isapprox from base?

[ranocha]

Can one express the logic/stopping criteria with reltol and abstol using isapprox from base?

isapprox(x, y) returns true if

norm(x-y) <= max(atol, rtol*max(norm(x), norm(y)))

Our stopping criterion is

norm(residual) <= max(abstol, reltol * initial_residual_norm)

Hence, I don't see a possibility to do that.

[mschauer]

I see, thanks.

[ranocha]

@haampie Can we merge this? The decreased coverage is only caused by the new depwarns.

[haampie]

Hi @ranocha , I haven't had the chance to review yet. Tomorrow I can do that

[haampie]

Thank you!