Return type of solvers?

[jiahao]

The purpose of this issue is to discuss what should be returned by iterative solvers. Obviously the solution vector should be returned, but we should agree on what other data should be returned as a general principle.

There is a case to be made that it is worth returning additional data about the convergence of the solution and/or the iteration history. Examples of such data are:

• isconverged flag (should be Bool, not numeric)
• number of iterations taken
• residual vector
• values of convergence criteria, e.g. norm of residual vector
• iteration history of the residual vector

Furthermore, should such additional information be returned as separate quantities in a tuple following the solution vector, or should everything be packaged into a new type?

[jiahao]

I've put in a prototype of the ConvergenceHistory type as of 645783e

[stevengj]

Yes, please use a type for these kinds of convergence statistics (which should be very similar between iterative solvers). I hate it when functions return a tuple of a zillion things.

[ViralBShah]

I was thinking along the same lines - of having a return type to avoid returning large tuples

[jiahao]

Closing with the choices indicated.

[JasonPries]

For solver with restarts, both the number of inner and outer iterations should be returned.

[stevengj]

I tend to think that the number of matrix-vector products is more useful than the number of "iterations", since it is too hard for the user to know what the algorithm defines as an "iteration". MVP counts are a common metric for benchmarking convergence rates of iterative solvers.

[jiahao]

We could certainly track the number of matvecs. I don't know how people report this conventionally in the literature though. Are A·v matvecs counted separately from preconditioning matvecs?

[JasonPries]

I think of one MVP as including the application of preconditioners, since I generally expect the preconditioners to be applied every time A*v is required.

[stevengj]

@priesj is right, I think. You just count the number of A*v products (and A'*v products); counting preconditioners separately is unnecessary since the count will normally be identical.

[jiahao]

I've added some code to track mvps in 9614e01.

[lopezm94]

I think this is a good place to talk about the return types of the new API.

The extra data should look something like this:

type ConvergenceHistory  #It doesn't have to be this name
    restart::Int
    isconverged::Bool
    threshold::Dict{Symbol, Real}
    mvps::Int
    MethodLog::Dict{Symbol, Vector}
end

Some methods have different convergence criteria, for example lsqr, lanczos, etc. Also, sometimes the user could want the estimate residuals of each eigenvalue which might need extra calculations or ritz or estimage eigenvalues values which aren't residuals.

With a dictionary we can give a name to tolerances and each extra info in the method, this way the user could potentially blacklist/whitelist information with keyword arguments and receive easy to access information.

Returning restart shouldn't hurt anyone, if it is unrestarted it could be the same as maxiter. The number of Iters and restarts could be calculated with functions operating on ConvergenceHistory.

@lruthotto @dpo