Loops over the entries of x computing dx_i (e_i is the unit vector in the ith direction) to
min || b - A(x + dx_i e_i ||_2
dx_i
That is, it changes a single entry of x to minimize the new residual norm.
Let A_i represent the ith column of A, then the minimization can be written as
min || r - (dx_i) A e_i ||_2
dx_i
or min || r - (dx_i) A_i ||_2
dx_i
take the derivative with respect to dx_i to obtain
dx_i = (A_i^T A_i)^(-1) A_i^T r
This algorithm can be thought of as Gauss-Seidel on the normal equations
These "projections" can be done simultaneously for all columns (similar to Jacobi) or sequentially (similar to Gauss-Seidel/SOR). This is only coded for SOR type.
This is related to, but not the same as "row projection" methods.
This is currently coded only for SeqAIJ matrices in sequential (SOR) form.