We discuss some properties of the matrix ATA, constructed from a m-by-n matrix A where m is significantly greater than n. This is normally the case in least-square problems.
The matrix ATA is always symmetric by definition.
The matrix ATA is positive semi-definite if it is invertible, since we havexTATAx=(Ax)TAx=∥Ax∥2≥0.
The matrix ATA is almost always invertible, because m is significantly greater than n, and hence the constructed matrix is high likely to be full-rank.
The dense linear solver ldlt() in Eigen is the fastest for positive or negative semi-definite matrices. Remember to use double precision to avoid numerical instability issues.
The following logs reveal one of the potential issues of using A.ldlt().solve(b);
The solver was used for Gauss-Newton optimization in an Iterative Closest Point algorithm. It was tested under the termination condition of max_iteration only.
The result suggests that we should check if A and b are healthy before sending to A.ldlt().solve(b);, and it might be better to add epsilon as another termination condition.
Note that float type supports a value range of 1.2E-38 to 3.4E+38, and 6-7 decimal digits, while double type supports a value range of 2.3E-308 to 1.7E+308, and 15 decimal digits.
Updates: This issue was resolved by switching to double precision. (Tested multiple times towards 100 iterations. The produced result is stable.)