Getting different eigenvalues between using numpy.linalg.eigh() and torch.symeig()
I am trying to understand why am I getting different eigenvalues between using numpy.linalg.eigh() and torch.symeig().
An example is as below:
Code:
import numpy as np
import torch

arr_symmetric = np.array([[1.,2,3], [2,5,6], [3,6,9]])
arr_symmetric, arr_symmetric.dtype

Output:
(array([[1., 2., 3.],
        [2., 5., 6.],
        [3., 6., 9.]]), dtype('float64'))

Code:
tsr_symmetric = torch.tensor(arr_symmetric)
tsr_symmetric

Output:
tensor([[1., 2., 3.],
        [2., 5., 6.],
        [3., 6., 9.]], dtype=torch.float64)

Code:
w, v = np.linalg.eigh(arr_symmetric)
w, v

Output:
(array([4.05517871e-16, 6.99264746e-01, 1.43007353e+01]),
 array([[-9.48683298e-01,  1.77819106e-01, -2.61496397e-01],
        [ 2.22044605e-16, -8.26924214e-01, -5.62313386e-01],
        [ 3.16227766e-01,  5.33457318e-01, -7.84489190e-01]]))

Code:
e, v = torch.symeig(tsr_symmetric, eigenvectors=True)
e, v

Output:
(tensor([-2.6047e-16,  6.9926e-01,  1.4301e+01], dtype=torch.float64),
 tensor([[ 9.4868e-01, -1.7782e-01,  2.6150e-01],
         [ 8.6389e-16,  8.2692e-01,  5.6231e-01],
         [-3.1623e-01, -5.3346e-01,  7.8449e-01]], dtype=torch.float64))

As you can see one of the eigenvalues is different, ie. 4.05517871e-16 vs. -2.6047e-16
Why is this happening?

4.05517871e-16 is very close to zero so is -2.6047e-16. They are very very close by. You can verify the same as below because input = V.e.V^T where e is a diagonal matrix with eigen values in the diagonal.
import numpy as np
import torch

arr_symmetric = np.array([[1.,2,3], [2,5,6], [3,6,9]])

e, v = np.linalg.eigh(arr_symmetric)
print (np.dot(v, np.dot(np.diag(e), v.T)))
for i in range(3):
    print (np.dot(arr_symmetric, v[:,i].reshape(-1,1)), e[i]*v[:,i])

e, v = torch.symeig(torch.tensor(arr_symmetric), eigenvectors=True)
print (torch.matmul(v, torch.matmul(e.diag_embed(), v.transpose(-2, -1))))
for i in range(3):
    print (np.dot(arr_symmetric, v[:,i].reshape(-1,1)), e[i]*v[:,i])

Output:
[[1. 2. 3.]
 [2. 5. 6.]
 [3. 6. 9.]]
[[3.33066907e-16]
 [8.88178420e-16]
 [8.88178420e-16]] [-3.84708031e-16  9.00430554e-32  1.28236010e-16]
[[ 0.12434263]
 [-0.57823895]
 [ 0.3730279 ]] [ 0.12434263 -0.57823895  0.3730279 ]
[[ -3.73959074]
 [ -8.04149487]
 [-11.21877222]] [ -3.73959074  -8.04149487 -11.21877222]
tensor([[1.0000, 2.0000, 3.0000],
        [2.0000, 5.0000, 6.0000],
        [3.0000, 6.0000, 9.0000]], dtype=torch.float64)
[[-3.33066907e-16]
 [ 0.00000000e+00]
 [-8.88178420e-16]] tensor([-2.4710e-16, -2.2502e-31,  8.2368e-17], dtype=torch.float64)
[[-0.12434263]
 [ 0.57823895]
 [-0.3730279 ]] tensor([-0.1243,  0.5782, -0.3730], dtype=torch.float64)
[[ 3.73959074]
 [ 8.04149487]
 [11.21877222]] tensor([ 3.7396,  8.0415, 11.2188], dtype=torch.float64)
​

