# How to convert an affine transform matrix into "theta" to use torch.nn.functional.affine_grid()?

Hi, all!
I met a problem when using `torch.nn.functioal.affine_grid(theta,size)`.
I’d like to know how to convert an affine transformation matrix described in `scipy/skimage.transform/opencv` into a right argument `theta` in `torch.nn.functioal.affine_grid(theta,size)`?

Now suppose we want to apply an affine transormation on an image with shape=(H,W,3), where

What’s the right `theta` which should be used in `torch.nn.functioal.affine_grid(theta,size)` ?

1 Like

Hi, I met same problem, Have you solved it?

Hi, i have solved it.Code is below.

import cv2
import torch.nn.functional as F
import skimage.transform as trans
import numpy as np

def convert_image_np(inp):
“”“Convert a Tensor to numpy image.”""
inp = inp.numpy().transpose((1, 2, 0))
inp = (inp*255).astype(np.uint8)
return inp

def param2theta(param, w, h):
param = np.linalg.inv(param)
theta = np.zeros([2,3])
theta[0,0] = param[0,0]
theta[0,1] = param[0,1]*h/w
theta[0,2] = param[0,2]*2/w + param[0,0] + param[0,1] - 1
theta[1,0] = param[1,0]*w/h
theta[1,1] = param[1,1]
theta[1,2] = param[1,2]*2/h + param[1,0] + param[1,1] - 1
return theta
tr = trans.estimate_transform(‘affine’, src=src, dst=dst)
M = tr.params[0:2,:]
img = cv2.warpAffine(image,M,(w,h))
theta = param2theta(tr.params, w, h)
image = Variable(image).unsqueeze(0).cuda()
theta = Variable(theta).unsqueeze(0).cuda()
grid = F.affine_grid(theta,image.size())
img_ = F.grid_sample(image,grid)
img_ = convert_image_np(img_.data.cpu().squeeze(0))

The visualizations of img and img_ should be same. Hope it can help you!

``````theta[0,0] = param[0,0]
theta[0,1] = param[0,1]*h/w
theta[0,2] = param[0,2]*2/w + theta[0,0] + theta[0,1] - 1
theta[1,0] = param[1,0]*w/h
theta[1,1] = param[1,1]
theta[1,2] = param[1,2]*2/h + theta[1,0] + theta[1,1] - 1
``````

I have solved this issue by the above code. Maybe your code only works under some non-trival cases.

2 Likes

I think it works under general case, not some non-trival cases.

2 Likes

I have a similar problem.
I already use ITK-snap to process an affine registration : 3D volume image A to B.
Then I get a matrix:

0.97464 0.0364447 0.00781475 1.95549
0.00375859 0.922498 0.0297265 -8.87417
-0.00727863 -0.0145295 0.961844 -1.10425
0 0 0 1

I think it’s incompatibal in F.affine_grid theta.
The shape of affine_grid shape is (N, 3, 4)
likes: R | T
[
[a, b, c, t0],
[e, f, g, t1],
[h, i, j, t2]
]
I think the anterior three rows of the matrix is theta, but it’s wrong.
the R is likely equal to theta[:3, :3],(I am not sure)
but the T (t0, t1, t2) must be incompatible.

How can I get a right theta?
By the way, how can I know what does the torch.nn.functioal.affine_grid really do ?

It seems that the current PyTorch API doesn’t support 3D affine transformation. Up to now, `affine_grid()` and `grid_sample()` can only support 2D affine transformation (especially, 2D perspective transformation is not supported yet).

As the advices from @HectorAnadon, to implement complicated geometric transformations, you can try Kornia.

Hi @ljqcava
@oldriver-wang Oh, you are right. `affine_grid()` begin to support 3D transformation. It’s good news and glad to see that! However, it doesn’t support perspective transformation.
{"ft_lang_label":"__label__en","ft_lang_prob":0.8311208,"math_prob":0.9967251,"size":1223,"snap":"2022-05-2022-21","text_gpt3_token_len":337,"char_repetition_ratio":0.15340443,"word_repetition_ratio":0.0,"special_character_ratio":0.25429273,"punctuation_ratio":0.2129964,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.98888284,"pos_list":[0],"im_url_duplicate_count":[null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2022-05-28T02:04:52Z\",\"WARC-Record-ID\":\"<urn:uuid:dca5a54f-76c4-499f-b606-3522de1c185a>\",\"Content-Length\":\"38410\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:0edb6eaa-a93d-49fb-9581-0f2f39c7430e>\",\"WARC-Concurrent-To\":\"<urn:uuid:e2aaf6ac-2072-4a73-9270-2bf26fbbedd0>\",\"WARC-IP-Address\":\"159.203.145.104\",\"WARC-Target-URI\":\"https://discuss.pytorch.org/t/how-to-convert-an-affine-transform-matrix-into-theta-to-use-torch-nn-functional-affine-grid/24315\",\"WARC-Payload-Digest\":\"sha1:AJJVRDNWXFWMF6XA2WGO56432LYGMJFN\",\"WARC-Block-Digest\":\"sha1:XQOZ6VTN7SRIFIM2UZ3BXWPSH7YJDYJD\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2022/CC-MAIN-2022-21/CC-MAIN-2022-21_segments_1652663011588.83_warc_CC-MAIN-20220528000300-20220528030300-00707.warc.gz\"}"}