Camera calibration vs. registration Suppose I have two cameras, c1 and c2 that are securely installed and do not move.  What I want is a matrix T, that takes a pixel of an image from camera c1 and calculates the according pixel in an image from camera c2. 

I now wonder:

Do I need to calibrate both c1 and c2 to get their intrinsic and extrinsic parameters and additionally find a relationship between the coordinate system of camera c1 and c2

or

Can I calculate a registration between an image of c1 and an image of c2 using interest points (e.g., by MATLABs cpselect and fitgeotrans functions) and use this registration matrix also for future images?

I believe that a generally suitable transformation matrix T between the two cameras' images is only possible to get if you first calibrate both cameras (in the sense that you calculate the extrinsic and intrinsic parameters). In my feeling it is otherwise impossible to reflect the fact that we are dealing with 3D objects. T will only be able to transform the 2D images. Is this correct? If so, can you elaborate more on why this is the case?
There is no matrix that maps a pixel in camera 1 to the corresponding pixel in camera 2. This is because the location of the corresponding pixel depends on the 3-D location of the corresponding point in the world.

What you have instead is the Fundamental matrix, which maps a pixel in camera 1 to a line in camera 2, called the epipolar line.