How to obtain the camera extrinsic matrix without using chessboard
I think the answer is yes, it is possible to get the extrinsic matrix of hte camera with just a single square, though the error might be higher as there is less redundancy.  Let's look at how much information we obtain:


How many unknowns?  The extrinsic matrix has 5 real-valued parameters (the rotation matrix has 2 angles; the translation matrix has a 3D location).  In other words, there are 5 degrees of freedom or 5 unknowns.
How many equations?  We obtain 8 equations/constraints.  I am assuming we know the 3D location of each corner of the square (in world coordinates).  We also know the 2D location of each corner (in image coordinates).  The camera matrix maps each 3D point to the corresponding 2D point.  So, each point gives us 2 equations (from the known 2D image coordinates), and there are 4 corners, so we get 8 equations.


In short, we have 8 equations in 5 unknowns.  That's enough that it should be possible to recover the extrinsic matrix.  Of course, there's not a lot of redundancy here, so the error might be higher than if you used a chessboard.  A chessboard would provide a lot more equations (each corner inside the chessboard provides an additional point with a known 3D->2D mapping).

How would you recover it?  Follow the guidelines in https://ksimek.github.io/2015/03/29/QA-recovering-pose-of-calibrated-camera/.  In particular, you could solve for the 5 parameters of the extrinsic matrix using some optimization routine, such as gradient descent or L-BFGS.

Given some value for those 5 parameters, you can compute the camera matrix and map the 3D world coordinates of each corner to its corresponding 2D image coordinates; then you can compute the $L_2$ error (sum of squared error) between the inferred 2D coordinates and the known 2D coordinates from the picture.  This gives you an objective function that maps a value for the 5 parameters to an error term (which you want to minimize).  Next, you can apply any off-the-shelf optimization library to try to minimize this objective function.  This sounds pretty doable, with not too many lines of code to implement.