Neural network architecture for line orientation prediction
A long shot: these guys have worked on a problem that might be relevant. They define "semantic" lines as lines delimiting significant regions or objects in an image. To detect such lines, they use the conv layers from a pre-trained VGG16 net and then add their own layers on top. The cool thing about their approach is that they run both classification and regression in parallel on the same network.

You might be able to adopt a similar technique to determine where the line is, and then run some simple analysis on the extracted line to determine the offset and the orientation.