How can I detect thin objects (like pens and pencils) without a bounding box but only 2 endpoints and the orientation? I am looking to detect thin objects, like pens, pencils, and surgical instruments. The bounding box is not important, but I am looking to see if I can train a model to detect both the object as well as its orientation.
Typical object detection networks, like R-CNN, YOLO, and SSD encode the class name and bounding boxes. Instead of bounding boxes, I'm looking to encode only 2 points, one starting $x,y$ point and one ending $x,y$ point.  The start point for objects is where one would grip the object.  For instance:

The pencil eraser(start point) is pointed 50 degrees to the top right.
The surgical instrument is 10 degrees from the x-axis and the handle is pointed to the bottom right.
Pen tip (endpoint) is pointing vertically upwards.
Fork, the start point would be the grip handle part, and the endpoint would be in the middle where the 4 prongs are.

As long as I can encode the start and endpoints, then I can determine the orientation. I would need to define these points during training.
The question is whether there is an existing model (mobile net/inception/RCNN) that I can encode this information in?  One potential way I was thinking was to use YOLO and for the bounding box, the top left $x,y$ would be the starting point $x,y$ (handle), whereas the bounding box's width and height would be replaced with the endpoint $x,y$ (pencil writing tip, fork prongs.
Recent work achieves a similar task: Object recognition together with the bounding box (e.g. YOLO---there are quite a few on Github too). The bounding box is not enough in your case, but it is an interesting pattern: Recognition plus some form of measurement. Such architectures could be good candidate to start with, and repurpose for stick orientation.

The problem could also leverage the current results in gait recognition. In fact, this looks closer to the problem at hand than object recognition. An example is this model based on multiview (many pictures input) recognition, with a demonstration on Github. Gait recognition is also popular these days, and many inspiring papers and OSS implementations are available.

The above presents two approaches your problem could benefit from, as a "combination". My gut feeling is that tilt and orientations may be easier than direction (i.e. where is the tip?).



The question calls for training a model. An alternative approach, perhaps to start with and get more insight, could be to go with "standard" computer vision algorithms, such as the Hough Transform. This transforms allows to find lines in an image. The mathematics are at reach, and it may work well enough for a quick demo. Also, your handle name suggests "embedded mobile" engineer, and a simple Hough Transform could be cheap on mobile.