Add a line between two objects?
I added these to clarify what I do


You could select one empty, then press ShiftS --> Cursor to selected!


Add a curve (since you can modify it's thickness), press Tab to enter 'edit mode', select one vertex, press ShiftS --> Selection to cursor. Press Tab to exit 'edit mode'


Select the other empty, ShiftS --> Cursor to selected. Then select the curve, press Tab to enter 'edit mode'

select the other vertex, press ShiftS --> selection to cursor


result: