How can I adjust the (yellow line) length of the light in Blender?
You are using an Area light.

For area and spot type lights, the line visualizes the Custom Distance > Distance, regard of wether Custom Distance is active. You can change it with the Distance property.



If you would like to remove the yellow tracking blob "Look At", uncheck Look At in the viewports Gizmo settings.



If you are using cycles (or another render engine), this Distance propetry is not accessible, although it is represented in the viewport. Briefly switch to the EEVEE render engine, change the distance value until the viewport looks good, the switch back to your actual render engine.