How do I use a multimeter to determine why one of my lights is not turning on?
From your description, it appears that you have 1 switch that controls one duplex outlet and 2 lights that are plugged into those outlet receptacles.  If my interpretation is correct, then do the following:


Reverse the light plugs at the outlet receptacles &amp; test to see if the lights illuminate.  If only the left light illuminates, then both of your outlet receptacles are working properly &amp; the fault is in the right plug/wiring/fixture/etc.  If the left light does not illuminate &amp; the right light does illuminate, then one of your outlet receptacles is bad.  If neither light illuminates, then you probably have multiple problems to troubleshoot.
Assuming you are in the US:  Use this video procedure to properly test your outlet power.  Note:  If your multimeter is not auto-ranging, then you would manually set your multimeter to AC mode &amp; your voltage scale to 200V before testing your 120V outlets.
To test an incandescent bulb or the wiring to your light fixture, you would first disconnect power to your light &amp; then use the Ohms mode of your multimeter.  Click here to see a video of generally how to do this.  You are essentially searching for an open circuit.  That means electric power cannot pass through the plug or wiring or contacts or bulb filament or through some other part of the light circuit.
In your situation, the most common fix actions are:  replace a bulb, clean corrosion off a metal contact point/s or switch, and/or fix a loose/disconnected/broken wire.  If you have florescent lights, then you may need to replace a starter (condenser) or a ballast.


hth, best regards!