How can I power an Arduino Nano with a LiPo battery, preferably 3.7v
In short: not reliably.  The Arduino Nano expects either a 5V regulated supply or a 6-20V unregulated supply (https://www.arduino.cc/en/Main/ArduinoBoardNano).  Even if you plugged it in and it worked initially, the battery voltage will decrease gradually as it discharges (LiPo cells can drop as low as ~3.2V), making it more and more unreliable.  You'd technically be overclocking the processor, since the Nano runs at 16 MHz and the datasheet for the microcontroller requires at least a 4.5V supply to run above 10 MHz.

A safer solution would be to use a 2S LiPo battery (~7.4V), which would be plenty to supply the regulated input pin (VIN).  On the other hand, if you're set on using a 3.7V LiPo, other Arduino boards (like certain versions of the Pro Mini) run at 3.3V.

As for your second question, 3.7V is plenty to power a few LEDs so long as they are connected in parallel (or to separate pins).  The only thing you need to be aware of is current draw--more current means shorter battery life.  For example, if your circuit draws 100 mA total from the battery, and your battery is specified at 500 mAh, you will get approximately (500 / 100) = 5 hours of battery life.