turning off all 4 LEDs leaves one on
You are running into the read-modify-write issue when writing to the port register.

Each line of C code most likely produces a bit setting instruction.  Those instructions read the whole register they operate on, set the specified bit, then write the whole word back.  This has a side affect on I/O ports of freezing any of the bits you're not setting at whatever state they were read at.

Let's say in one cycle you do BSF PORTB,0.  That starts the output drivers going high.  If in the next cycle you do BSF PORTB,1, then it will first read what the pins state of RB0 actually is, not what you set it to in the previous instruction.  If the external circuit keeps the port pin from going high fast enough, it will still be 0 when read by the second instruction.  Since that instruction isn't operating on that bit, it will write it back the way it found it, which is 0.  Now the RB0 pin will be driven actively low again.

The best way around this is to do the bit manipulation instructions on the LAT register, if this PIC has them.  If you need to do a bunch of pin settings on a port at a time, read the port, do all the settings on the local copy, then write that back to the port once.  Or, leave enough time before read-modify-write operations on port registers so that the actual pin voltage has stabalized.

Some knee-jerkers are sure to say that you should always use shadow registers, but that is usually unnecessary and way more than needed.  I've done well over 100 PIC projects, and I don't think I've used shadow registers to get around port R-M-W issues once.  Once you understand what is going on, a little common sense goes a long way.