What could cause a toilet to leave the flapper up and keep running?
Check for any debris around the flapper and its hinges.  It is the flapper moving back to the original closed position that puts the handle back in the correct position, not the handle that moves the flapper (when closing, obviously when you flush the handle opens the flapper).  You have the right idea with adjusting the chain to ensure there is always slack.

There are usually a couple positions for the chain to attach on the handle itself - try a different position.

If all else fails, try replacing the flapper.