"Recent Items" count is not saved in System Preferences-OS X 10.11 "El Capitan"
Sounds like the .plist file is corrupted. Go to Library/Preferences and drag the file named com.apple.recentitems.plist into the Trash and empty it. When you open System Preferences again a new com.apple.recentitems.plist file will be created and you should be able to change the number of Recent Items without issue.