Keychain login password not working for Allow access
Ok, so there was a detail that I didn't think was important, and apparently I was I wrong. I was connected to the system over VNC. 

When I opened the system log and tried to perform the action over VNC I saw the entry "SecurityAgent: Ignoring user action since the dialog has recieved events from an untrusted source".

When I tried to perform the action on the system it self I was successful.