FileVault - One account can unlock but preventing full login and forcing logout and login again with other user
I can answer this question myself out of accident which I somehow was able to even retrace what I did. 

To prevent a login you do this: 


Add a new user, preferably a non admin user, and enter that password for that user which you wish to use as a filevault password. 
Revoke all other users right to use filevault using: sudo fdesetup remove -user "username" from terminal. Note, do keep the user you just created. In fact you can not remove that user. One user must be on the filevault user list. 
Go into the user settings again. And reset the password for the user just created. That will through I guess a bug, in my case feature, only change the password and not the filevault user. 


Now, restart computer. When you are prompted for the fielvault password it is the old and not the resetted password. This will unlock disk, but since the password is wrong for login, system won't be able to login user. Press esc and login your user. 

You are welcome.