Difference between hide() and ['#access'] = FALSE I want to remove the delete button from a standard edit form in certain cases and I'm looking for the right way to do that. 

So far I found that hide() marks an element as already rendered so it basically tricks the rendering process.

['#access'] = FALSE does something similar I guess, but still a little different.

In both cases, nothing is rendered concerning the delete button, but does either of them prevent the delete code from being executed on submit (e.g. if a user was manipulating the POST request)?

Or would I need a form_validate() method for that anyway?
Hiding the element is done during the rendering process.  If you look at the code, it just marks the element as already being rendered.

The #access key on the element is used during the form build process (can't find my chart, so I can't tell you exactly which part).  It says that this element is part of the form, but a particular user may or may not have access to be able to use it.

Unsetting the element, just throws the element away and says it is not part of the form anymore.

Of the three options, using #access is preferred: it's the API so you should use it.  When you just hide the element, a user can still submit that element git GET/POST as the Form API thinks it is still valid.  For non-submit elements, I believe the #default_value will also appear in $form_state['values'] when you validate/submit (however I don't recall the last time I tested this).