Postgres hstore check if a key exists and does not match value
The usage of defined() in the documentation:


  does hstore contain non-NULL value for key?


In addition to a NULL value for an existing key, it returns FALSE also when the key does not exist.

To keep it clean, you might want to use the exist() function instead:

SELECT exist('"bla" =&gt; 1234', 'bloo');
 exist 
───────
 f


So your query would look like

SELECT changed_fields -&gt; 'updated_by' 
  FROM audit.logged_actions 
 WHERE changed_fields -&gt; 'updated_by' &lt;&gt; 'system' 
       OR NOT exist(changed_fields, 'updated_by')
 ORDER by event_id desc;


Notes: 


never test logical values using the equality operator (unless you can be really sure the value cannot be NULL).  You can directly use the value if it is guaranteed to not be NULL (like in my query), or use the IS operator.
In the current form, the query is checking a simple 'OR' - from your wording it is not quite clear what should happen when both conditions are true.