title: Postgres 9.4 detects Deadlock when read-modify-write on single table
body: <p>We have an application with a simple table </p>
  
  <pre><code>given_entity{
      UUID id;
      TimeStamp due_time;
      TimeStamp process_time;
  } 
  </code></pre>
  
  <p>This is a spring boot (1.2.5.RELEASE) application that uses spring-data-jpa.1.2.5.RELEASE with hibernate-4.3.10.FINAL as jpa provier.<br>
  We have 5 instances of this application with each of them having a scheduler running every 2 second and querying the database for rows that have a due_time of last 2 mins until now that are not yet processed;</p>
  
  <pre><code>SELECT * FROM given_entity 
  WHERE process_time is null and due_time between  now() and NOW() - INTERVAL '2 minutes' 
  FOR UPDATE
  </code></pre>
  
  <p>Requirement is each row of above table gets successfully processed by exactly one of application instances.
  Then the application instance processes these rows and update its process_time field in one transaction.
  This may or may not take more than 2 seconds, which is scheduler interval.
  Also we don't have any index but PK index on this table. 
  Second point worth noting is that these instances might insert rows this table which is called separately by clients.</p>
  
  <p>Problem: in the logs I see this message from postgresql (rarely but it happens)</p>
  
  <pre><code>ERROR: deadlock detected
  Detail: Process 10625 waits for ShareLock on transaction 25382449; blocked by process 10012.
  Process 10012 waits for ShareLock on transaction 25382448; blocked by process 12238.
  Process 12238 waits for AccessExclusiveLock on tuple (1371,45) of relation 19118 of database 19113; blocked by process 10625.
  Hint: See server log for query details.
  Where: while locking tuple (1371,45) in relation "given_entity"
  </code></pre>
  
  <p>Question:
    How does this happen? 
    I checked postgresql locks and searched internet. I didn't find anything that says deadlock is possible on only one simple table.
    I also couldn't reproduce this error using test.</p>