title: Pgpool Log error: Primary 0 owns only 0 standbys out of 1
body: <p>I have pgpool-II v4.0.2 running on an app server managing two PostgreSQL 11.2 database nodes (0 - primary and 1 - standby). The psql -c 'show pool_nodes' command shows both nodes being up:</p>
  
  <pre><code>$ psql -c 'show pool_nodes'
   node_id | hostname | port | status | lb_weight | role | select_cnt | load_balance_node | replication_delay | last_status_change 
  ---------+-----------+------+--------+-----------+---------+------------+-------------------+-------------------+---------------------
   0 | 10.1.1.20 | 5432 | up | 0.500000 | primary | 0 | true | 0 | 2019-02-22 21:37:55
   1 | 10.1.3.20 | 5432 | up | 0.500000 | standby | 0 | false | 0 | 2019-02-22 21:44:49
  </code></pre>
  
  <p>However, I've started seeing this error today written to the log every 10 secs after testing online recovery and fallback processes:</p>
  
  <pre><code>$ sudo -u postgres pgpool -n
  ---
  2019-02-22 22:15:47: pid 22204: LOG: Setting up socket for 0.0.0.0:5432
  2019-02-22 22:15:47: pid 22204: LOG: Setting up socket for :::5432
  2019-02-22 22:15:47: pid 22204: LOG: find_primary_node_repeatedly: waiting for finding a primary node
  2019-02-22 22:15:47: pid 22204: LOG: verify_backend_node_status: primary 0 owns only 0 standbys out of 1
  2019-02-22 22:15:47: pid 22204: LOG: find_primary_node: primary node is 0
  2019-02-22 22:15:47: pid 22204: LOG: find_primary_node: standby node is 1
  2019-02-22 22:15:47: pid 22204: LOG: pgpool-II successfully started. version 4.0.2 (torokiboshi)
  2019-02-22 22:15:47: pid 22204: LOG: node status[0]: 1
  2019-02-22 22:15:47: pid 22204: LOG: node status[1]: 2
  2019-02-22 22:15:47: pid 22238: LOG: verify_backend_node_status: primary 0 owns only 0 standbys out of 1
  2019-02-22 22:15:57: pid 22238: LOG: verify_backend_node_status: primary 0 owns only 0 standbys out of 1
  ...
  (repeats each 10 seconds)
  </code></pre>
  
  <p>I can't figure out what this means and how to fix it. The PostgreSQL streaming replication appears to work properly. That is when I create and populate a test table on primary, I can see the same happening on standby.</p>
  
  <p>Any idea where I can look? Thank you.</p>
  
  <p><strong>Update 2/23/19:</strong></p>
  
  <p>I've narrowed it down a little more. It appears that, though my backend node is running in streaming replication, the following query returns NULL (<a href="https://github.com/beocommedia/pgpool-II/blob/4.0.2/src/main/pgpool_main.c#L3265-L3268" rel="nofollow noreferrer">https://github.com/beocommedia/pgpool-II/blob/4.0.2/src/main/pgpool_main.c#L3265-L3268</a>):</p>
  
  <pre><code>2019-02-23 15:32:32: pid 17383: DEBUG: verify_backend_node_status: pg_stat_wal_receiver status for standby 1 is NULL
  2019-02-23 15:32:32: pid 17383: LOCATION: pgpool_main.c:3271
  </code></pre>
  
  <p>If I query the same standby node via pgpool, I get the status back:</p>
  
  <pre><code>$ psql -x -h 10.1.3.20 -c "SELECT status, conninfo FROM pg_stat_wal_receiver";
  -[ RECORD 1 ]----------------------------------------------------------------
  status   | streaming
  conninfo | user=repl passfile=/var/lib/postgresql/.pgpass dbname=replication 
             host=10.1.1.20 port=5432 fallback_application_name=walreceiver
             sslmode=prefer sslcompression=0 krbsrvname=postgres
             target_session_attrs=any
  </code></pre>
  
  <p>Any idea what I may be missing here?</p>