title: Openshift + Ansible : Failed when Play Approve node certificates when bootstrapping
body: <p>I'm trying to install Openshift Origin 3.11 with Ansible in offline mode. I got an issue when execute  deploy_cluster.yml and the error is :</p>
  
  <pre><code>TASK [Approve node certificates when bootstrapping] *************************************************************************************************************************************************************************************
  Wednesday 15 April 2020  11:18:45 +0700 (0:00:00.056)       0:03:48.565 *******
  FAILED - RETRYING: Approve node certificates when bootstrapping (30 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (29 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (28 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (27 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (26 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (25 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (24 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (23 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (22 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (21 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (20 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (19 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (18 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (17 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (16 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (15 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (14 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (13 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (12 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (11 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (10 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (9 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (8 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (7 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (6 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (5 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (4 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (3 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (2 retries left).
  FAILED - RETRYING: Approve node certificates when bootstrapping (1 retries left).
  
  PLAY RECAP ******************************************************************************************************************************************************************************************************************************
  devoriginapp01             : ok=264  changed=45   unreachable=0    failed=1
  devoriginapp02             : ok=94   changed=12   unreachable=0    failed=0
  devoriginapp03             : ok=93   changed=12   unreachable=0    failed=0
  localhost                  : ok=11   changed=0    unreachable=0    failed=0
  
  
  INSTALLER STATUS ************************************************************************************************************************************************************************************************************************
  Initialization              : Complete (0:00:11)
  Node Bootstrap Preparation  : Complete (0:01:17)
  Health Check                : Complete (0:00:00)
  etcd Install                : Complete (0:00:17)
  Master Install              : Complete (0:01:41)
  Master Additional Install   : Complete (0:00:19)
  Node Join                   : In Progress (0:03:19)
          This phase can be restarted by running: playbooks/openshift-node/join.yml
  Wednesday 15 April 2020  11:22:02 +0700 (0:03:16.421)       0:07:04.987 *******
  ===============================================================================
  Approve node certificates when bootstrapping ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 196.42s
  openshift_node : Check status of node image pre-pull ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 30.61s
  openshift_control_plane : Check status of control plane image pre-pull ---------------------------------------------------------------------------------------------------------------------------------------------------------- 30.32s
  openshift_manageiq : Configure role/user permissions ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.70s
  openshift_control_plane : Wait for APIs to become available ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.69s
  openshift_node : Add firewalld allow rules --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.18s
  openshift_node : Install node, clients, and conntrack packages ------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.92s
  openshift_node : Update journald setup ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.84s
  tuned : Ensure files are populated from templates -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.74s
  tuned : Restart tuned service ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.72s
  tuned : Restart tuned service ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.68s
  openshift_control_plane : Add firewalld allow rules ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 1.65s
  openshift_ca : Install the base package for admin tooling ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 1.54s
  openshift_control_plane : Wait for all control plane pods to come up and become ready -------------------------------------------------------------------------------------------------------------------------------------------- 1.53s
  openshift_master_certificates : Check status of master certificates -------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.38s
  openshift_node : Pre-pull node image --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.36s
  openshift_examples : Remove defunct quickstart templates from openshift namespace ------------------------------------------------------------------------------------------------------------------------------------------------ 1.35s
  openshift_node : pre-pull pod image ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.34s
  tuned : Ensure files are populated from templates -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.26s
  openshift_node_group : Make temp directory for templates ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.18s
  
  
  Failure summary:
  
  
    1. Hosts:    devoriginapp01
       Play:     Approve any pending CSR requests from inventory nodes
       Task:     Approve node certificates when bootstrapping
       Message:  Could not find csr for nodes: devoriginapp02, devoriginapp03
  </code></pre>
  
  <p>Description about my environment :</p>
  
  <ul>
  <li><p>Ansible version</p>
  
  <pre><code>ansible 2.6.5
    config file = /openshift-ansible/ansible.cfg
    configured module search path = [u'/home/origin/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
    ansible python module location = /usr/lib/python2.7/site-packages/ansible
    executable location = /usr/bin/ansible
    python version = 2.7.5 (default, Jun 11 2019, 14:33:56) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
  </code></pre></li>
  <li><p>OS Version (Master - Client same version)</p>
  
  <p>Red Hat Enterprise Linux Server release 7.7 (Maipo)</p></li>
  <li><p>Docker version</p>
  
  <p>Client:
   Version:           18.09.1
   API version:       1.39
   Go version:        go1.10.6
   Git commit:        4c52b90
   Built:             Wed Jan  9 19:35:01 2019
   OS/Arch:           linux/amd64
   Experimental:      false</p>
  
  <p>Server: Docker Engine - Community
   Engine:
    Version:          18.09.1
    API version:      1.39 (minimum version 1.12)
    Go version:       go1.10.6
    Git commit:       4c52b90
    Built:            Wed Jan  9 19:06:30 2019
    OS/Arch:          linux/amd64
    Experimental:     false</p></li>
  </ul>
  
  <p>Steps that I did : </p>
  
  <ol>
  <li>ansible-playbook openshift-ansible/playbooks/prerequisites.yml (Success)</li>
  <li>ansible-playbook openshift-ansible/playbooks/deploy_cluster.yml</li>
  </ol>
  
  <p>Additional :</p>
  
  <pre><code>[root@devoriginapp01 openshift-ansible]# cat inventory.ini
  [OSEv3:children]
  masters
  nodes
  etcd
  
  [OSEv3:vars]
  #openshift_additional_repos=[{'id': 'centos-paas', 'name': 'centos-paas', 'baseurl' :'https://buildlogs.centos.org/centos/7/paas/x86_64/openshift-origin311', 'gpgcheck' :'0', 'enabled' :'1'}]
  ansible_ssh_user=origin
  ansible_become=yes
  openshift_deployment_type=origin
  ansible_sudo=true
  enable_excluders=False
  enable_docker_excluder=False
  ansible_service_broker_install=False
  openshift_disable_check=disk_availability,docker_storage,memory_availability,docker_image_availability,package_version,package_availability
  #docker_install=true
  os_firewall_use_firewalld=True
  
  containerized=True
  os_sdn_network_plugin_name='redhat/openshift-ovs-multitenant'
  
  #openshift_node_kubelet_args={'pods-per-core': ['10']}
  #use HTPasswd for authentication
  openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider'}]
  openshift_master_htpasswd_file='/etc/origin/master/htpasswd'
  
  deployment_type=origin
  
  #define default sub-domain for Master node
  openshift_public_hostname=console.origin.dev.com
  openshift_master_default_subdomain=apps.origin.dev.com
  
  template_service_broker_selector={"region":"infra"}
  openshift_metrics_image_version="v3.11"
  openshift_logging_image_version="v3.11"
  openshift_logging_elasticsearch_proxy_image_version="v1.0.0"
  openshift_logging_es_nodeselector={"node-role.kubernetes.io/infra":"true"}
  logging_elasticsearch_rollout_override=false
  osm_use_cockpit=true
  
  openshift_metrics_install_metrics=True
  openshift_logging_install_logging=True
  
  openshift_master_api_port=8443
  openshift_master_console_port=8443
  
  [masters]
  devoriginapp01 openshift_schedulable=true ansible_connection=local ansible_become=yes
  
  [etcd]
  devoriginapp01 ansible_connection=local ansible_become=yes
  
  [nodes]
  devoriginapp01 openshift_schedulable=true openshift_node_group_name='node-config-master-infra'
  devoriginapp02 openshift_node_group_name='node-config-compute' openshift_schedulable=True ansible_become=yes ansible_user=origin ansible_ssh_pass=P@ssw0rd
  devoriginapp03 openshift_node_group_name='node-config-compute' openshift_schedulable=True ansible_become=yes ansible_user=origin ansible_ssh_pass=P@ssw0rd
  
  In /etc/hosts all nodes
  127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
  #::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
  x.x.x.x     DEVORIGINAPP01 console.origin.dev.com apps.origin.dev.com
  x.x.x.x     DEVORIGINAPP02
  x.x.x.x     DEVORIGINAPP03
  </code></pre>
  
  <p>For complete logs : <a href="https://drive.google.com/open?id=1-2pLXIXraRALTaRyNbe-ZvhaFtN3p8aw" rel="nofollow noreferrer">log.txt</a></p>
  
  <p>Could not find csr for nodes: devoriginapp02, devoriginapp03 but the master did it well. Anyone can help me to solve this issue ? </p>
  
  <p>Thanks.</p>