Chapter 9. Configuring and managing Red Hat OpenStack Platform with Ansible
Chapter 9. Configuring and managing Red Hat OpenStack Platform with Ansible You can use Ansible to configure and register the overcloud, and to manage containers. 9.1. Ansible-based overcloud registration Director uses Ansible-based methods to register overcloud nodes to the Red Hat Customer Portal or to a Red Hat Satellite Server. I:f you used the rhel-registration method from Red Hat OpenStack Platform versions, you must disable it and switch to the Ansible-based method. For more information, see Section 9.1.6, "Switching to the rhsm composable service" and Section 9.1.7, "rhel-registration to rhsm mappings" . In addition to the director-based registration method, you can also manually register after deployment. For more information, see Section 9.1.9, "Running Ansible-based registration manually" 9.1.1. Red Hat Subscription Manager (RHSM) composable service You can use the rhsm composable service to register overcloud nodes through Ansible. Each role in the default roles_data file contains a OS::TripleO::Services::Rhsm resource, which is disabled by default. To enable the service, register the resource to the rhsm composable service file: The rhsm composable service accepts a RhsmVars parameter, which you can use to define multiple sub-parameters relevant to your registration: You can also use the RhsmVars parameter in combination with role-specific parameters, for example, ControllerParameters , to provide flexibility when enabling specific repositories for different nodes types. RhsmVars sub-parameters Use the following sub-parameters as part of the RhsmVars parameter when you configure the rhsm composable service. For more information about the Ansible parameters that are available, see the role documentation . rhsm Description rhsm_method Choose the registration method. Either portal , satellite , or disable . rhsm_org_id The organization that you want to use for registration. To locate this ID, run sudo subscription-manager orgs from the undercloud node. Enter your Red Hat credentials at the prompt, and use the resulting Key value. For more information on your organization ID, see Understanding the Red Hat Subscription Management Organization ID . rhsm_pool_ids The subscription pool ID that you want to use. Use this parameter if you do not want to auto-attach subscriptions. To locate this ID, run sudo subscription-manager list --available --all --matches="*Red Hat OpenStack*" from the undercloud node, and use the resulting Pool ID value. rhsm_activation_key The activation key that you want to use for registration. rhsm_autosubscribe Use this parameter to attach compatible subscriptions to this system automatically. Set the value to true to enable this feature. rhsm_baseurl The base URL for obtaining content. The default URL is the Red Hat Content Delivery Network. If you use a Satellite server, change this value to the base URL of your Satellite server content repositories. rhsm_server_hostname The hostname of the subscription management service for registration. The default is the Red Hat Subscription Management hostname. If you use a Satellite server, change this value to your Satellite server hostname. rhsm_repos A list of repositories that you want to enable. rhsm_username The username for registration. If possible, use activation keys for registration. rhsm_password The password for registration. If possible, use activation keys for registration. rhsm_release Red Hat Enterprise Linux release for pinning the repositories. This is set to 9.2 for Red Hat OpenStack Platform rhsm_rhsm_proxy_hostname The hostname for the HTTP proxy. For example: proxy.example.com . rhsm_rhsm_proxy_port The port for HTTP proxy communication. For example: 8080 . rhsm_rhsm_proxy_user The username to access the HTTP proxy. rhsm_rhsm_proxy_password The password to access the HTTP proxy. Important You can use rhsm_activation_key and rhsm_repos together only if rhsm_method is set to portal . If rhsm_method is set to satellite , you can only use either rhsm_activation_key or rhsm_repos . 9.1.2. RhsmVars sub-parameters Use the following sub-parameters as part of the RhsmVars parameter when you configure the rhsm composable service. For more information about the Ansible parameters that are available, see the role documentation . rhsm Description rhsm_method Choose the registration method. Either portal , satellite , or disable . rhsm_org_id The organization that you want to use for registration. To locate this ID, run sudo subscription-manager orgs from the undercloud node. Enter your Red Hat credentials at the prompt, and use the resulting Key value. For more information on your organization ID, see Understanding the Red Hat Subscription Management Organization ID . rhsm_pool_ids The subscription pool ID that you want to use. Use this parameter if you do not want to auto-attach subscriptions. To locate this ID, run sudo subscription-manager list --available --all --matches="*Red Hat OpenStack*" from the undercloud node, and use the resulting Pool ID value. rhsm_activation_key The activation key that you want to use for registration. rhsm_autosubscribe Use this parameter to attach compatible subscriptions to this system automatically. Set the value to true to enable this feature. rhsm_baseurl The base URL for obtaining content. The default URL is the Red Hat Content Delivery Network. If you use a Satellite server, change this value to the base URL of your Satellite server content repositories. rhsm_server_hostname The hostname of the subscription management service for registration. The default is the Red Hat Subscription Management hostname. If you use a Satellite server, change this value to your Satellite server hostname. rhsm_repos A list of repositories that you want to enable. rhsm_username The username for registration. If possible, use activation keys for registration. rhsm_password The password for registration. If possible, use activation keys for registration. rhsm_release Red Hat Enterprise Linux release for pinning the repositories. This is set to 9.2 for Red Hat OpenStack Platform rhsm_rhsm_proxy_hostname The hostname for the HTTP proxy. For example: proxy.example.com . rhsm_rhsm_proxy_port The port for HTTP proxy communication. For example: 8080 . rhsm_rhsm_proxy_user The username to access the HTTP proxy. rhsm_rhsm_proxy_password The password to access the HTTP proxy. Important You can use rhsm_activation_key and rhsm_repos together only if rhsm_method is set to portal . If rhsm_method is set to satellite , you can only use either rhsm_activation_key or rhsm_repos . 9.1.3. Registering the overcloud with the rhsm composable service Create an environment file that enables and configures the rhsm composable service. Director uses this environment file to register and subscribe your nodes. Procedure Create an environment file named templates/rhsm.yml to store the configuration. Include your configuration in the environment file. For example: The resource_registry section associates the rhsm composable service with the OS::TripleO::Services::Rhsm resource, which is available on each role. The RhsmVars variable passes parameters to Ansible for configuring your Red Hat registration. To apply the rhsm composable service on a per-role basis, include your configuration in the environment file. For example, you can apply different sets of configurations to Controller nodes, Compute nodes, and Ceph Storage nodes: The ControllerParameters , ComputeParameters , and CephStorageParameters parameters each use a separate RhsmVars parameter to pass subscription details to their respective roles. Note Set the RhsmVars parameter within the CephStorageParameters parameter to use a Red Hat Ceph Storage subscription and repositories specific to Ceph Storage. Ensure the rhsm_repos parameter contains the standard Red Hat Enterprise Linux repositories instead of the Extended Update Support (EUS) repositories that Controller and Compute nodes require. Save the environment file. 9.1.4. Applying the rhsm composable service to different roles You can apply the rhsm composable service on a per-role basis. For example, you can apply different sets of configurations to Controller nodes, Compute nodes, and Ceph Storage nodes. Procedure Create an environment file named templates/rhsm.yml to store the configuration. Include your configuration in the environment file. For example: The resource_registry associates the rhsm composable service with the OS::TripleO::Services::Rhsm resource, which is available on each role. The ControllerParameters , ComputeParameters , and CephStorageParameters parameters each use a separate RhsmVars parameter to pass subscription details to their respective roles. Note Set the RhsmVars parameter within the CephStorageParameters parameter to use a Red Hat Ceph Storage subscription and repositories specific to Ceph Storage. Ensure the rhsm_repos parameter contains the standard Red Hat Enterprise Linux repositories instead of the Extended Update Support (EUS) repositories that Controller and Compute nodes require. Save the environment file. 9.1.5. Registering the overcloud to Red Hat Satellite Server Create an environment file that enables and configures the rhsm composable service to register nodes to Red Hat Satellite instead of the Red Hat Customer Portal. Procedure Create an environment file named templates/rhsm.yml to store the configuration. Include your configuration in the environment file. For example: The resource_registry associates the rhsm composable service with the OS::TripleO::Services::Rhsm resource, which is available on each role. The RhsmVars variable passes parameters to Ansible for configuring your Red Hat registration. Save the environment file. 9.1.6. Switching to the rhsm composable service The rhel-registration method runs a bash script to handle the overcloud registration. The scripts and environment files for this method are located in the core heat template collection at /usr/share/openstack-tripleo-heat-templates/extraconfig/pre_deploy/rhel-registration/ . Complete the following steps to switch from the rhel-registration method to the rhsm composable service. Procedure Exclude the rhel-registration environment files from future deployments operations. In most cases, exclude the following files: rhel-registration/environment-rhel-registration.yaml rhel-registration/rhel-registration-resource-registry.yaml If you use a custom roles_data file, ensure that each role in your roles_data file contains the OS::TripleO::Services::Rhsm composable service. For example: Add the environment file for rhsm composable service parameters to future deployment operations. This method replaces the rhel-registration parameters with the rhsm service parameters and changes the heat resource that enables the service from: To: You can also include the /usr/share/openstack-tripleo-heat-templates/environments/rhsm.yaml environment file with your deployment to enable the service. 9.1.7. rhel-registration to rhsm mappings To help transition your details from the rhel-registration method to the rhsm method, use the following table to map your parameters and values. rhel-registration rhsm / RhsmVars rhel_reg_method rhsm_method rhel_reg_org rhsm_org_id rhel_reg_pool_id rhsm_pool_ids rhel_reg_activation_key rhsm_activation_key rhel_reg_auto_attach rhsm_autosubscribe rhel_reg_sat_url rhsm_satellite_url rhel_reg_repos rhsm_repos rhel_reg_user rhsm_username rhel_reg_password rhsm_password rhel_reg_release rhsm_release rhel_reg_http_proxy_host rhsm_rhsm_proxy_hostname rhel_reg_http_proxy_port rhsm_rhsm_proxy_port rhel_reg_http_proxy_username rhsm_rhsm_proxy_user rhel_reg_http_proxy_password rhsm_rhsm_proxy_password 9.1.8. Deploying the overcloud with the rhsm composable service Deploy the overcloud with the rhsm composable service so that Ansible controls the registration process for your overcloud nodes. Procedure Include rhsm.yml environment file with the openstack overcloud deploy command: This enables the Ansible configuration of the overcloud and the Ansible-based registration. Wait until the overcloud deployment completes. Check the subscription details on your overcloud nodes. For example, log in to a Controller node and run the following commands: 9.1.9. Running Ansible-based registration manually You can perform manual Ansible-based registration on a deployed overcloud with the dynamic inventory script on the director node. Use this script to define node roles as host groups and then run a playbook against them with ansible-playbook . Use the following example playbook to register Controller nodes manually. Procedure Create a playbook that uses the redhat_subscription modules to register your nodes. For example, the following playbook applies to Controller nodes: This play contains three tasks: Register the node. Disable any auto-enabled repositories. Enable only the repositories relevant to the Controller node. The repositories are listed with the repos variable. After you deploy the overcloud, you can run the following command so that Ansible executes the playbook ( ansible-osp-registration.yml ) against your overcloud: This command performs the following actions: Runs the dynamic inventory script to get a list of host and their groups. Applies the playbook tasks to the nodes in the group defined in the hosts parameter of the playbook, which in this case is the Controller group. 9.2. Configuring the overcloud with Ansible Ansible is the main method to apply the overcloud configuration. This chapter provides information about how to interact with the overcloud Ansible configuration. Although director generates the Ansible playbooks automatically, it is a good idea to familiarize yourself with Ansible syntax. For more information about using Ansible, see https://docs.ansible.com/ . Note Ansible also uses the concept of roles, which are different to OpenStack Platform director roles. Ansible roles form reusable components of playbooks, whereas director roles contain mappings of OpenStack services to node types. 9.2.1. Ansible-based overcloud configuration (config-download) The config-download feature is the method that director uses to configure the overcloud. Director uses config-download in conjunction with OpenStack Orchestration (heat) to generate the software configuration and apply the configuration to each overcloud node. Although heat creates all deployment data from SoftwareDeployment resources to perform the overcloud installation and configuration, heat does not apply any of the configuration. Heat only provides the configuration data through the heat API. As a result, when you run the openstack overcloud deploy command, the following process occurs: Director creates a new deployment plan based on openstack-tripleo-heat-templates and includes any environment files and parameters to customize the plan. Director uses heat to interpret the deployment plan and create the overcloud stack and all descendant resources. This includes provisioning nodes with the OpenStack Bare Metal service (ironic). Heat also creates the software configuration from the deployment plan. Director compiles the Ansible playbooks from this software configuration. Director generates a temporary user ( tripleo-admin ) on the overcloud nodes specifically for Ansible SSH access. Director downloads the heat software configuration and generates a set of Ansible playbooks using heat outputs. Director applies the Ansible playbooks to the overcloud nodes using ansible-playbook . 9.2.2. config-download working directory The ansible-playbook command creates an Ansible project directory, default name ~/config-download/overcloud . This project directory stores downloaded software configuration from heat. It includes all Ansible-related files which you need to run ansible-playbook to configure the overcloud. The contents of the directory include: tripleo-ansible-inventory.yaml - Ansible inventory file containing hosts and vars for all the overcloud nodes. ansible.log - Log file from the most recent run of ansible-playbook . ansible.cfg - Configuration file used when running ansible-playbook . ansible-playbook-command.sh - Executable script used to rerun ansible-playbook . ssh_private_key - Private ssh key used to access the overcloud nodes. Reproducing ansible-playbook After the project directory is created, run the ansible-playbook-command.sh command to reproduce the deployment. You can run the script with additional arguments, such as check mode --check , limiting hosts --limit , and overriding variables -e . 9.2.3. Checking config-download log During the config-download process, Ansible creates a log file, named ansible.log , in the /home/stack directory on the undercloud. Procedure View the log with the less command: 9.2.4. Performing Git operations on the working directory The config-download working directory is a local Git repository. Every time a deployment operation runs, director adds a Git commit to the working directory with the relevant changes. You can perform Git operations to view configuration for the deployment at different stages and compare the configuration with different deployments. Be aware of the limitations of the working directory. For example, if you use Git to revert to a version of the config-download working directory, this action affects only the configuration in the working directory. It does not affect the following configurations: The overcloud data schema: Applying a version of the working directory software configuration does not undo data migration and schema changes. The hardware layout of the overcloud: Reverting to software configuration does not undo changes related to overcloud hardware, such as scaling up or down. The heat stack: Reverting to earlier revisions of the working directory has no effect on the configuration stored in the heat stack. The heat stack creates a new version of the software configuration that applies to the overcloud. To make permanent changes to the overcloud, modify the environment files applied to the overcloud stack before you rerun the openstack overcloud deploy command. Complete the following steps to compare different commits of the config-download working directory. Procedure Change to the config-download working directory for your overcloud, usually named overcloud : Run the git log command to list the commits in your working directory. You can also format the log output to show the date: By default, the most recent commit appears first. Run the git diff command against two commit hashes to see all changes between the deployments: 9.2.5. Deployment methods that use config-download There are four main methods that use config-download in the context of an overcloud deployment: Standard deployment Run the openstack overcloud deploy command to automatically run the configuration stage after the provisioning stage. This is the default method when you run the openstack overcloud deploy command. Separate provisioning and configuration Run the openstack overcloud deploy command with specific options to separate the provisioning and configuration stages. Run the ansible-playbook-command.sh script after a deployment Run the openstack overcloud deploy command with combined or separate provisioning and configuration stages, then run the ansible-playbook-command.sh script supplied in the config-download working directory to re-apply the configuration stage. Provision nodes, manually create config-download, and run Ansible Run the openstack overcloud deploy command with a specific option to provision nodes, then run the ansible-playbook command with the deploy_steps_playbook.yaml playbook. 9.2.6. Running config-download on a standard deployment The default method for executing config-download is to run the openstack overcloud deploy command. This method suits most environments. Prerequisites A successful undercloud installation. Overcloud nodes ready for deployment. Heat environment files that are relevant to your specific overcloud customization. Procedure Log in to the undercloud host as the stack user. Source the stackrc file: Run the deployment command. Include any environment files that you require for your overcloud: Wait until the deployment process completes. During the deployment process, director generates the config-download files in a ~/config-download/overcloud working directory. After the deployment process finishes, view the Ansible playbooks in the working directory to see the tasks director executed to configure the overcloud. 9.2.7. Running config-download with separate provisioning and configuration The openstack overcloud deploy command runs the heat-based provisioning process and then the config-download configuration process. You can also run the deployment command to execute each process individually. Use this method to provision your overcloud nodes as a distinct process so that you can perform any manual pre-configuration tasks on the nodes before you run the overcloud configuration process. Prerequisites A successful undercloud installation. Overcloud nodes ready for deployment. Heat environment files that are relevant to your specific overcloud customization. Procedure Log in to the undercloud host as the stack user. Source the stackrc file: Run the deployment command with the --stack-only option. Include any environment files you require for your overcloud: Wait until the provisioning process completes. Enable SSH access from the undercloud to the overcloud for the tripleo-admin user. The config-download process uses the tripleo-admin user to perform the Ansible-based configuration: Perform any manual pre-configuration tasks on nodes. If you use Ansible for configuration, use the tripleo-admin user to access the nodes. Run the deployment command with the --config-download-only option. Include any environment files required for your overcloud: Wait until the configuration process completes. During the configuration stage, director generates the config-download files in a ~/config-download/overcloud working directory. After the deployment process finishes, view the Ansible playbooks in the working directory to see the tasks director executed to configure the overcloud. 9.2.8. Running config-download with the ansible-playbook-command.sh script When you deploy the overcloud, either with the standard method or a separate provisioning and configuration process, director generates a working directory in ~/config-download/overcloud . This directory contains the playbooks and scripts necessary to run the configuration process again. Prerequisites An overcloud deployed with the one of the following methods: Standard method that combines provisioning and configuration process. Separate provisioning and configuration processes. Procedure Log in to the undercloud host as the stack user. Run the ansible-playbook-command.sh script. You can pass additional Ansible arguments to this script, which are then passed unchanged to the ansible-playbook command. This makes it possible to take advantage of Ansible features, such as check mode ( --check ), limiting hosts ( --limit ), or overriding variables ( -e ). For example: Warning When --limit is used to deploy at scale, only hosts included in the execution are added to the SSH known_hosts file across the nodes. Therefore, some operations, such as live migration, may not work across nodes that are not in the known_hosts file. Note To ensure that the /etc/hosts file, on all nodes, is up-to-date, run the following command as the stack user: Wait until the configuration process completes. Additional information The working directory contains a playbook called deploy_steps_playbook.yaml , which manages the overcloud configuration tasks. To view this playbook, run the following command: The playbook uses various task files contained in the working directory. Some task files are common to all OpenStack Platform roles and some are specific to certain OpenStack Platform roles and servers. The working directory also contains sub-directories that correspond to each role that you define in your overcloud roles_data file. For example: Each OpenStack Platform role directory also contains sub-directories for individual servers of that role type. The directories use the composable role hostname format: The Ansible tasks in deploy_steps_playbook.yaml are tagged. To see the full list of tags, use the CLI option --list-tags with ansible-playbook : Then apply tagged configuration using the --tags , --skip-tags , or --start-at-task with the ansible-playbook-command.sh script: When you run the config-download playbooks against the overcloud, you might receive a message regarding the SSH fingerprint for each host. To avoid these messages, include --ssh-common-args="-o StrictHostKeyChecking=no" when you run the ansible-playbook-command.sh script: 9.2.9. Running config-download with manually created playbooks You can create your own config-download files outside of the standard workflow. For example, you can run the openstack overcloud deploy command with the --stack-only option to provision the nodes, and then manually apply the Ansible configuration separately. Prerequisites A successful undercloud installation. Overcloud nodes ready for deployment. Heat environment files that are relevant to your specific overcloud customization. Procedure Log in to the undercloud host as the stack user. Source the stackrc file: Run the deployment command with the --stack-only option. Include any environment files required for your overcloud: Wait until the provisioning process completes. Enable SSH access from the undercloud to the overcloud for the tripleo-admin user. The config-download process uses the tripleo-admin user to perform the Ansible-based configuration: Generate the config-download files: --stack specifies the name of the overcloud. --stack-only ensures that the command only deploys the heat stack and skips any software configuration. --config-dir specifies the location of the config-download files. Change to the directory that contains your config-download files: Generate a static inventory file: Replace <overcloud> with the name of your overcloud. Use the ~/overcloud-deploy/overcloud/config-download/overcloud files and the static inventory file to perform a configuration. To execute the deployment playbook, run the ansible-playbook command: Note When you run the config-download/overcloud playbooks against the overcloud, you might receive a message regarding the SSH fingerprint for each host. To avoid these messages, include --ssh-common-args="-o StrictHostKeyChecking=no" in your ansible-playbook command: Wait until the configuration process completes. Generate an overcloudrc file manually from the ansible-based configuration: Manually set the deployment status to success: Replace <overcloud> with the name of your overcloud. Note The ~/overcloud-deploy/overcloud/config-download/overcloud/ directory contains a playbook called deploy_steps_playbook.yaml . The playbook uses various task files contained in the working directory. Some task files are common to all Red Hat OpenStack Platform (RHOSP) roles and some are specific to certain RHOSP roles and servers. The ~/overcloud-deploy/overcloud/config-download/overcloud/ directory also contains sub-directories that correspond to each role that you define in your overcloud roles_data file. Each RHOSP role directory also contains sub-directories for individual servers of that role type. The directories use the composable role hostname format, for example Controller/overcloud-controller-0 . The Ansible tasks in deploy_steps_playbook.yaml are tagged. To see the full list of tags, use the CLI option --list-tags with ansible-playbook : You can apply tagged configuration using the --tags , --skip-tags , or --start-at-task with the ansible-playbook-command.sh script: 9.2.10. Limitations of config-download The config-download feature has some limitations: When you use ansible-playbook CLI arguments such as --tags , --skip-tags , or --start-at-task , do not run or apply deployment configuration out of order. These CLI arguments are a convenient way to rerun previously failed tasks or to iterate over an initial deployment. However, to guarantee a consistent deployment, you must run all tasks from deploy_steps_playbook.yaml in order. You can not use the --start-at-task arguments for certain tasks that use a variable in the task name. For example, the --start-at-task arguments does not work for the following Ansible task: If your overcloud deployment includes a director-deployed Ceph Storage cluster, you cannot skip step1 tasks when you use the --check option unless you also skip external_deploy_steps tasks. You can set the number of parallel Ansible tasks with the --forks option. However, the performance of config-download operations degrades after 25 parallel tasks. For this reason, do not exceed 25 with the --forks option. 9.2.11. config-download top level files The following file are important top level files within a config-download working directory. Ansible configuration and execution The following files are specific to configuring and executing Ansible within the config-download working directory. ansible.cfg Configuration file used when running ansible-playbook . ansible.log Log file from the last run of ansible-playbook . ansible-errors.json JSON structured file that contains any deployment errors. ansible-playbook-command.sh Executable script to rerun the ansible-playbook command from the last deployment operation. ssh_private_key Private SSH key that Ansible uses to access the overcloud nodes. tripleo-ansible-inventory.yaml Ansible inventory file that contains hosts and variables for all the overcloud nodes. overcloud-config.tar.gz Archive of the working directory. Playbooks The following files are playbooks within the config-download working directory. deploy_steps_playbook.yaml Main deployment steps. This playbook performs the main configuration operations for your overcloud. pre_upgrade_rolling_steps_playbook.yaml Pre upgrade steps for major upgrade upgrade_steps_playbook.yaml Major upgrade steps. post_upgrade_steps_playbook.yaml Post upgrade steps for major upgrade. update_steps_playbook.yaml Minor update steps. fast_forward_upgrade_playbook.yaml Fast forward upgrade tasks. Use this playbook only when you want to upgrade from one long-life version of Red Hat OpenStack Platform to the . 9.2.12. config-download tags The playbooks use tagged tasks to control the tasks that they apply to the overcloud. Use tags with the ansible-playbook CLI arguments --tags or --skip-tags to control which tasks to execute. The following list contains information about the tags that are enabled by default: facts Fact gathering operations. common_roles Ansible roles common to all nodes. overcloud All plays for overcloud deployment. pre_deploy_steps Deployments that happen before the deploy_steps operations. host_prep_steps Host preparation steps. deploy_steps Deployment steps. post_deploy_steps Steps that happen after the deploy_steps operations. external All external deployment tasks. external_deploy_steps External deployment tasks that run on the undercloud only. 9.2.13. config-download deployment steps The deploy_steps_playbook.yaml playbook configures the overcloud. This playbook applies all software configuration that is necessary to deploy a full overcloud based on the overcloud deployment plan. This section contains a summary of the different Ansible plays used within this playbook. The play names in this section are the same names that are used within the playbook and that are displayed in the ansible-playbook output. This section also contains information about the Ansible tags that are set on each play. Gather facts from undercloud Fact gathering for the undercloud node. Tags: facts Gather facts from overcloud Fact gathering for the overcloud nodes. Tags: facts Load global variables Loads all variables from global_vars.yaml . Tags: always Common roles for TripleO servers Applies common Ansible roles to all overcloud nodes, including tripleo-bootstrap for installing bootstrap packages, and tripleo-ssh-known-hosts for configuring ssh known hosts. Tags: common_roles Overcloud deploy step tasks for step 0 Applies tasks from the deploy_steps_tasks template interface. Tags: overcloud , deploy_steps Server deployments Applies server-specific heat deployments for configuration such as networking and hieradata. Includes NetworkDeployment, <Role>Deployment, <Role>AllNodesDeployment, etc. Tags: overcloud , pre_deploy_steps Host prep steps Applies tasks from the host_prep_steps template interface. Tags: overcloud , host_prep_steps External deployment step [1,2,3,4,5] Applies tasks from the external_deploy_steps_tasks template interface. Ansible runs these tasks only against the undercloud node. Tags: external , external_deploy_steps Overcloud deploy step tasks for [1,2,3,4,5] Applies tasks from the deploy_steps_tasks template interface. Tags: overcloud , deploy_steps Overcloud common deploy step tasks [1,2,3,4,5] Applies the common tasks performed at each step, including puppet host configuration, container-puppet.py , and tripleo-container-manage (container configuration and management). Tags: overcloud , deploy_steps Server Post Deployments Applies server specific heat deployments for configuration performed after the 5-step deployment process. Tags: overcloud , post_deploy_steps External deployment Post Deploy tasks Applies tasks from the external_post_deploy_steps_tasks template interface. Ansible runs these tasks only against the undercloud node. Tags: external , external_deploy_steps 9.3. Managing containers with Ansible Red Hat OpenStack Platform 17.1 uses the tripleo_container_manage Ansible role to perform management operations on containers. You can also write custom playbooks to perform specific container management operations: Collect the container configuration data that heat generates. The tripleo_container_manage role uses this data to orchestrate container deployment. Start containers. Stop containers. Update containers. Delete containers. Run a container with a specific configuration. Although director performs container management automatically, you might want to customize a container configuration, or apply a hotfix to a container without redeploying the overcloud. Note This role supports only Podman container management. 9.3.1. tripleo-container-manage role defaults and variables The following excerpt shows the defaults and variables for the tripleo_container_manage Ansible role. 9.3.2. tripleo-container-manage molecule scenarios Molecule is used to test the tripleo_container_manage role. The following shows a molecule default inventory: Usage Red Hat OpenStack 17.1 supports only Podman in this role. Docker support is on the roadmap. The Molecule Ansible role performs the following tasks: Collect container configuration data, generated by the TripleO Heat Templates. This data is used as a source of truth. If a container is already managed by Molecule , no matter its present state, the configuration data will reconfigure the container as needed. Manage systemd shutdown files. It creates the TripleO Container systemd service, required for service ordering when shutting down or starting a node. It also manages the netns-placeholder service. Delete containers that are nonger needed or that require reconfiguration. It uses a custom filter, named needs_delete() which has a set of rules to determine if the container needs to be deleted. A container will not be deleted if, the container is not managed by tripleo_ansible or the container config_id does not match the input ID. A container will be deleted, if the container has no config_data or the container has config_data which does not match data in input. Note that when a container is removed, the role also disables and removes the systemd services and healtchecks. Create containers in a specific order defined by start_order container config, where the default is 0. If the container is an exec , a dedicated playbook for execs is run, using async so multiple execs can be run at the same time. Otherwise, the podman_container is used, in async, to create the containers. If the container has a restart policy , systemd service is configured. If the container has a healthcheck script, systemd healthcheck service is configured. Note tripleo_container_manage_concurrency parameter is set to 1 by default, and putting a value higher than 2 can expose issues with Podman locks. Example of a playbook: 9.3.3. tripleo_container_manage role variables The tripleo_container_manage Ansible role contains the following variables: Table 9.1. Role variables Name Default value Description tripleo_container_manage_check_puppet_config false Use this variable if you want Ansible to check Puppet container configurations. Ansible can identify updated container configuration using the configuration hash. If a container has a new configuration from Puppet, set this variable to true so that Ansible can detect the new configuration and add the container to the list of containers that Ansible must restart. tripleo_container_manage_cli podman Use this variable to set the command line interface that you want to use to manage containers. The tripleo_container_manage role supports only Podman. tripleo_container_manage_concurrency 1 Use this variable to set the number of containers that you want to manage concurrently. tripleo_container_manage_config /var/lib/tripleo-config/ Use this variable to set the path to the container configuration directory. tripleo_container_manage_config_id tripleo Use this variable to set the ID of a specific configuration step. For example, set this value to tripleo_step2 to manage containers for step two of the deployment. tripleo_container_manage_config_patterns *.json Use this variable to set the bash regular expression that identifies configuration files in the container configuration directory. tripleo_container_manage_debug false Use this variable to enable or disable debug mode. Run the tripleo_container_manage role in debug mode if you want to run a container with a specific one-time configuration, to output the container commands that manage the lifecycle of containers, or to run no-op container management operations for testing and verification purposes. tripleo_container_manage_healthcheck_disable false Use this variable to enable or disable healthchecks. tripleo_container_manage_log_path /var/log/containers/stdouts Use this variable to set the stdout log path for containers. tripleo_container_manage_systemd_order false Use this variable to enable or disable systemd shutdown ordering with Ansible. tripleo_container_manage_systemd_teardown true Use this variable to trigger the cleanup of obsolete containers. tripleo_container_manage_config_overrides {} Use this variable to override any container configuration. This variable takes a dictionary of values where each key is the container name and the parameters that you want to override, for example, the container image or user. This variable does not write custom overrides to the JSON container configuration files and any new container deployments, updates, or upgrades revert to the content of the JSON configuration file. tripleo_container_manage_valid_exit_code [] Use this variable to check if a container returns an exit code. This value must be a list, for example, [0,3] . 9.3.4. tripleo-container-manage healthchecks Until Red Hat OpenStack 17.1, container healthcheck was implemented by a systemd timer which would run podman exec to determine if a given container was healthy. Now, it uses the native healthcheck interface in Podman which is easier to integrate and consume. To check if a container (for example, keystone) is healthy, run the following command: The return code should be 0 and "healthy" . 9.3.5. tripleo-container-manage debug The tripleo_container_manage Ansible role allows you to perform specific actions on a given container. This can be used to: Run a container with a specific one-off configuration. Output the container commands to manage containers lifecycle. Output the changes made on containers by Ansible. Note To manage a single container, you need to know two things: At which step during the overcloud deployment was the container deployed. The name of the generated JSON file containing the container configuration. The following is an example of a playbook to manage HAproxy container at step 1 which overrides the image setting: If Ansible is run in check mode , no container is removed or created, however at the end of the playbook run a list of commands is displayed to show the possible outcome of the playbook. This is useful for debugging purposes. Adding the diff mode will show the changes that would have been made on containers by Ansible. The tripleo_container_manage_clean_orphans parameter is optional. It can be set to false meaning orphaned containers, with a specific config_id , will not be removed. It can be used to manage a single container without impacting other running containers with same config_id . The tripleo_container_manage_config_overrides parameter is optional and can be used to override a specific container attribute, for example the image or the container user. The parameter creates dictionary with container name and the parameters to override. These parameters have to exist and they define the container configuration in TripleO Heat Templates. Note the dictionary does not update the overrides in the JSON file so if an update or upgrade is executed, the container will be re-configured with the configuration in the JSON file.
resource_registry: OS::TripleO::Services::Rhsm: /usr/share/openstack-tripleo-heat-templates/deployment/rhsm/rhsm-baremetal-ansible.yaml parameter_defaults: RhsmVars: rhsm_repos: - rhel-9-for-x86_64-baseos-eus-rpms - rhel-9-for-x86_64-appstream-eus-rpms - rhel-9-for-x86_64-highavailability-eus-rpms ... rhsm_username: "myusername" rhsm_password: "p@55w0rd!" rhsm_org_id: "1234567" rhsm_release: 9.2 resource_registry: OS::TripleO::Services::Rhsm: /usr/share/openstack-tripleo-heat-templates/deployment/rhsm/rhsm-baremetal-ansible.yaml parameter_defaults: RhsmVars: rhsm_repos: - rhel-9-for-x86_64-baseos-eus-rpms - rhel-9-for-x86_64-appstream-eus-rpms - rhel-9-for-x86_64-highavailability-eus-rpms ... rhsm_username: "myusername" rhsm_password: "p@55w0rd!" rhsm_org_id: "1234567" rhsm_pool_ids: "1a85f9223e3d5e43013e3d6e8ff506fd" rhsm_method: "portal" rhsm_release: 9.2 parameter_defaults: ControllerParameters: RhsmVars: rhsm_repos: - rhel-9-for-x86_64-baseos-eus-rpms - rhel-9-for-x86_64-appstream-eus-rpms - rhel-9-for-x86_64-highavailability-eus-rpms - openstack-17.1-for-rhel-9-x86_64-rpms - fast-datapath-for-rhel-9-x86_64-rpms - rhceph-6-tools-for-rhel-9-x86_64-rpms rhsm_username: "myusername" rhsm_password: "p@55w0rd!" rhsm_org_id: "1234567" rhsm_pool_ids: "55d251f1490556f3e75aa37e89e10ce5" rhsm_method: "portal" rhsm_release: 9.2 ComputeParameters: RhsmVars: rhsm_repos: - rhel-9-for-x86_64-baseos-eus-rpms - rhel-9-for-x86_64-appstream-eus-rpms - rhel-9-for-x86_64-highavailability-eus-rpms - openstack-17.1-for-rhel-9-x86_64-rpms - rhceph-6-tools-for-rhel-9-x86_64-rpms - fast-datapath-for-rhel-9-x86_64-rpms rhsm_username: "myusername" rhsm_password: "p@55w0rd!" rhsm_org_id: "1234567" rhsm_pool_ids: "55d251f1490556f3e75aa37e89e10ce5" rhsm_method: "portal" rhsm_release: 9.2 CephStorageParameters: RhsmVars: rhsm_repos: - rhel-9-for-x86_64-baseos-rpms - rhel-9-for-x86_64-appstream-rpms - rhel-9-for-x86_64-highavailability-rpms - openstack-17.1-deployment-tools-for-rhel-9-x86_64-rpms rhsm_username: "myusername" rhsm_password: "p@55w0rd!" rhsm_org_id: "1234567" rhsm_pool_ids: "68790a7aa2dc9dc50a9bc39fabc55e0d" rhsm_method: "portal" rhsm_release: 9.2 resource_registry: OS::TripleO::Services::Rhsm: /usr/share/openstack-tripleo-heat-templates/deployment/rhsm/rhsm-baremetal-ansible.yaml parameter_defaults: ControllerParameters: RhsmVars: rhsm_repos: - rhel-9-for-x86_64-baseos-eus-rpms - rhel-9-for-x86_64-appstream-eus-rpms - rhel-9-for-x86_64-highavailability-eus-rpms - openstack-17.1-for-rhel-9-x86_64-rpms - fast-datapath-for-rhel-9-x86_64-rpms - rhceph-6-tools-for-rhel-9-x86_64-rpms rhsm_username: "myusername" rhsm_password: "p@55w0rd!" rhsm_org_id: "1234567" rhsm_pool_ids: "55d251f1490556f3e75aa37e89e10ce5" rhsm_method: "portal" rhsm_release: 9.2 ComputeParameters: RhsmVars: rhsm_repos: - rhel-9-for-x86_64-baseos-eus-rpms - rhel-9-for-x86_64-appstream-eus-rpms - rhel-9-for-x86_64-highavailability-eus-rpms - openstack-17.1-for-rhel-9-x86_64-rpms - rhceph-6-tools-for-rhel-9-x86_64-rpms - fast-datapath-for-rhel-9-x86_64-rpms rhsm_username: "myusername" rhsm_password: "p@55w0rd!" rhsm_org_id: "1234567" rhsm_pool_ids: "55d251f1490556f3e75aa37e89e10ce5" rhsm_method: "portal" rhsm_release: 9.2 CephStorageParameters: RhsmVars: rhsm_repos: - rhel-9-for-x86_64-baseos-rpms - rhel-9-for-x86_64-appstream-rpms - rhel-9-for-x86_64-highavailability-rpms - openstack-17.1-deployment-tools-for-rhel-9-x86_64-rpms rhsm_username: "myusername" rhsm_password: "p@55w0rd!" rhsm_org_id: "1234567" rhsm_pool_ids: "68790a7aa2dc9dc50a9bc39fabc55e0d" rhsm_method: "portal" rhsm_release: 9.2 resource_registry: OS::TripleO::Services::Rhsm: /usr/share/openstack-tripleo-heat-templates/deployment/rhsm/rhsm-baremetal-ansible.yaml parameter_defaults: RhsmVars: rhsm_activation_key: "myactivationkey" rhsm_method: "satellite" rhsm_org_id: "ACME" rhsm_server_hostname: "satellite.example.com" rhsm_baseurl: "https://satellite.example.com/pulp/repos" rhsm_release: 9.2 - name: Controller description: | Controller role that has all the controller services loaded and handles Database, Messaging and Network functions. CountDefault: 1 ServicesDefault: - OS::TripleO::Services::Rhsm resource_registry: OS::TripleO::NodeExtraConfig: rhel-registration.yaml resource_registry: OS::TripleO::Services::Rhsm: /usr/share/openstack-tripleo-heat-templates/deployment/rhsm/rhsm-baremetal-ansible.yaml openstack overcloud deploy <other cli args> -e ~/templates/rhsm.yaml sudo subscription-manager status sudo subscription-manager list --consumed --- - name: Register Controller nodes hosts: Controller become: yes vars: repos: - rhel-9-for-x86_64-baseos-eus-rpms - rhel-9-for-x86_64-appstream-eus-rpms - rhel-9-for-x86_64-highavailability-eus-rpms - openstack-17.1-for-rhel-9-x86_64-rpms - fast-datapath-for-rhel-9-x86_64-rpms tasks: - name: Register system redhat_subscription: username: myusername password: p@55w0rd! org_id: 1234567 release: 9.2 pool_ids: 1a85f9223e3d5e43013e3d6e8ff506fd - name: Disable all repos command: "subscription-manager repos --disable *" - name: Enable Controller node repos command: "subscription-manager repos --enable {{ item }}" with_items: "{{ repos }}" ansible-playbook -i /usr/bin/tripleo-ansible-inventory ansible-osp-registration.yml ./ansible-playbook-command.sh ./ansible-playbook-command.sh --check less ~/ansible.log cd ~/config-download/overcloud git log --format=format:"%h%x09%cd%x09" a7e9063 Mon Oct 8 21:17:52 2018 +1000 dfb9d12 Fri Oct 5 20:23:44 2018 +1000 d0a910b Wed Oct 3 19:30:16 2018 +1000 git diff a7e9063 dfb9d12 source ~/stackrc openstack overcloud deploy --templates -e environment-file1.yaml -e environment-file2.yaml source ~/stackrc openstack overcloud deploy --templates -e environment-file1.yaml -e environment-file2.yaml --stack-only openstack overcloud admin authorize openstack overcloud deploy --templates -e environment-file1.yaml -e environment-file2.yaml --config-download-only ./ansible-playbook-command.sh --limit Controller (undercloud)USD cd /home/stack/overcloud-deploy/overcloud/config-download/overcloud (undercloud)USD ANSIBLE_REMOTE_USER="tripleo-admin" ansible allovercloud -i /home/stack/overcloud-deploy/overcloud/tripleo-ansible-inventory.yaml -m include_role -a name=tripleo_hosts_entries -e @global_vars.yaml less deploy_steps_playbook.yaml ls Controller/ ls Controller/overcloud-controller-0 ansible-playbook -i tripleo-ansible-inventory.yaml --list-tags deploy_steps_playbook.yaml ./ansible-playbook-command.sh --tags overcloud ./ansible-playbook-command.sh --tags overcloud --ssh-common-args="-o StrictHostKeyChecking=no" source ~/stackrc openstack overcloud deploy --templates -e environment-file1.yaml -e environment-file2.yaml --stack-only openstack overcloud admin authorize openstack overcloud deploy --stack overcloud --stack-only --config-dir ~/overcloud-deploy/overcloud/config-download/overcloud/ cd ~/config-download tripleo-ansible-inventory --stack <overcloud> --ansible_ssh_user tripleo-admin --static-yaml-inventory inventory.yaml ansible-playbook -i inventory.yaml -e gather_facts=true -e @global_vars.yaml --private-key ~/.ssh/id_rsa --become ~/overcloud-deploy/overcloud/config-download/overcloud/deploy_steps_playbook.yaml ansible-playbook -i inventory.yaml -e gather_facts=true -e @global_vars.yaml --private-key ~/.ssh/id_rsa --ssh-common-args="-o StrictHostKeyChecking=no" --become --tags overcloud ~/overcloud-deploy/overcloud/config-download/overcloud/deploy_steps_playbook.yaml openstack action execution run --save-result --run-sync tripleo.deployment.overcloudrc '{"container":"overcloud"}' | jq -r '.["result"]["overcloudrc.v3"]' > overcloudrc.v3 openstack workflow execution create tripleo.deployment.v1.set_deployment_status_success '{"plan": "<overcloud>"}' ansible-playbook -i tripleo-ansible-inventory.yaml --list-tags deploy_steps_playbook.yaml ansible-playbook -i inventory.yaml -e gather_facts=true -e @global_vars.yaml --private-key ~/.ssh/id_rsa --become --tags overcloud ~/overcloud-deploy/overcloud/config-download/overcloud/deploy_steps_playbook.yaml - name: Run puppet host configuration for step {{ step }} All variables intended for modification should place placed in this file. tripleo_container_manage_hide_sensitive_logs: '{{ hide_sensitive_logs | default(true) }}' tripleo_container_manage_debug: '{{ ((ansible_verbosity | int) >= 2) | bool }}' tripleo_container_manage_clean_orphans: true All variables within this role should have a prefix of "tripleo_container_manage" tripleo_container_manage_check_puppet_config: false tripleo_container_manage_cli: podman tripleo_container_manage_concurrency: 1 tripleo_container_manage_config: /var/lib/tripleo-config/ tripleo_container_manage_config_id: tripleo tripleo_container_manage_config_overrides: {} tripleo_container_manage_config_patterns: '*.json' Some containers where Puppet is run, can take up to 10 minutes to finish in slow environments. tripleo_container_manage_create_retries: 120 Default delay is 5s so 120 retries makes a timeout of 10 minutes which is what we have observed a necessary value for nova and neutron db-sync execs. tripleo_container_manage_exec_retries: 120 tripleo_container_manage_healthcheck_disabled: false tripleo_container_manage_log_path: /var/log/containers/stdouts tripleo_container_manage_systemd_teardown: true hosts: all: hosts: instance: ansible_host: localhost ansible_connection: local ansible_distribution: centos8 - name: Manage step_1 containers using tripleo-ansible block: - name: "Manage containers for step 1 with tripleo-ansible" include_role: name: tripleo_container_manage vars: tripleo_container_manage_config: "/var/lib/tripleo-config/container-startup-config/step_1" tripleo_container_manage_config_id: "tripleo_step1" sudo podman healthcheck run keystone "Healthcheck": { "Status": "healthy", "FailingStreak": 0, "Log": [ { "Start": "2020-04-14T18:48:57.272180578Z", "End": "2020-04-14T18:48:57.806659104Z", "ExitCode": 0, "Output": "" }, (...) ] } - hosts: localhost become: true tasks: - name: Manage step_1 containers using tripleo-ansible block: - name: "Manage HAproxy container at step 1 with tripleo-ansible" include_role: name: tripleo_container_manage vars: tripleo_container_manage_config_patterns: 'haproxy.json' tripleo_container_manage_config: "/var/lib/tripleo-config/container-startup-config/step_1" tripleo_container_manage_config_id: "tripleo_step1" tripleo_container_manage_clean_orphans: false tripleo_container_manage_config_overrides: haproxy: image: quay.io/tripleomastercentos9/centos-binary-haproxy:hotfix ansible-playbook haproxy.yaml --check ansible-playbook haproxy.yaml --check --diff