Chapter 6. Using Quality of Service (QoS) policies to manage data traffic
Chapter 6. Using Quality of Service (QoS) policies to manage data traffic You can offer varying service levels for VM instances by using quality of service (QoS) policies to apply rate limits to egress and ingress traffic in Red Hat OpenStack Services on OpenShift (RHOSO) environments. You can apply QoS policies to individual ports, or apply QoS policies to a project network, where ports with no specific policy attached inherit the policy. Note Internal network owned ports, such as DHCP and internal router ports, are excluded from network policy application. You can apply, modify, or remove QoS policies dynamically. However, for guaranteed minimum bandwidth QoS policies, you can only apply modifications when there are no instances that use any of the ports the policy is assigned to. 6.1. QoS rules You can configure the following rule types to define a quality of service (QoS) policy in the Red Hat OpenStack Services on OpenShift (RHOSO) Networking service (neutron): Minimum bandwidth ( minimum_bandwidth ) Provides minimum bandwidth constraints on certain types of traffic. If implemented, best efforts are made to provide no less than the specified bandwidth to each port on which the rule is applied. Bandwidth limit ( bandwidth_limit ) Provides bandwidth limitations on networks, ports, floating IPs (FIPs), and router gateway IPs. If implemented, any traffic that exceeds the specified rate is dropped. DSCP marking ( dscp_marking ) Marks network traffic with a differentiated services code point (DSCP) value. Minimum packet rate ( minimum-packet-rate ) Provides minimum rate of packet transmission constraints on certain types of traffic. If implemented, best efforts are made to provide no less than the specified rate of packet transmission to each port on which the rule is applied. Currently, only placement enforcement is supported. QoS policies can be enforced in various contexts, including virtual machine instance placements, floating IP assignments, and gateway IP assignments. Depending on the enforcement context and on the mechanism driver you use, a QoS rule affects egress traffic (upload from instance), ingress traffic (download to instance), or both. Note In ML2/OVN deployments, you can enable minimum bandwidth and bandwidth limit egress policies for hardware offloaded ports. You cannot enable ingress policies for hardware offloaded ports. For more information, see Section 6.2, "Configuring the Networking service for QoS policies" . Table 6.1. Supported traffic direction by driver (all QoS rule types) Rule [1] Supported traffic direction by mechanism driver ML2/SR-IOV ML2/OVN Minimum bandwidth Egress only Currently, no support [2] Bandwidth limit Egress only [3] Egress and ingress DSCP marking N/A Egress only [4] [1] RHOSO does not support QoS for trunk ports. [2] https://bugzilla.redhat.com/show_bug.cgi?id=2060310 [3] The mechanism drivers ignore the max-burst-kbits parameter because they do not support it. [4] ML2/OVN does not support DSCP marking on tunneled protocols. Table 6.2. Supported traffic direction by driver for placement reporting and scheduling (minimum bandwidth only) Enforcement type Supported traffic by direction mechanism driver ML2/SR-IOV ML2/OVN Placement Egress and ingress Technology preview [1] [1] See OSPRH-507 . Table 6.3. Supported traffic direction by driver for enforcement types (bandwidth limit only) Enforcement type Supported traffic direction by mechanism driver ML2/OVN Floating IP Egress and ingress Gateway IP Egress and ingress [1] [1] Technology preview in RHOSP 17.1. See BZ 2088291 . 6.2. Configuring the Networking service for QoS policies The quality of service feature in the Red Hat OpenStack Services on OpenShift (RHOSO) Networking service (neutron) is provided through the qos service plug-in. With the ML2/OVN mechanism driver, qos is loaded by default. However, this is not true for ML2/SR-IOV. When using the qos service plug-in with the ML2/SR-IOV mechanism driver, you must also load the qos extension on their respective agents. The following list summarizes the tasks that you must perform to configure the Networking service for QoS. The task details follow this list: For all types of QoS policies: Add the qos service plug-in. Add qos extension for the agents (SR-IOV only). In ML2/OVN deployments, you can enable minimum bandwidth and bandwidth limit egress policies for hardware offloaded ports. You cannot enable ingress policies for hardware offloaded ports. Additional tasks for scheduling VM instances using minimum bandwidth policies only: Specify the hypervisor name if it differs from the name that the Compute service (nova) uses. Configure the resource provider ingress and egress bandwidths for the relevant agents on each Compute node. (Optional) Mark vnic_types as not supported. Additional task for DSCP marking policies: Enable edpm_ovn_encap_tos . By default, edpm_ovn_encap_tos is disabled. Prerequisites You have the oc command line tool installed on your workstation. You are logged on to a workstation that has access to the RHOSO control plane as a user with cluster-admin privileges. Procedure If you are using the ML2/SR-IOV mechanism driver, you must enable the qos agent extension on the Compute nodes, also referred to as the RHOSO data plane. For more information, see Configuring the Networking service for QoS policies for SR-IOV . Add the required QoS configuration. Place the configuration in the edpm_network_config_template under ansibleVars : If you want to create DSCP marking policies, add edpm_ovn_encap_tos: '1' under ansibleVars : When edpm_ovn_encap_tos is enabled (has a value of 1 ), the Networking service copies the DSCP value of the inner header to the outer header. The default is 0 . Save the OpenStackDataPlaneNodeSet CR definition file. Apply the updated OpenStackDataPlaneNodeSet CR configuration: USD oc apply -f my_data_plane_node_set.yaml Verify that the data plane resource has been updated: USD oc get openstackdataplanenodeset Sample output NAME STATUS MESSAGE my-data-plane-node-set False Deployment not started Create a file on your workstation to define the OpenStackDataPlaneDeployment CR, for example, my_data_plane_deploy.yaml : apiVersion: dataplane.openstack.org/v1beta1 kind: OpenStackDataPlaneDeployment metadata: name: my-data-plane-deploy Tip Give the definition file and the OpenStackDataPlaneDeployment CR a unique and descriptive name that indicates the purpose of the modified node set. Add the OpenStackDataPlaneNodeSet CR that you modified: spec: nodeSets: - my-data-plane-node-set Save the OpenStackDataPlaneDeployment CR deployment file. Deploy the modified OpenStackDataPlaneNodeSet CR: USD oc create -f my_data_plane_deploy.yaml -n openstack You can view the Ansible logs while the deployment executes: USD oc get pod -l app=openstackansibleee -n openstack -w USD oc logs -l app=openstackansibleee -n openstack -f \ --max-log-requests 10 Verify that the modified OpenStackDataPlaneNodeSet CR is deployed: Example USD oc get openstackdataplanedeployment -n openstack Sample output NAME STATUS MESSAGE my-data-plane-node-set True Setup Complete Repeat the oc get command until you see the NodeSet Ready message: Example USD oc get openstackdataplanenodeset -n openstack Sample output NAME STATUS MESSAGE my-data-plane-node-set True NodeSet Ready For information on the meaning of the returned status, see Data plane conditions and states in the Deploying Red Hat OpenStack Services on OpenShift guide. Verification Confirm that the qos service plug-in is loaded: USD openstack network qos policy list If the qos service plug-in is loaded, then you do not receive a ResourceNotFound error. 6.3. Configuring the Networking service for QoS policies for SR-IOV The quality of service feature in the Red Hat OpenStack Services on OpenShift (RHOSO) Networking service (neutron) is provided through the qos service plug-in. If your Networking service ML2 mechanism driver is SR-IOV, then you must also load the qos extension driver for the NIC switch agent, neutron-sriov-nic-agent , which runs on the Compute nodes, also referred to as the RHOSO data plane. Prerequisites You have the oc command line tool installed on your workstation. You are logged on to a workstation that has access to the RHOSO control plane as a user with cluster-admin privileges. Procedure Open the OpenStackDataPlaneNodeSet CR definition file for the node set you want to update, for example, my_data_plane_node_set.yaml . Add the required QoS configuration, NeutronSriovAgentExtensions: "qos" . Place the configuration in the edpm_network_config_template under ansibleVars : Save the OpenStackDataPlaneNodeSet CR definition file. Apply the updated OpenStackDataPlaneNodeSet CR configuration: USD oc apply -f my_data_plane_node_set.yaml Verify that the data plane resource has been updated: USD oc get openstackdataplanenodeset Sample output NAME STATUS MESSAGE my-data-plane-node-set False Deployment not started Create a file on your workstation to define the OpenStackDataPlaneDeployment CR, for example, my_data_plane_deploy.yaml : apiVersion: dataplane.openstack.org/v1beta1 kind: OpenStackDataPlaneDeployment metadata: name: my-data-plane-deploy Tip Give the definition file and the OpenStackDataPlaneDeployment CR a unique and descriptive name that indicates the purpose of the modified node set. Add the OpenStackDataPlaneNodeSet CR that you modified: spec: nodeSets: - my-data-plane-node-set Save the OpenStackDataPlaneDeployment CR deployment file. Deploy the modified OpenStackDataPlaneNodeSet CR: USD oc create -f my_data_plane_deploy.yaml -n openstack You can view the Ansible logs while the deployment executes: USD oc get pod -l app=openstackansibleee -n openstack -w USD oc logs -l app=openstackansibleee -n openstack -f \ --max-log-requests 10 Verify that the modified OpenStackDataPlaneNodeSet CR is deployed: Example USD oc get openstackdataplanedeployment -n openstack Sample output NAME STATUS MESSAGE my-data-plane-node-set True Setup Complete Repeat the oc get command until you see the NodeSet Ready message: Example USD oc get openstackdataplanenodeset -n openstack Sample output NAME STATUS MESSAGE my-data-plane-node-set True NodeSet Ready For information on the meaning of the returned status, see Data plane conditions and states in the Deploying Red Hat OpenStack Services on OpenShift guide. Verification Confirm that the NIC switch agent, neutron-sriov-nic-agent , has loaded the qos extension. Obtain the UUID for the NIC switch agent: USD openstack network agent list With the neutron-sriov-nic-agent UUID, run the following command: USD openstack network agent show <uuid> Example USD openstack network agent show 8676ccb3-1de0-4ca6-8fb7-b814015d9e5f \ --max-width 70 Sample output You should see an agent object with a field called configuration . When the qos extension is loaded, the extensions field should contain qos in its list.
apiVersion: dataplane.openstack.org/v1beta1 kind: OpenStackDataPlaneNodeSet metadata: name: my-data-plane-node-set spec: nodeTemplate: ansible: ansibleVars: edpm_network_config_template: | --- OvnHardwareOffloadedQos: true apiVersion: dataplane.openstack.org/v1beta1 kind: OpenStackDataPlaneNodeSet metadata: name: my-data-plane-node-set spec: nodeTemplate: ansible: ansibleVars: edpm_network_config_template: | --- OvnHardwareOffloadedQos: true edpm_ovn_encap_tos: 1 oc apply -f my_data_plane_node_set.yaml oc get openstackdataplanenodeset NAME STATUS MESSAGE my-data-plane-node-set False Deployment not started apiVersion: dataplane.openstack.org/v1beta1 kind: OpenStackDataPlaneDeployment metadata: name: my-data-plane-deploy spec: nodeSets: - my-data-plane-node-set oc create -f my_data_plane_deploy.yaml -n openstack oc get pod -l app=openstackansibleee -n openstack -w oc logs -l app=openstackansibleee -n openstack -f --max-log-requests 10 oc get openstackdataplanedeployment -n openstack NAME STATUS MESSAGE my-data-plane-node-set True Setup Complete oc get openstackdataplanenodeset -n openstack NAME STATUS MESSAGE my-data-plane-node-set True NodeSet Ready openstack network qos policy list apiVersion: dataplane.openstack.org/v1beta1 kind: OpenStackDataPlaneNodeSet metadata: name: my-data-plane-node-set spec: nodeTemplate: ansible: ansibleVars: edpm_network_config_template: | --- NeutronSriovAgentExtensions: "qos" oc apply -f my_data_plane_node_set.yaml oc get openstackdataplanenodeset NAME STATUS MESSAGE my-data-plane-node-set False Deployment not started apiVersion: dataplane.openstack.org/v1beta1 kind: OpenStackDataPlaneDeployment metadata: name: my-data-plane-deploy spec: nodeSets: - my-data-plane-node-set oc create -f my_data_plane_deploy.yaml -n openstack oc get pod -l app=openstackansibleee -n openstack -w oc logs -l app=openstackansibleee -n openstack -f --max-log-requests 10 oc get openstackdataplanedeployment -n openstack NAME STATUS MESSAGE my-data-plane-node-set True Setup Complete oc get openstackdataplanenodeset -n openstack NAME STATUS MESSAGE my-data-plane-node-set True NodeSet Ready openstack network agent list openstack network agent show <uuid> openstack network agent show 8676ccb3-1de0-4ca6-8fb7-b814015d9e5f --max-width 70 ------------------- ------------------------------------------------+ | Field | Value | ------------------- ------------------------------------------------+ | admin_state_up | UP | | agent_type | NIC Switch agent | | alive | :-) | | availability_zone | None | | binary | neutron-sriov-nic-agent | | configuration | { device_mappings : {}, devices : 0, extensi | | | ons : [ qos ], resource_provider_bandwidths : | | | {}, resource_provider_hypervisors : {}, reso | | | urce_provider_inventory_defaults : { allocatio | | | n_ratio : 1.0, min_unit : 1, step_size : 1, | | | reserved : 0}} | | created_at | 2024-08-08 08:22:57 | | description | None | | ha_state | None | | host | edpm-compute-0.ctlplane.example.com | | id | 8676ccb3-1de0-4ca6-8fb7-b814015d9e5f | | last_heartbeat_at | 2024-08-08 08:24:27 | | resources_synced | None | | started_at | 2024-08-08 08:22:57 | | topic | N/A | ------------------- ------------------------------------------------+