essential part of optimizing resources. Overbooking ratios are adjusted by monitoring KPIs and KCIs. Another important operational practice is implementing closed-loop automated functions to manage VNFs and the associated resources. Closed-loop operational metrics are used to measure and report automated VNF lifecycle management events. AT&T has implemented closed-loop oper- ations in ONAP and metrics that report effectiveness of these closed-loop events (e.g., detection of VM stall and latency of a VM restart) are important to operational scorecards. The closed-loop event metrics include measures of frequency, delay, and success rates of targeted lifecycle management. These can be used as a health and resource consumption indicators (e.g., increase in closed-loop event occurrence frequency such as auto restarts of VNF, can be an indicator of VNF resiliency risk or resource bottleneck). 16.9.2 VNF SLAs AND KPI The SLAs typically specify the targets for the KPIs, which need to be met by the VNF implementa- tion. The SLAs are determined by user expectations and in some cases, by contractual or regulatory constraints. Typical performance KPIs for VNFs include 1. Latency 2. Availability 3. Completion rate 4. Defect rate Latency bounds are needed to ensure that application transaction flows complete in a timely manner. The application layer latency constraints depend on protocol bounds for expected response times. Protocols that allow for retries of unacknowledged transactions need to be considered as part of the VNF latency bounds. Availability is a time-based metric that represents the percent of uptime for the VNF. Downtime measures the amount of time the VNF is not available, and is typically expressed in seconds, min- utes, or hours. Completion rate measures the rate of successful transactions through the VNF; transactions are specific to the VNF application. Defects are unsuccessful VNF transactions; in other words, transactions that do not complete as expected. Defects are measured in absolute counts as well as