Predictive Intent Maintenance with Intent Drift Detection in Next Generation Network
Chukwuemeka Muonagor Mounir Bensalem Admela Jukan











Predictive Intent Maintenance with Intent Drift Detection in Next Generation  Networks





    
  Chukwuemeka Muonagor, 
  Mounir Bensalem and 
  Admela Jukan 

  Technical University of Braunschweig, Germany {c.muonagor, mounir.bensalem, a.jukan}@tu-bs.de






    April 29, 2024
===========================================================================================================================================================================







Intent-Based Networking (IBN) is a known concept for enabling the autonomous configuration and self-adaptation of networks. One of the major issues in IBN is maintaining the applied intent due the effects of drifts over time, which is the gradual degradation in the fulfillment of the intents, before they fail. Despite its critical role to intent assurance and maintenance, intent drift detection was largely overlooked in the literature. To fill this gap, we propose an intent drift detection algorithm for predictive maintenance of intents which can use various unsupervised learning techniques (Affinity Propagation, DBSCAN, Gaussian Mixture Models, Hierarchical clustering, K-Means clustering, OPTICS, One-Class SVM), here applied and comparatively analyzed due to their simplicity, yet efficiency in detecting drifts. The results show that DBSCAN is the best model for detecting the intent drifts. The worst performance is exhibited by the Affinity Propagation model, reflected in its poorest accuracy and latency values. 









§ INTRODUCTION

Intent-Based Networking (IBN) has emerged as one of the key technology solutions for autonomous networks <cit.>. IBN is a network management system that makes use of natural language processing to compile humanly understandable commands to the network management system, while using machine learning and artificial intelligence in general, as well as orchestration of network resources to automate the activation of these commands <cit.>.  An example of intent could be expressed as permit all traffic from nodes A, F, and K to go through port 80 and not traffic from other nodes in the system, which expressed in this simple form can assure that the policy in the system is enforced and maintained. According to Cisco <cit.>, intent assurance is a core part of any IBN system. It is also referred to as intent validation or verification, which is a network management process that continuously monitors the network to verify that the desired intent which has been applied is functioning properly.



A typical intent-based network life cycle considers three phases: Phase 1 -  when the network operates without an active intent; Phase 2 - the period during which the intent is active and the network behaviour complies with the intent; Phase 3 - the intent failure.  Let take an example of average latency as network parameter to be monitored (Figure <ref>).When an intent is entered into the network (Phase 2) after an hour of network operation when the latency was 4t ms (Phase 1)  the latency value is brought down value of 1t ms instead. In Phase 2, which is intent fulfillment period, the latency remains on the average at this value of t ms for about 75 mins before it starts fluctuating gradually in a generally increasing trend. The failure of the intent is confirmed after one and half hours of these fluctuations in Phase 3 - the intent failure period when the network finally goes back to its original state where the latency was 4t ms before the intent was entered. It is precisely in the time period where the latency fluctuating gradually in a generally increasing trend that illustrates the period of the so-called Intent Drift. Although a major part of intent maintenance should consider whether intent drift occurs over time <cit.>, only a few studies addressed intent drift detection. 



This paper proposes a novel function for the predictive maintenance of intents in the IBN architecture, i.e., the intent drift detection. We develop and apply an intent drift detection algorithm that can selectively test seven machine learning (ML) models to detect intent drifts, which we analyze on their ability to predictably  maintain intents based on their resulting accuracy, response time and complexity. Specifically, unsupervised learning techniques (Affinity Propagation, DBSCAN, Gaussian Mixture Models (GMM), Hierarchical clustering, K-Means clustering, OPTICS algorithm, and One-Class SVM) are adapted to this end, while a greedy algorithm is also used for comparison purposes. The results show that DBSCAN has the best accuracy, and the lowest average response time (latency) in detecting intent drifts, though its space complexity is high. The worst detection performance is exhibited by Affinity Propagation algorithm reflected in its poorest accuracy value and maximum latency. To the best of our knowledge, this is the first work to address the problem of intent drift detection, and predictive intent maintenance. 

 
The rest of the paper is organized as follows. We present and discuss related work in Section II. Section III proposes the reference IBN architecture with intent drift detection module. Section IV focuses on intent drift detection algorithms. Section V analyzes numerical results. Section VI concludes the paper. 






§ RELATED WORK

Previous work majorly focused on intent assurance and measures to overcome failures <cit.>. Studies in <cit.>resorted to ML techniques to build a self-assurance IBN system on top of programmable networks. Alternatively to intent assurance methods, the use of threshold can be adopted, for instance in <cit.> with rerouting when packet losses exceed the threshold. The problem with setting of thresholds or establishment of specific reference points for the network is, according to <cit.>, that when network behaviour is dynamic, fixed thresholds specifications may not reflect actual network behavior. A drift in the intent may be detected, due to dynamic network state changes, whereby IBN should automatically take actions to revert the network back to compliance state <cit.>. However, detecting this gradual degradation of intents and at the same time avoiding false positives is a challenge. 

The problem of intent drift can be solved by analyzing the network behaviour during intent fulfillment and normal network operation, which in turn is a problem of statistical or analytical modeling of the network behavior, or based on time series analysis and forecasting, e.g., as in <cit.>. While statistical modelling may not be best suited to a large amount of data, time-series analysis and rule-based heuristic algorithms also have difficulty scaling <cit.>. We opt for unsupervised machine learning here, due to their simplicity, and due to the fact that the network data used in the analyses would come unlabeled. The unsupervised learning models used in our algorithms are <cit.>: Affinity Propagation, DBSCAN, Gaussian Mixture Models (GMM), Hierarchical clustering, K-Means clustering, OPTICS, and One Class SVM. 



§ INTENT DRIFT DETECTION



 §.§ Reference Architecture


The newly introduced module of Intent Drift Detection into the  typical IBN architecture <cit.> is illustrated in Figure <ref>. Generally, an IBN system fulfills 3 main objectives; the translation where the business intents are mapped into policies, the activation where the intents and policies are deployed and enforced into the network, and finally the assurance where validity of the intent is verified and changes are detected, predicted and prevented, using AI and ML techniques. We also illustrate how IBN interacts with the network management systems, with examples of the selected functions <cit.>, including  Security Management, QoS provisioning, etc.




In the IBN architecture, the Intent Entry Interface receives high-level intents from the user and sends same to the Intent Manager, which parses them into low-level intents and forwards them to the Policy Configurator. The Intent Manager also sends the intents to the Intent Store. The Policy Configurator matches the intents received from Intent Manager with the policies in the Policy Store to generate the appropriate workflows to be sent to the Network Manager. The Knowledge Base is the storage component of the IBN system. It contains the Policy, Intent and Data Stores. The Policy Store contains the policies that the Policy Configurator matches the intents with so as to generate the appropriate workflows corresponding to the desired intents. The Intent Store stores the received intents from the Intent Manager. When an intent is deleted from the system, it is also deleted from the Intent Store. The Data Store stores network data received from the Monitoring function block of the Network Manager. 

In this architecture, we propose to add a new component, i.e., the Intent Drift Detector. This  component retrieves data from the Data Store and analyses it to be able to detect any drift in the fulfillment of the intents. It enables predictive maintenance of intents, and prevents its failure.
It should be noted that while the existing IBN architectures consider an intent assurance engine or module, it does not implement any explicit  intent drift detection. This is a novel architectural component introduced in this work which replaces the conventional intent assurance engine. The latter is often hard coded to check periodically the validity of an intent, which is not efficient. 




 §.§ Intent Drifts Exemplified



We now illustrate a few intents and then exemplify their drifts, see Figure <ref>.  Intents A and B are examples of security intents. Intent A entails that the bandwidth allocation to the attack victim is to be increased by an extra 3 Gb/s in the event of the attack. Intent B requires that the entire incoming traffic to the attack victim would be minimised, i.e., many packets would be dropped while very few packets are allowed. Intents C and D are examples of QoS provisioning intents. Intent C implies that there should not be any restriction to the amount of bandwidth allocated to the host. Intent D requires that the bandwidth allocation to all types of traffic to and from each host across the entire network be minimised.  Intent E is a load balancing intent. Finally, Intent F is a reachability assurance intent that ensures that the host or server is reachable for a period of 7200 secs. This implies that all traffic directed to this host must be able to reach it.   

An intent drift can be caused by a  manual intervention of the network administrator: when Intent B is enforced, the Security Management drops several incoming packets to Host  when a DDoS attack is detected. Assuming that an operator manually makes an intervention or changes by allowing some traffic to access the host, this change can cause a drift in the security intent, which needs an additional verification. Another drift example can be caused by an automated reconfiguration: when intent E is enforced, the load balancing distributes the traffic approaching the host. Assuming that the monitoring system can suggest updates to the traffic distribution due to CPU, memory, or bandwidth telemetry, this might lead to unbalanced traffic distribution, which causes intent E to drift. Furthermore, intent drift can be caused by a conflict between intents. For instance, an existing Intent B, and a new intent C can conflict, since due to maximisation of QoS assurance for a host which is what Intent C is, i.e., ensuring the usage of maximum bandwidth by that host, there is conflict with intent B. Similarly, there maybe a conflict between the existing intent A, and new intent D, etc. 






§ ALGORITHMS FOR INTENT DRIFT DETECTION


In this section, we propose an intent drift detection algorithm Algorithm <ref>, which uses different ML models to detect intent drifts from monitoring time series data. The algorithm receives stored time series data,  X_train, which can be specified depending on the intent, a new incoming data to be evaluated, such as throughput,  X_test, which can be a recently measured telemetry, and finally an algorithm selection, Model_Type, where a list of ML algorithm and heuristics is provided to support drift detection. In Step 1, we check if the chosen model requires a training phase, then run and save the learnt model. In Step 2, we evaluate our new data either using the trained model, or run clustering phase for models like K-means. Using the output of the evaluation phase, the algorithm decides whether there is an intent drift or not depending on how each ML and heuristic works (Step 3), and then send the results to the user (Step 4).




 §.§ Detection algorithms based on various ML models




  §.§.§ Affinity Propagation

Affinity propagation creates clusters by sending messages between pairs of samples until convergence. A dataset is then described using a small number of exemplars, which are representative points that have high similarity to other points in the same cluster, and low similarity to points in other clusters. Affinity Propagation has the preference parameter, which is the “availability” a(p, q) sent from point q to point p reflecting how much point q sees itself as an exemplar of point p against other potential exemplars. 
For the detection of intent drift according to Algorithm 1, the multiplier value is initialised at first. Then, the learning process in Step 1 starts with the computation of the suitable preference parameter automatically as the minimum of the pairwise distances between the network throughput values, after which the training is done by generating the number of clusters automatically by assigning samples to exemplars. The higher the variation in the throughput data the higher the number of generated clusters. The same process is repeated for new measurements of throughput values. In Step 3, a drift in the intent is indicated by an increase in the number of generated clusters from the data.








  §.§.§ DBSCAN

The intent drift detection process implemented with DBSCAN is based on <cit.> and <cit.>. We first initialise MinPts which is the minimum number of samples in a given neighborhood. The learning process starts in Step 1. After the training is done, the neighborhoods are generated automatically based on data variability. The extracted throughput values are then assigned to different neighborhoods or discovered clusters using equation 3. Throughput values that are within the radius of the neighborhood. The more similar the throughput values are, the less the number of generated or discovered clusters or neighborhoods. The same process is repeated for new measurements of throughput values. In Step 3, a drift in the intent is indicated by an increase in the number of generated or discovered clusters from the data. 



  §.§.§ Hierarchical clustering

Hierarchical clustering builds nested clusters by merging or splitting them successively. The linkage criteria determines the metric used for the merge strategy. 
For the detection of intent drift according to Algorithm 1, the parameters affinity and linkage are initialised at first. The learning process in Step 1 starts with the automatic computation of the ideal distance threshold value which is based on the mean of the extracted throughput data. After which the training is done by generating the number of clusters automatically by assigning samples to clusters. The same process is repeated for new measurements of throughput values. A drift in the intent according to Step 3 can be decided when the number of generated clusters from the data increases. 



  §.§.§ K-Means


Assuming one is given input data points p_1, p_2, ..., p_n, the first step in K-Means clustering involves picking K points randomly as the initial centroids or cluster centers q_1, q_2, ..., q_k. After which the Euclidean distance between each point in the dataset and the identified K centroids is computed as: 

    d_ij=√((p_i-q_i)^2 + (p_j-q_j)^2)1

Then, each data point p_i is assigned to the cluster with the closest centroid based on the d_ij estimated. That is, each point p would belong to a cluster c_i with centroid q_i if:

    d_ij= min_c_i∈ C d_q_i, p2

Then, the new centroid is calculated by finding the mean of the entire data points in the cluster as follows:

    q_i^'=1/N_i∑_p∈ c_ip, i=1,2, ⋯, k 3


The description above is the clustering process implemented by the K-Means clustering algorithm. 


At first, the value of a multiplier, later used in Step 3, is initialised. The input data are the old and new throughput values referred to as X_train and X_test, respectively. In Step 1, the learning process is done which involves first determining the optimal number of clusters using the Silhouette analysis since number of clusters determines how accurate the algorithm would be, after which training is done on the extracted throughput data using the determined optimal number of clusters. These throughput values are partitioned into the various clusters and their new centroid values are computed. An array of this new centroid values is created and sorted. The differences between neighbouring centroid values and the maximum of the differences are estimated, the lower this maximum value, the more similar the entire throughput values from different clusters are. The same process is repeated whenever new measurements of the throughput values are received by the model. 
In Step 3, the new maximum is compared with the old maximum. If the new maximum exceeds the product of the old maximum and the multiplier, then there is drift.



  §.§.§ Gaussian Mixture Models (GMM)

Gaussian mixture model generalises K-Means clustering, also taking note of the input data covariance structure as well as the Gaussian components centroids. One of the important parameters in GMM is the number of desired components. The intent drift detection process using GMM is very similar to that of K-Means. The number of clusters in K-Means is similar to the number of components in GMM and the same process is followed according to the steps in Algorithm 1.



  §.§.§ OPTICS algorithm

DBSCAN and OPTICS algorithms are similar. For OPTICS algorithm, at first the minimum number of samples, the reachability distance and the minimum cluster size are initialised. Points with similar reachability distances are likely to be in the same cluster. The learning process in Step 1 starts with generating the neighborhoods automatically, assigning the throughput data to different neighborhoods based on their reachability distances. In Step 3, a drift in the intent is indicated by an increase in the number of generated or discovered clusters from the data, after which Step 4 is implemented which reports the intent drift. 



  §.§.§ Support Vector Machine (SVM)

The SVM algorithm used here is One-class SVM (1-SVM). According to Algorithm 1, the first action is to initialise the kernel function to be used and the nu parameter which is upper bounded by the fraction of outliers considered by the model to be in the data and lower bounded by the fraction of support vectors. This is followed by implementing Step 1. Step 1 involves learning the 1-SVM on the data to group the old extracted throughput values into normal data and outliers. In Step 2, the trained model is used to make inference on the newly extracted data to be able to classify them into either normal data or outliers. The presence of outliers in the newly extracted data indicates possibility of drift, and the drift detection result is sent to the administrator at the controller level.



  §.§.§ Greedy algorithm

Greedy algorithm is used  for comparison. It is based on the determination of the maximum measured throughput data at any point in time. When the maximum of the incoming traffic over a short period exceeds this network monitoring maximum with a large margin, which is defined at the inception, the intent drift is detected.






 


 §.§ Complexity Analysis

In terms of data size, aside the GMM and the Affinity Propagation algorithms, the algorithms used here are scalable as they are not affected by the size of data. K-Means, DBSCAN and OPTICS are the most scalable of the models. However, in terms of number of nodes and with a relatively fixed amount of data, the entire algorithms are scalable as they can be deployed on each node and also as many nodes as possible. 

The Affinity Propagation algorithm has a time complexity of the order O(N2T) where N is the number of samples and  T is the number of iterations until convergence. Further, the memory complexity is of maximum O(N2). The time complexity of k-means algorithm is O(N2). The overall complexity is O(N2) for DBSCAN, O(N3) for GMM, O(N) for OPTICS, and O(N) for Greedy algorithm. The complexity of One-Class SVM is at best O(N2). The time complexity of the agglomerative hierarchical clustering is O(N3) because in each of the N-1 iterations the N × N matrix dist matrix is scanned exhaustively for the lowest distance, and the memory complexity is O(N2).



§ MEASUREMENTS AND RESULTS



 §.§ Network Emulation Setup and Evaluation Metrics

We used a network emulation setup which contains three hosts and one virtual switch hosted as virtual machines provisioned with the virtualisation platform, VirtualBox, together with an external host where the VirtualBox platform is hosted. The number of hosts was decided arbitrarily. The external host serves as the controller where the IBN software runs and controls how the rest of the hosts which are the virtual machines operate, and also has the network manager emulator that implements the workflows as commands to the other hosts through . The network data are the ingress throughput at a host. Two iterations each were run on three virtual machines and the performance of the models were measured. The intent drift detection algorithms are implemented on the hosts while they send the results of their analyses to the controller in the external host to be viewed at the intent entry interface.










Since any intent implementation needs to be specific to the intent, we consider two specific intents in the analyses which are applied one after the other. These intents are Intents B and D respectively which have been exemplified earlier. Intent B is an already-existing security and reliability intent which is to deploy an action in the event of an attack so as to mitigate the attack. In the event of the attack the execution of the security intent installed at the controller machine which is to mitigate the attack is triggered. The second intent, Intent D is a QoS intent that entails minimising ingress bandwidth to a host due to load or congestion. This implies that only part of the ingress traffic is selected and routed to the host while the remaining traffic is dropped. For each of the two intents, during the period of the intent fulfillment, the ingress throughput to the host is observed and analysed with the aid of the models to determine when possible drift in the intent starts occurring and/or when the intent fails entirely. The measurement of the ingress throughput is conducted by the host with the aid of the  linux command every second, and the data can be extracted in batches periodically or singularly. Here, an arbitrary choice of every nine seconds is made.

Each measurement iteration contains normal operation, intent fulfillment, intent drift, and intent failure phases. Our focus is on the intent drift phase. The metrics used to judge the performances of the different models are the accuracies of the models observed over the period of the entire iterations, the average latencies of the models while detecting the intent drift periods, and the space complexity of the models.


For the two intents, Intents B and D, two of the iterations run on different machines were selected and the plots of the network behaviour capturing their life cycles as illustrated in Figures <ref> (Intent B - security intent) and <ref> (Intent D - QoS intent). Figure <ref> shows that the host experienced an attack at about 140s in time, at which a traffic restriction action according to the existing reliability intent is deployed to mitigate this attack. This precedes the intent fulfillment phase were the attack is mitigated for over 120s due to the action deployed by the intent. This is followed by an intent drift period where the intent fulfillment starts experiencing degradation. This intent drift period lasts for a while before the intent fails eventually when the host starts to experience an attack again. The life cycle for the QoS intent is shown in Figure <ref>. The intent fulfillment phase starts from almost 180s and lasts for some time before the drift in the intent starts being noticed at almost 400s.








 §.§ Performances of the Models





  §.§.§ Model Accuracy

From Figure <ref>, it can be seen that DBSCAN had the highest accuracy (0.846) of all the models in detecting intent drift. This is followed by GMM and Hierarchical clustering methods with a joint accuracy of 0.768, and K-Means clustering, then SVM. The higher accuracy of the DBSCAN model can be attributed to two reasons. One is the fact that in DBSCAN algorithms number of clusters are generated automatically according to statistical dispersion of the data, otherwise known as data variability as opposed to the K-Means, GMM and some other methods that make use of fixed number of clusters which can't always reflect network behaviour or the Greedy algorithm that only computes the maximum values of the throughput data. Another reason is that DBSCAN algorithms are also robust to noise and outliers since they are not assigned to any cluster. The low accuracy of SVM can be attributed to the fact that it tends to suffer when there is no clear separation between the normal and abnormal data, which is the case in this work. 



  §.§.§ Average Latency

 The resilience and reliability of networks is dependent on the amount of time it takes the network to detect a problem or failure and implement a recovery strategy to ensure resilience. From Figure <ref>, it is noticed that the SVM model had the lowest average latency in reporting the drifts. In a case like this where the amount of data is not very large, SVM tends to execute fast. Worthy of note is that SVM reported drifts more frequently than any other model even when actual drifts didn't occur, this contributed to its low latency even though it had poor accuracy. DBSCAN algorithm had the second lowest delay while detecting the drift in the network which is followed by GMM. Other algorithms like KMeans, Optics and the Greedy Algorithm showed very high latencies with Affinity Propagation being the poorest due to its general inability to quickly detect the drifts. This can be attributed to the fact that DBSCAN works well with data that are not very well-separated, e.g., network data, and does its clustering as new data are observed unlike K-Means and the others whose strength is with well-separated data, so the former would require less time to draw conclusion that a drift in the intent exists based on the data it observes.





  §.§.§ Space Complexity

Space complexity can be estimated as the amount of allocated bytes, whereby algorithms that generally require less spaces run faster. From Figure <ref> which illustrates the average memory usages of the models it can be noticed that OPTICS, DBSCAN, Hierarchical clustering, and Affinity Propagation methods required generally more amount of memory than the others followed by SVM. This high complexity of DBSCAN and the others is due to the fact that they spend time to generate the appropriate clusters themselves automatically before assigning data points to these clusters. GMM and KMeans have the lowest complexity. This is because they view the dataset as well-separated and try to accommodate noise, unlike the previously mentioned algorithms that take strong cognisance of the data variability and do extra work of noticing noise and treating it as outlier.






§ CONCLUSIONS AND OUTLOOK

We conclude that the concept of intent drift detection is very important in predictive intent maintenance.   The DBSCAN model had the best accuracy in detecting the drift due to its automatic cluster generation and tendency to ignore noise and outliers. It also responded second fastest to classifying data traffic as anomaly which is indicated by its lowest average latency of 0.8s, with SVM being the fastest to report a drift. The worst performance was exhibited by the Affinity Propagation model. This is reflected in its poorest accuracy value of 0.428 and maximum latency of 5.152s and also its inability to detect the intent drift phase quick enough. The OPTICS algorithm had the highest complexity while GMM and K-Means had the least complexities.

There are many open areas for further work in this subject. The problem of intent drift is much more complex than illustrated here. Reducing the amount of false positives to the barest minimum is part of the basis for further development in this area of research. Further, the intent drift detection technique is peculiar to the type of intent, so replicating this work for other types of intents in different kinds of network is of very good interest. Also, other methods like neural networks and reinforcement learning can't entirely be ruled out and need to be studied on efficiency. 








IEEEtran


