Learning Optimal Solutions for Extremely Fast AC Optimal Power Flow
Ahmed Zamzam Kyri Baker

 
[
    [
    December 30, 2023
=====================






In this paper, we develop an online method that leverages machine learning to obtain feasible solutions to the AC optimal power flow (OPF) problem with negligible optimality gaps on extremely fast timescales (e.g., milliseconds), bypassing solving an AC OPF altogether. This is motivated by the fact that as the power grid experiences increasing amounts of renewable power generation, controllable loads, and other inverter-interfaced devices, faster system dynamics and quicker fluctuations in the power supply are likely to occur. Currently, grid operators typically solve AC OPF every 15 minutes to determine economic generator settings while ensuring grid constraints are satisfied. Due to the computational challenges with solving this nonconvex problem, many efforts have focused on linearizing or approximating the problem in order to solve the AC OPF on faster timescales. However, many of these approximations can be fairly poor representations of the actual system state and still require solving an optimization problem, which can be time consuming for large networks. In this work, we leverage historical data to learn a mapping between the system loading and optimal generation values, enabling us to find near-optimal and feasible AC OPF solutions on extremely fast timescales without actually solving an optimization problem.





§ INTRODUCTION

AC optimal power flow (OPF) problems are solved by grid operators in order to achieve the most economic generation dispatch to meet  network power demands while adhering to the physical constraints of the network. With the increasing integration of fluctuating renewable energy sources, system operators are required to perform more frequent adjustments of the generators set-points. Currently, real-time adjustments are realized using the automatic generation control (AGC) which may lead to sub-optimal operational points due to the use of restrictive affine control policies. Various linearizations or approximations of the AC OPF problem have been developed throughout the years to address the computational burden of solving the AC OPF in real time, but in many cases cannot guarantee feasibility of the approximate solution, and still require solving an optimization problem, which can be prohibitive as the network size grows. 

While not an exhaustive list, there are three general techniques that have been used in the literature to pursue OPF solutions on fast timescales <cit.>: 1) Leveraging linearizations, approximations, or convexifications of the AC power flow equations (e.g. <cit.>); 2) Distributed techniques which allocate the computation of the AC OPF problem across devices, agents, or distributed computational platforms (e.g. <cit.>); and 3) So-called “real-time" or “online" optimization techniques (e.g., <cit.>). Combinations of these three, i.e., distributed online approaches which utilize linearizations of the power flow equations, are also used <cit.>.

Very recently, machine learning (ML) approaches have been leveraged for solving difficult optimization tasks <cit.>. The data-driven nature of ML approaches is very pertinent as large quantities of measurements are being generated in modern systems but are not yet being fully utilized <cit.>. In the area of power systems, ML approaches have been proposed to recover the power systems state <cit.>, to enhance solving optimal power flow problems (i.e., for learning a good starting point for AC OPF: <cit.>, for learning the active set of the DC OPF problem: <cit.>) and for directly bypassing the use of solvers or iterative methods to learn solutions in an online fashion (i.e. leveraging ML to obtain the solution of mixed-integer quadratic programs: <cit.>, and for learning a feasible and optimal DC OPF solution: <cit.>). 


In <cit.>, neural networks are used to learn a mapping from uncertainty realizations to the active set of a DC OPF problem as an intermediate step towards learning the optimal solution. Once the active set is determined, the optimal solution to the original problem can be recovered by solving a linear system of equations. This  work illustrated the benefits of using data to exploit the structure of the OPF problem and solve DC OPF on timescales appropriate for corrective control. However, showing the benefit of this particular framework for AC OPF problems, which are very difficult to solve in real time, has not yet been demonstrated. In addition, if the correct active set is not identified during the classification, the resulting solution of the DC OPF will be incorrect, and additional resources must be used for identifying the correct active set.

The work in <cit.> demonstrated that it is possible to learn the solution of the DC OPF problem directly and preserve feasibility. Encouraged by the extremely fast solution times for DC OPF with feasible solutions and negligible optimality losses when using a neural network in <cit.>, we explored the potential and limitations of learning a mapping from the system loading to optimal generation setpoints and voltages in AC OPF. Towards this goal, we developed a framework for using learning to obtain solutions to AC OPF problems that lends itself to real-time applications without using the DC approximation or other linearizations of the problem. 

Our approach thus offers the following benefits over traditional approaches to solve the the AC OPF problem and over learning approaches to solve the DC OPF problem:


    
  * There is no need to directly tackle the AC OPF optimization problem.
    
  * No approximations, linearizations, or convexifications of the power flow equations are used.
    
  * No distributed techniques or computationally intensive platforms are needed; the neural network maps system loading onto optimal generation values and voltage magnitudes in roughly 1 millisecond on a laptop computer (in the considered networks).
    
  * Recovery of the full AC OPF solution is ensured through a fast iterative procedure, resulting in a final optimal and feasible solution.


Recently, the authors of <cit.> proposed a machine learning model that targets approximating the AC OPF solutions directly. The AC OPF problem is treated as a regression task where the generator set-points are estimated from the grid demand profile. Fixing the active power injection at all generators, including the slack bus, drastically affects the feasibility of the solution. Hence, the reported feasibility percentage is almost 50% for a maximum of 10% change in the base load profile in a small network. In addition, the design of the learning approach in <cit.> does not allow for a post-processing step where a feasible solution is obtained by refining the infeasible solution. In addition, the design of the test scenarios with only 10% deviation without considering correlation makes the representativeness of the study questionable.


The emergence of learning techniques in general can be attributed to the availability of increased computing power and an abundance of data collection and storage. Within the power systems realm, a large quantity of data is generated by grid operators by repeatedly solving AC OPF problems throughout the day. This data is utilized for other tasks, and often is not stored for long periods of time, as grid operators struggle to see additional use cases for the data <cit.>. This paper demonstrates an additional purpose for this data by using it to train a neural network (offline) which can produce optimal solutions to the AC OPF problem (online). To achieve this, we learning a mapping from the network loads to the optimal generator set-points, where a novel parameterization of the AC OPF solutions is used to enforce generation limits. In addition, strictly feasible solutions are used to train the learning model, and the output of the model is constrained to adhere to active power generation and voltage magnitude constraints. In order to circumvent solutions that violate the system requirements, the solution feasibility is ensured by solving a set of power flow equations. Fig. <ref> shows the architecture of the proposed approach in the operation phases. By using this framework, we can determine optimal generation set-points on timescales that are appropriate for balancing fast fluctuations in renewable generation and load.



The remainder of this paper is organized as follows. In Section II, the AC OPF problem formulation is presented. Then, the propoed learning approach is outlined in Section III, and the simulation results are presented in Section IV. Section V conculdes the paper and presents several directions for future research.







§ PROBLEM FORMULATION

In this section, we will describe the AC OPF problem and our data-driven approach to reformulate the problem appropriately. First, consider a power network with N buses collected in set 𝒩, and set 𝒢 collects the set of nodes with generators, and set ℒ collects the set of load buses. 
The nodal power injection at bus n  ∈𝒩 is denoted by p_n + j q_n. Also, let p_g,n+j q_g,n, and p_l,n+j q_l,n denote the  power generated at bus n∈𝒢 and the load demand and bus n∈ℒ, respectively. Note that the sets 𝒢 and ℒ are both subsets of 𝒩 but need not to be disjoint. The voltage at bus n is denoted by v_n, and define v := [v_n]_n∈𝒩, p := [p_n]_n∈𝒩, and q := [q_n]_n∈𝒩 which collect the voltage phasors, active power injection, and reactive power injection at all buses, respectively. Therefore, the AC OPF problem can be formulated as


    v, {p_g,n, q_g,n}_n∈𝒢minimize      c( p_g) = ∑_n∈𝒢 c_n(p_g,n)
    subject to       p_g,n≤ p_g,n≤p_g,n
           q_g,n≤ q_g,n≤q_g,n
           | v |≤ | v| ≤| v |
            h( v,  p,  q) =  0


where p_g,n (q_g,n)  and p_g,n (q_g,n)  denote the minimum and maximum active (reactive) power injection from the generator installed at bus n ∈𝒢, and | v | and | v | represent the lower and upper limits on the voltage magnitudes. The vector collecting the active power injection from all generators is denoted by p_g, and the cost of generating power from the generator installed at bus n is denoted by c_n(p_g,n). The nonlinear power flow equations are collected in the equality constraint (<ref>).


Given the active and reactive power demand at the load buses, the active power generated by the generators, and the voltage magnitudes at the generator buses, the vector v can be recovered by solving the power flow equations <cit.>. Utilizing this property, we treat the AC OPF problem as an operator with inputs representing the load profile while the generators' active power injections and voltage magnitudes are the output of this operator. In other words, we treat the OPF as an operator <cit.> with inputs comprising p_l,n and q_l,n for all n∈ℒ and with output representing |v_n| for buses n ∈𝒢 and p_g,n for nodes n∈𝒢{n_0}, where n_0 is the index of the reference bus. Therefore, the goal of the proposed data-driven approach is learn the underlying OPF mapping between the load demand at all load buses and the generators' active power and voltage magnitude set-points. 


Any active power generation p_g,n that satisfies (<ref>) can be written as 

    p_g,n = p_g,n  + α_n ( p_g,n - p_g,n),

where 0 ≤α_n ≤ 1. Similarly, the voltage magnitude at bus n∈𝒢 can be parameterized by β_n∈ [0, 1] which is given by

    |v_n| =  |v|+ β_n (|v| - |v|).

Define the sets 𝒜 := {[α_n]_n∈𝒢{n_0} | 0 ≤α_n ≤ 1 } and ℬ := {[β_n]_n∈𝒢 | 0 ≤β_n ≤ 1 }.  Therefore, the AC OPF operator can be written as 

    Ω : 𝒫_l ×𝒬_l →𝒜×ℬ,

where 𝒫_l and 𝒬_l represent the sets of active and reactive load demand at all buses n∈ℒ, respectively. 
Paremterizing the solution of the AC OPF problem using the parameters α_n and β_n will prove useful when approximating the mapping Ω using artificial neural networks.  



§ LEARNING FEASIBLE AC OPF SOLUTIONS


Artificial neural networks (hereafter, NN) are well-suited for modeling high-dimensional, complex, non-linear relationships between input features and output variables. They are capable of considering massive amounts of training data (here, solutions to the AC OPF problem) and uncovering natural relationships in the data to perform classification or regression tasks extremely quickly. It is for this reason that they have been used previously for predicting OPF solutions <cit.> and the active set of OPF problems <cit.>. In this section, we discuss the generation of training data for the NN, training of the NN, and how the overall feasible solution to the AC OPF problem is obtained.









 §.§ Approximating the mapping between loads and the optimal solution


In order to approximate the mapping Ω, we utilize a deep neural network model which leverages sigmoid activation functions. For finite-width deep neural networks, smooth functions can be approximated up to a specified accuracy which depends on the smoothness parameters of the function as well as the neural network size <cit.>. That is, if a neural network g(· ;θ) is optimally trained with samples of f(·) on a closed set, then the distance between the output of the neural network ŷ = g(· ; θ) and ground-truth mapping y = f(·) can be bounded as

    ŷ -  y_∞≤ϵ,

where ϵ is function of the smoothness parameters of f(·) and the size of the neural network g(·;θ). 
As the solution of AC OPF is often on the boundary of the feasibility set <cit.>, we aim at ensuring that the neural network outputs are in the interior of the AC OPF feasibility set. 

In order to avoid these cases where the voltage magnitude is exactly at the maximum or the lower voltage limits, we generate the training by solving the following restricted AC OPF problem (R-ACOPF).


    v, {p_g,n, q_g,n}_n∈𝒢minimize  ∑_n∈𝒢   c_n(p_g,n)
    subject to    p_g,n   ≤ p_g,n≤p_g,n
        q_g,n   ≤ q_g,n≤q_g,n
        | v | + λ    ≤  | v|  ≤ | v | -λ
            h( v,  p,  q) =  0


Therefore, the generated solutions are guaranteed to be in the interior of the feasibility region with respect to the voltage magnitudes. Note that this data-driven approach approximates a mapping that deviates from the original AC OPF mapping. However, the training samples generated by solving (<ref>) are strictly in the interior of the voltage magnitudes feasibility set. Therefore, any bounded deviations in the learned mapping are expected to remain within the voltage limits. The parameter λ can thus be considered as an algorithmic tuning parameter that addresses the optimality and feasibility trade-off. Notice that large values of λ may render the original problem infeasible, and hence, the parameter λ has to be tuned in order to obtain strictly feasible solutions.


In addition, to ensure the generation limits adhere to the prescribed AC OPF constraints, we use sigmoid functions at the output layer of the NN. This ensures that the output is bounded between 0 and 1 for the values of α_n and β_n. Designing the output layer of the neural network to be a sigmoid function acts as an implicit constraint on the learned AC OPF mapping, which illustrates the reasoning for reparameterizing the AC OPF solutions in terms of α_n and β_n. Although enforcing general constraints on the output of the neural network makes the training process challenging, incorporating a sigmoid function at the output layer does not increase the complexity of the training process.






 §.§ Generating training samples

In a physical grid, historical AC OPF runs would be used as the training set for the neural network, yielding an abundance of data representing a wide variety of system states. However, for the purposes of simulation and testing, we must generate this set of training data. Let p_l ∈ℛ^|ℒ| denote the vector collecting the load demand at all load buses. We generate the instances of load demand using the truncated Gaussian distribution 

    p_l ∼𝒯𝒩 ( p_l,0, Σ_l, (1-μ) p_l,0, (1+μ)  p_l,0),

where p_l,0 denotes a base loading profile, Σ_l denotes the covariance matrix, and μ denotes the maximum allowable deviation from the base load profile, which we set to 0.7 in our simulations. The covariance matrix accounts for the relationships between loading patterns at different locations throughout the network. Here, we generated load profile realizations using a Gibbs sampler algorithm <cit.>.
For considering deviations in the reactive power demand, we randomly chose the power factor of each load at each instance from a uniform distribution between 0.8 and 1.
Rarely, the generated load profile can result in an infeasible solution for the AC OPF problem, and hence infeasible for the R-AC OPF problem as well. In such cases, we discard this load profile from the training set, as it does not represent a physically realizable state of the power system.



 §.§ Learning model

A deep NN model (i.e., a NN with more than two layers) is used to approximate the mapping between the load profile and the optimal voltage magnitude and active power injections of the generators. The size of the input of the NN is 2|ℒ| representing the active and reactive load demand at the load buses, and the number of outputs is 2|𝒢|-1 representing the voltage magnitude at the generator buses and the active power injection at all generators except the one at the slack/reference bus. Fig. <ref> depicts the architecture of the deep NN learning model utilized in our approach. Let ( x_i,  y_i) denote the i-th pair of data within the training set. The model is thus trained using the classical empirical loss minimization formulation which is given by

    min_θ1/I∑_i^I  y_i - g( x_i; θ) _2^2,

where I represents the total number of the training samples, g(·;θ) represents the NN mapping where θ collects the trainable parameters.





 §.§ Recovering feasible solutions

As mentioned in Section <ref>, we pursue feasibility by both restricting the training set to include strictly feasible solutions and adhere to generation and voltage limits by constraining the output of the neural network. While this ensures feasibility of the learned values, the overall power flow equations and other generation constraints must be adhered to. Thus, the power flow equation are solved in order to recover and ensure feasibility of the overall AC OPF solution. Let p^o_g,n and |v^o_n| denote the output of the neural network representing the active power injection and voltage magnitude of bus n∈𝒢. Then, solving the power flow problem can be formulated as follows where the output of the NN representing the voltage magnitude and active power injection at generator n are denoted by |v_n^o| and p_g,n^o, respectively.


    find        v,  p_g,  q_g
    subject to        |v_n| = |v^o_n|   ∀ n∈𝒢
            p_g,n = p_g, n^o   ∀ n∈𝒢 n_0
            h( v,  p_g,  q_g) =  0



In rare cases, the solution of the power flow equations result in reactive power injections at the generators that exceed the limits in (<ref>). Let the solution of the power flow equations in (<ref>) representing the reactive power injection at bus n∈𝒢 be denoted by q_g,n^o. Also, denote the set of buses where the reactive power limits are violated by 𝒢_Q. Then, we define q_g,n^r at the buses 𝒢_Q as 


    q_g,n^r = min(q_g,n,  max(q_g,n^o,  q_g,n)).

Consequently, a modified power flow problem is to be solved in order to obtain the feasible operating point of the network which is formulated as


    find        v,  p_g,  q_g
    subject to        |v_n| = |v^o_n|   ∀ n∈𝒢𝒢_Q
            p_g,n = p_g, n^o   ∀ n∈𝒢 n_0
            q_g, n = q_g, n^r   ∀ n ∈𝒢_Q
            h( v,  p_g,  q_g) =  0




The proposed approach is summarized in Algorithm  <ref>. While an additional small amount of time is needed to solve the power flow equations and check the feasibility of all of the remaining variables, the overall solution procedure from end-to-end is extremely fast, as we are dealing with a simpler problem which is solving the power equations and not solving a nonconvex optimization problem. In addition, note that the objective function value in the original problem (<ref>) is not altered by adjusting reactive power and voltage to preserve feasibility.

inputAInput: 
outputAOutput: 




§ SIMULATION RESULTS

In this section, we discuss the considered networks and analyze the optimality and feasibility obtained using the proposed approach. MATPOWER <cit.> was used to generate 100,000 training samples for each considered network and to solve the power flow equations for the obtained optimal generator set-points.



 §.§ Test networks and setup


Table <ref> shows the number of nodes, generators, lines, and the base operating costs of the IEEE 118-, 57-, and 39-bus systems used in this paper. 


For the NN network model, we utilized a NN with 3 hidden layers with sigmoid activation function for all layers. The width of the first two hidden layers is equal to the number of inputs (2 |ℒ|) and the width of the third hidden layer is equal to the number of outputs (2 |𝒢| -1) for all networks. The learning model was implemented using the Python-based TensorFlow software library <cit.> and trained using the Adam optimizer <cit.>.




 §.§ Time comparisons

We denote the time consumed to obtain an AC OPF solution using our method for the t-th test instance by τ_o,t, where this time includes the time needed to evaluate the neural network, the time needed to solve the power flow equations, and the time consumed by solving any additional power flow equations to ensure the feasibility of the solution. For comparison purposes, let τ_OPF, t denote the time consumed by the 'PDIPM' (primal-dual interior point method) solver to solve the original AC OPF problem optimally. In the considered experiments, this solver was on average the fastest and most successful solver. To measure the computational improvements, we define a speedup factor (SF) as


    SF = 1/T∑_t=1^Tτ_OPF,t/τ_o, t,

which we can use to measure the relative speedup of the learning-based framework versus solving the AC OPF directly. 

In Fig. <ref>, the relative histogram of the speedup factor is depicted for the IEEE 118-bus network with λ = 0.005. The speedup factor ranges from 6x to 22x for this case, which shows the potential of proposed approach to identify feasible AC OPF solutions extremely quickly, even in the slower cases. The two humps in the histogram result from the fact that the second stage of the algorithm is only needed for a portion of the test cases.   





 §.§ Feasibility and optimality

After passing the network demands through the learning model, we obtain the generator set-points. Those set points are guaranteed to satisfy constraints (<ref>) and (<ref>) given the design of the output layer of the neural network. Further, to ensure feasibility of the overall AC OPF solution, the infeasibility of the generated solution with respect to the reactive power limit constraints (<ref>) needs to be ensured by solving the power flow equations, as outlined in problem (<ref>). We evaluate the infeasibility of this constraint for the solutions obtained directly using our learning model by defining a metric δ_q which is given by

    δ_q = 1/T∑_t=1^T1/|𝒢|ξ_q,t_2,

where ξ_q,t collects all the reactive power limit violations across all buses. The n-th element of the vector ξ_q,t is given by

    ξ_q,t,n =  max{q_g,n - q_g,n^o, 0} + max{ q_g,n^o - q_g,n, 0},

where q_g,n^r denote the recovered reactive power injection at bus n by solving the power flow equations at the set point given by the output of the neural network.

Let p_g^⋆ denote the optimal generation set-points obtained by solving the AC OPF problem (<ref>), and denote the generators' active power injection set-points obtained by our data-driven approach by p_g^o. The average optimality of the solutions obtained through the proposed approach is evaluated with the following

    Optimality = 1/T∑_t=1^Tc ( p_g,t^o) - c ( p_g,t^*)/c ( p_g,t^*) ,

where c ( p_g,t^o) and c ( p_g,t^⋆) denote the total objective function value resulting from the solution obtained using our approach and the optimal cost found from solving the original AC OPF, respectively.

Denote by p_t^o, q_t^o, and v_t^o the solution obtained using our approach for a particular test scenario t. The feasibility of the solution is measured by assessing the satisfaction of the power flow equations. The maximum infeasibility is thus evaluated as follows:

    Feasibility = max_th(  v_t^r,  p_t^r,  q_t^r ) _2.


Tables <ref>, <ref>, and <ref> illustrate the results from evaluating 1,000 AC OPF test scenarios for each network. 
In the 118-bus test case, the initial infeasibility of the solution is higher when the value of lambda is set to be zero. This is attributed to the fact that the training samples in this case (λ =0) are on the boundary on the original problem feasibility set. For this reason, we did not include the results for (λ =0) for the other test cases.

As seen in all of the network simulations, the maximum infeasibility observed in the all of the test scenarios is very low (and well-within most solver tolerances for feasibility). The average computational speedup by using the data-driven approach versus solving the AC OPF directly is also significant, ranging from nearly a 8x speedup factor up to over a 15x SF. These tables also show the average infeasibility, δ_q, of the reactive power limits (<ref>) encountered, in MVAR, after solving the power flow equations with the learned generator active power set-points and voltages. Note, however, that the feasibility of these constraints is ensured using the procedure in Algorithm <ref>, and the time required to do this is taken into account in the results.  








Further restricting the feasible set of voltages by increasing λ from 0.005 to 0.01 generally has a positive impact on feasibility while mildly impacting the overall optimality of the solution. However, regardless of the λ chosen here, the impact on the optimality of the learned solution is minimal for all considered networks, indicating that using the learning-assisted approach of recovering AC OPF solutions can preserve optimality and feasibility while providing significant computational speedups. 




§ CONCLUSIONS AND FUTURE WORK

In this paper, we presented a framework for solving AC OPF problems which utilizes machine learning to learn a mapping between system loading and optimal generation values. We provided a method to guarantee feasibility of the solution in the original AC OPF problem by training the model on strictly feasible solutions and then solving the power flow equations to recover the remainder of the complex network voltages and the reactive power generation values. From start to finish, the framework can recover a feasible solution to the AC OPF problem extremely quickly with a negligible sacrifice in optimality. Thus, the learning framework is overall faster and can provide a more accurate solution than the use of traditional methods that are used to solve AC OPF quickly such as linearizing or approximating the nonlinear power flow equations.

Future work includes exploring the efficacy of this framework on much larger networks (i.e., thousands of buses), and using learning to solve computationally intense, mixed integer problems such as security constrained AC OPF problems on faster timescales. The results in this paper also prove encouraging for using learning as a tool for quickly finding solutions to nonconvex optimization problems in areas outside of power systems. In potential applications where learning the optimal solution while ensuring feasibility is not possible or very difficult, we ask the reader to consider developing or using learning-assisted or learning-enhanced optimization techniques, wherein machine learning and optimization work hand-in-hand to find optimal solutions of the original problem, either by using learning to quickly find a starting point for the optimization, or by learning the solution of a subset of variables. 





IEEEtran




