Abstract
This paper examines uncertainty in the maneuver execution delay for data linked conflict resolution maneuvers. This uncertainty could cause the previously cleared primary conflict to reoccur or a secondary conflict to appear. Results show that the likelihood of a primary conflict reoccurring during a horizontal conflict resolution maneuver increases with larger initial turn-out angles and with shorter times until loss of separation. There is also a significant increase in the probability of a primary conflict reoccurring when the time until loss falls under three minutes. Increasing horizontal separation by an additional 1.5 nmi lowers the risk, but does not completely eliminate it. Secondary conflicts were shown to have a small probability of occurring in all tested configurations.
Introduction
As part of the transition to the Next Generation Air Transportation System, or NextGen [1] , it is important to understand the sources and effects of uncertainty. This is especially important in the area of separation assurance, as resolutions probed for conflicts by automation will often assume the aircraft follows its flight plan to allow the algorithms to check for conflicts in the eight to ten minute range. However, when an aircraft receives a conflict resolution trajectory that has been checked for conflicts, there is some natural variation in how long it will take the flight crew to implement that resolution. This time can depend on the current work load of the flight crew, the complexity of the maneuver, or other factors. The effects of this delay, referred to as the maneuver execution delay, need to be understood before automation-probed resolutions are implemented in the real world. Specifically, can variation in the maneuver execution delay of a conflict-probed resolution maneuver result in the conflict reoccurring, or a different conflict occurring?
If so, what is the rate of occurrence, and are there simple ways to reduce this rate?
There is some recent work studying the effects of varying pilot response times to automated resolutions. Consiglio et al [2] examined the effects delayed pilot responses to conflict alerts in an airborne self-separation system, and McNally et al. [3] included uncertainties in the execution of resolution maneuvers as part of an overall Trajectory Based Operations study. There has also been a human-in-the-loop study measuring pilot response times to data linked conflict resolution messages using a data link equipped aircraft [4] . These studies do not explicitly explore the effects of maneuver execution delay, however.
The purpose of this study is primarily to determine the probability of a primary conflict reoccurring after a resolution has been issued due to variation in maneuver execution delay, and how this probability varies with various parameters. This work should give a more complete picture of the effects of varying pilot response times and some preliminary thoughts on ways of mitigating the risks of this uncertainty.
Background
Automated conflict probes can be used to increase the certainty that a conflict has been cleared and attempt to find conflict resolutions that add as little delay as possible to the original flight plan. More delay efficient resolutions keep an aircraft closer to its original schedule and should help minimize the extra fuel burn required by the maneuver.
These probes are especially effective when one can use "closed-form" resolutions, which are simply maneuvers that take an aircraft off of its flight plan to avoid a conflict and then return it to the original flight plan at some later point. These types of maneuvers allow for a degree of certainty in conflict probing, since the entire resolution trajectory is defined. They also allow the use of a longer "look ahead" time, which is how far into the future the algorithm is looking for conflicts. These closed-form maneuvers can also be optimized to minimize the 3.B.2-2 amount of delay absorbed by a flight, as the delay for a resolution maneuver can be estimated by comparing the expected flight times of the original trajectory and the resolution trajectory.
These closed-form conflict resolutions do have some drawbacks. One is that these maneuvers can be complicated to issue and may require the presence of a Flight Management System to implement. As a result, these maneuvers tend to be more suited to data link communication. These resolutions are also built with an implicit starting point, such as an initial turn or begin climb point. However, transmitting this point as an explicit part of a resolution with the expectation that the aircraft will begin maneuvering exactly when expected significantly increases the time required by the flight crew to implement a maneuver.
For horizontal maneuvers, for example, this delay is on the order of two minutes [4] , which is a little more than twice the nominal time observed in that same reference. Using an expected delay of two minutes means a conflict detected with eight minutes must use a resolution that only has six minutes available for the aircraft actually to maneuver, while planning for the nominal execution delay of about one minute allows for seven minutes of maneuvering time. This extra minute of expected maneuver execution delay will generally result in the creation of less efficient resolutions, especially in the horizontal plane (where a larger turn might be required to get clear). Thus using an explicit start point effectively forces most aircraft to fly resolutions that are designed for longer execution delays than they actually required, which could mean unnecessary efficiency losses for those flights.
For vertical maneuvers there is a different complication: in current systems altitude changes are dialed in manually by the flight crew. This makes executing the climb or descent portion of a vertical conflict resolution at a specific point much more difficult to do.
It is therefore desirable to send a conflict resolution message without an explicit starting point, if possible, as doing so will allow for simpler messages and reduced nominal execution times. This approach comes with the risk of an aircraft executing a maneuver at some off-nominal time, and perhaps a very off-nominal time, hence the importance of understanding the effects of maneuver execution delay.
For this study, variation in maneuver execution delay was implemented as a deviation around some nominal execution delay for each maneuver, and those deviations were checked for conflicts. As different procedures are needed for implementing different maneuver types, different nominal delays for horizontal and vertical maneuvers were used. The values of these delays were taken from the previously mentioned reference [4] .
Conflict Resolution Maneuver Execution Delay
As mentioned previously, conflict resolution maneuver execution delay is the delay between when a conflict resolution maneuver is sent to an aircraft via data link and when the aircraft actually begins flying this maneuver. In effect, the error in maneuver execution delay takes an aircraft off of its predicted trajectory for the initial portion of the maneuver. For closed-form horizontal maneuvers, this difference is mostly constrained to the first turn. In the case of path stretches, the aircraft does eventually recapture its original flight path at the turn-back waypoint, but it will either be late or early in arriving at that point depending on whether the aircraft turned earlier or later than expected. An illustration of path stretch maneuver execution delay uncertainty is shown below in Figure 1 , where d is the range of uncertainty.
Figure 1. Path Stretch Execution Delays
For vertical maneuvers, an aircraft would begin climbing or descending earlier or later than expected (climb/descent rate uncertainty is not studied in this paper), then hold its cleared altitude. However, unlike horizontal maneuvers, the return is also an area of uncertainty for vertical resolutions. In today's system, a closed trajectory resolution that brings an aircraft back to its initial altitude would require two messages: an initial maneuver and a return maneuver. The return maneuver for vertical resolutions would 3.B.2-3 be subject to the same uncertainty as the initial one. This is illustrated in Figure 2 , where d 1 and d 2 are the maneuver execution delay uncertainties for the initial and return maneuvers, respectively.
Figure 2. Step Altitude Execution Delay
Overall, these errors are relatively small when compared to the overall length of most resolutions, but they do result in an aircraft flying slightly off of its predicted path. This leads to a small probability that the conflict resolution maneuver, if executed at the wrong time, may no longer clear the initial, or "primary," conflict. Additionally, these errors can force an aircraft that is nearby but not involved in the primary conflict to come into conflict with the maneuvering aircraft. This is called a "secondary" conflict.
Dealing With Uncertainty
A secondary objective of this study is to look at the effects of some simple mitigation strategies for reducing the risk of a primary conflict reoccurring or an unexpected secondary conflict occurring due to variations in the resolution maneuver execution delay. There are three mitigation ideas that will be discussed in this paper: an explicit starting point for a resolution maneuver, increased horizontal spacing requirements for a resolution to be successful, and scanning a range of airspace rather than a single trajectory for conflicts along a possible resolution. As the pros and cons of an explicit maneuver starting point have already been pointed out, only the other two will be discussed here.
Increasing the horizontal spacing requirements for a resolution to be successful is in effect adding a "buffer" onto the required horizontal separation for all of those resolutions. This is a strategy that has been used to mitigate other types of risks associated with uncertainty, such as wind errors [5] . The advantage of this approach is that it is relatively simple, both in concept and implementation. It increases the minimum spacing that aircraft have when a maneuver is issued, and thus it increases the amount of error that can be absorbed before the fixed minimum separation (5 nmi) is violated. The drawback to using this approach to cover maneuver execution delay uncertainty is that there are delay efficiency penalties. For example, nearly all of the error in path stretches due to execution delay after the turn-back waypoint is in the along-track direction, as the aircraft is simply flying from one defined waypoint to another. Adding a buffer as they are currently implemented to account for this along-track error would also create unneeded additional crosstrack separation. This would force the aircraft to fly a longer and less delay efficient resolution than it really needed to. The effect of increasing buffers is explored in this study, but only in terms of its effectiveness at reducing the risk of a conflict occurring. Delay and other system metrics are beyond the scope of this paper, though previous work examining the effects of increasing horizontal separation buffers has shown increased delay per resolution with larger buffers [6] .
Scanning an area of airspace rather than a single trajectory is an approach that has been used to deal with things such as climb uncertainty [7] , and could be used here, as well. Effectively, if the aircraft is expected to maneuver within plus or minus 20 seconds of some nominal execution delay, the conflict detection algorithm is asked to check trajectories that cover that range of possibilities instead of just the nominal value. The advantage to this approach is that it can cover the range of uncertainty directly, without forcing resolution maneuvers to lose more efficiency than needed. The downsides to this approach include extra computational time requirements for each candidate resolution, and the time required to develop such a system, as there is not one currently in the field. This is similar to the approach used in this study to determine if a conflict occurs due to maneuver execution delay uncertainty, as will be discussed in the Simulation Implementation section. However, fielding an algorithm such as this would require additional development work and testing.
3.B.2-4

Experimental Approach

Simulation Implementation
The simulation used for this study is the Airspace Concept Evaluations System, or ACES [8] . It is a fast-time simulation capable of simulating the entire NAS. For this study, a full day of high volume, low weather-impact traffic from April 2007 was run through Cleveland Center.
The Autoresolver from the Advanced Airspace Concept, or AAC [9, 10] , was used to create possible resolutions, or "trial plans," for this simulation. There have been numerous studies using the Autoresolver, though for the purposes of this study it was primarily used to generate successful conflict resolution maneuvers for a variety of conflicts. The AAC arrival manager was also used to meter aircraft at the major hub airports within Cleveland Center.
For each conflict, the Autoresolver produced and checked a variety of trial plans and attempted to find horizontal, vertical, and speed maneuvers that successfully resolved each conflict. For this study, every successful trial plan for each conflict was called a resolution and was used as a baseline, nominal case. As system metrics were not being examined, there was no special note made of which of the successful trial plans AAC actually decided to implement to resolve a specific conflict.
This study took those resolutions and rechecked them assuming the aircraft executed the maneuver at a different time than the nominal delay value. This was done by building a new trial plan with an alternative start time determined by adding a discrete maneuver execution delay error on to the nominal, baseline value. ACES conflict detection logic was then used to determine if the new trajectory was also free of conflicts. These trial plans with alternative start times were not sent to either the aircraft or AAC's Autoresolver. This approach allowed for any number of delay values to be checked for each resolution using a single data run.
If the variation in maneuver execution delay caused a conflict to occur, the new conflict information was stored and analyzed. Details of interest included whether or not the conflict was the primary reoccurring or a secondary appearing, and if moving early, late, or both resulted in a conflict.
Variables
The primary independent variable in this study was conflict resolution maneuver execution delay. This is the delay between when a conflict resolution maneuver is sent to an aircraft via data link and when the aircraft actually begins flying this maneuver. For the purposes of this study, the baseline resolutions used a nominal delay of 56 seconds for horizontal maneuvers and 39 seconds for vertical ones based on pilot response data from the Mueller and Lozito study [4] .
There were four alternative start times used for horizontal resolution maneuvers: 35 seconds early, 20 seconds early, 20 seconds late, and 50 seconds late. These values cover roughly 95% of the response times observed in the Mueller study [4] . The use of four additional execution delay times was due to the wide range of response times observed.
The alternative start times for vertical maneuvers times were 20 seconds early and 20 seconds late. These maneuvers were simpler to implement for the pilots and thus had lower nominal times and a tighter distribution. This variation covers the full range of response times observed in the previously mentioned Mueller study [4] .
A secondary independent variable was the horizontal resolution buffer added for conflict resolution. This is simply an amount of horizontal separation added to the 5 nmi minimum that is required for a trial plan to be considered a successful resolution. For example, an added 0.5 nmi buffer would mean that a conflict flagged because two aircraft got within 5 nmi of each other would only be cleared if the resolution kept the aircraft at least 5.5 nmi apart. As discussed previously, these buffers are commonly used to prevent the reoccurrence of conflicts due to general uncertainties [5] .
For the baseline cases in this study, there was no buffer added for conflict resolution. In other words, five nautical miles was used as the required separation for both conflict detection and resolution. Buffers of 0.5 and 1.5 miles were added to see what the effects of these buffers would be on the likelihood of a conflict occurring in a cleared resolution due to variation in the maneuver execution delay.
3.B.2-5

Metrics
The only primary metric of this study was the probability of a conflict occurring in a "successful" resolution after the addition of variations in maneuver execution delay. This was divided into two cases: the primary conflict reoccurring, or a secondary conflict unexpectedly occurring. The probability of the primary conflict reoccurring is the main focus because the likelihood of a secondary conflict occurring is dependent on the state of the surrounding airspace for each conflict. This made it more difficult to determine which specific factor caused a secondary conflict to occur.
Data Analysis Approach
For each type of maneuver, several key factors were examined to check their impact on the primary metric. For horizontal maneuvers, these included the initial turn-out angle for a resolution, the time until loss of separation, and the minimum horizontal separation achieved by the baseline resolution. These factors were chosen because they seemed likely to influence the probability of a conflict occurring due to variations in maneuver execution delay.
For vertical maneuvers, the previously discussed complication that each maneuver was actually two maneuvers combined required each vertical trial plan to be treated as two maneuvers. Thus, in this study, each vertical maneuver is checked to see if aircraft maneuvering early or late both at the beginning or end of a maneuver would cause a conflict to occur. Climb rate uncertainty is not part of this study.
In total the simulation consisted of six data runs. Each run collected data for all execution delay values for either en route or merging-arrival cases at specific buffer size. Merging-arrival conflicts are conflicts between two aircraft headed to the same fix, and both within 20 minutes of that fix. These were treated as a special case. En route conflicts consisted of anything that was not a merging-arrival conflict. Maneuver execution delay trial plans based on en route resolutions were checked for conflicts up to 12 minutes from the time the resolution was issued, while merging-arrival trial plans were checked up to 20 minutes ahead. In total, there were three en route data collection runs (one for each horizontal separation buffer value), and three merging-arrival runs. It should be noted that merging-arrival conflicts were still being solved by the Autoresolver during the en route data collection runs (and visa versa), they just weren't analyzed.
Results
This section will present simulation results and analysis. Data for the en route portion of flight will be presented first, followed by the special case of merging-arrival conflicts. "En route" conflicts consist of anything that is not a merging-arrival conflict. As mentioned above, merging-arrival conflicts are conflicts between two aircraft headed to the same fix, and both within 20 minutes of that fix. Because they are required to go through the same point in space at the fix and are nearing or past their top of descent point, the maneuvering options for these conflicts are very limited.
En Route Results
The en route results presented in this paper consist primarily of horizontal conflict resolutions. Vertical results are for step altitude changes only, and do not include results for aircraft leveling off in a climb or descent. Note that these results reflect the worst cases of aircraft maneuver execution times, both early and late. Therefore, a 10% probability of a conflict occurring does not mean that every resolution will have this 10% probability. Instead, it means that if an aircraft maneuvers at the extreme early or late edges of the maneuver execution delay distribution for a certain type of maneuver, there is a 10% probability of a conflict occurring.
Horizontal Resolutions
The first focus area is horizontal resolution maneuvers. The analysis here is focused on path stretch maneuvers, which consist of a single auxiliary waypoint and a return fix. These maneuvers are quite effective for clearing conflicts and are one of the primary resolution types checked by the AAC. Other types of horizontal maneuvers include offset or Direct-To maneuvers.
For these data a total of 2068 path stretch maneuvers were considered successful resolutions, as shown in Table 1 . Of these, in 252 cases the primary conflict reoccurred for at least one of the delay variations, and in 83 cases secondary conflicts appeared. That means there is a 12% probability of a 3.B.2-6 primary conflict reoccurring if an aircraft executes a resolution maneuver at the edges of the expected distribution. There is also a 4% probability that a secondary conflict will occur given similar aircraft behavior. To further understand the data, it is helpful to break down the conflict information by when an aircraft executes the horizontal maneuver. This data is in Table 2 . The first thing to note is that the number of cases in this table does not add up to the number of cases in Table 1 . The reason for this is that each value of maneuver execution delay was recorded independently of the others. In other words, if a case saw a primary conflict reoccur when the aircraft maneuvered 20 seconds late and 50 seconds late, it would get recorded in both rows. There were also a handful of cases where maneuvering 50 seconds late or 35 seconds early both caused a primary to reoccur, though this was very rare.
As seen in Table 2 , it is more detrimental to execute an en route horizontal resolution maneuver late than early with regard to the primary conflict. While the larger range of values for late maneuvers is part of the reason they had the largest value in the table, the cases with plus and minus 20 seconds of delay reflect the same result. There is no clear trend in the secondary conflicts except that greater deviations from the nominal execution delay produce a greater likelihood of becoming involved in a secondary conflict, as expected. The effect of initial turn-out angle on the likelihood of a conflict reoccurring was also analyzed. The initial turn-out angle is the turn angle an aircraft must make to reach the off-trajectory waypoint, assuming it begins maneuvering at the nominal value of maneuver execution delay. It is a feature of horizontal resolutions since aircraft are generally given some off-trajectory waypoint and return point in a closed trajectory conflict resolution. The data for primary conflicts are shown in Figure 3 . They show there are more low-angle resolutions than high-angle ones, which is to be expected from a system that is trying to minimize delays. The most interesting result is how quickly the likelihood of a primary conflict reoccurring increases as the turn-out angle increases. This is most likely due to along-track errors caused by the aircraft being either behind or ahead of its expected resolution flight plan. This error is caused by the fact that an aircraft executing a maneuver early will turn to the given waypoint at a shallower angle and actually fly a shorter distance to that point than expected and reach the end of the resolution earlier than anticipated. Similarly, executing a maneuver late will cause the aircraft to arrive at its auxiliary waypoint late and fly the rest of the resolution behind the scheduled time. The probability of a conflict reoccurring increases from 5% for 15-degree turn-out angles to around 25% for 45-degree turn-out angles.
Figure 3. En Route: Turn-Out Angle Effects on Primary Conflicts
The effect of turn-out angle on the probability of a secondary conflict is noticeably less, as shown in Figure 4 . While larger initial turn-out angles do cause an increase in the likelihood of a secondary conflict, the slope is much shallower than the one for primary conflicts. At 75 degrees, the probability of a secondary is 13%. While this is lower than the 50% seen in the previous chart, it is not an insignificant number.
3.B.2-7

Figure 4. En Route: Turn-Out Angle Effects on Secondary Conflicts
Increasing the required separation for a resolution to be considered successful generally decreased the probability of a primary reoccurring (see Figure 5) . However, the amount of improvement was only on the order of a few percent for any given turn angle for the buffer sizes examined. It is difficult to get a strong sense of the actual rates at higher turnout angles due to smaller sample sizes.
Figure 5. En Route: Turn-Out Angle Effects on Primary Conflicts with Different Buffers
The next item looked at was the effect of the time until loss of separation when the resolution maneuver was issued. Results are shown in Figure 6 . The large number of cases in the six-to eight-minute range is due to the preference of the Autoresolver to do everything it can to solve conflicts early. There is a noticeable jump in the probability of a primary conflict reoccurring when the time until loss at the moment the maneuver would be issued falls to three minutes or less. Much of this is due to physical constraints. Each horizontal resolution has a nominal maneuver execution delay of 56 seconds. When that is coupled with an additional 50 seconds of delay, it results in a worst case scenario where the aircraft waits nearly two minutes from the time a resolution is created until it begins to execute it. Even though only one minute of that was unplanned by the trial planner, if a conflict only had three minutes until loss of separation to begin with, two minutes of inaction leaves very little time to actually resolve the conflict. This points to a possible lower bound for allowable use of a closed-form horizontal conflict resolution. While it could be possible to issue a horizontal conflict resolution with a command to move immediately, it might be more logical to just look for alternative resolutions once the time until loss gets below three minutes. There are projects that are built for this time range, such as NASA's TSAFE [11] , but work looking at the effects of adding those tools was well beyond the scope of this project. Figure 7 shows the likelihood of a secondary conflict occurring as a function of time until loss of separation. The results indicate that secondary conflicts have a small but not insignificant probability of occurring regardless of the time until loss.
3.B.2-8

Figure 7. En Route: Time-Until-Loss Effects on Secondary Conflicts
Increasing the required separation for resolutions had little effect on conflicts with under three minutes until loss of separation, as seen in Figure 8 . However, conflicts with longer times until loss did see the likelihood of a primary conflict decrease by half or more. As with turn-out angle, there was little effect on the probability of secondary conflict occurrence when increasing the required separation, though there seems to be a small improvement overall.
Figure 8. En Route: Time-Until-Loss Effects on Primary Conflicts with Different Buffers
The final variable used to sort these results was the minimum separation achieved by the conflict resolution. These results are shown in Figure 9 and give an indication of the importance of this minimum spacing, especially when dealing with primary conflicts that might reoccur. It is intuitive that the larger the separation achieved, the less likely it is a conflict will reoccur due to some uncertainty that moves the aircraft off its predicted trajectory. However, larger separation distances also mean longer resolution trajectories, which mean longer delays for aircraft flying those trajectories.
The curves in Figure 9 show no local minima or plateaus, but things do start leveling out around 6.5 to 7.0 nmi. A near-zero value for primary conflict reoccurrence probability would require over 9 nmi of required separation. This would significantly increase the average delay per resolution for the system, based on the results of a previous study [6] . The reason there are still conflict resolutions given with less than 5.5 nmi of separation, even in the buffered cases, has to do with the way the Autoresolver handles difficult conflicts. If the Autoresolver decides it has no other options, it will accept "reduced separation" resolutions that have a minimum separation which is less than the buffered, desired value and greater than the absolute minimum value (5 nmi in all cases). That is also why the probability of a primary conflict reoccurring is mrogers@example.org nmi and less for the buffered cases than the baseline case. In the buffered cases, any resolutions with separation values that small have only been accepted because the conflicts were difficult to solve. Thus, adding additional uncertainty to the system is more likely to result in the primary conflict reoccurring in these reduced separation cases. 
Vertical Resolutions
The only vertical resolution maneuvers analyzed in detail were step altitude changes. These are effectively temporary altitudes given to an aircraft in level flight, with the expectation that the aircraft will 3.B.2-9 be returned to its original altitude generally within the next 15 minutes. The analysis of these maneuvers was complicated by the fact that each resolution was actually two maneuvers due to the closed-form nature of the Autoresolver resolutions: an initial maneuver and a return. On the other hand, the analysis was done with only two off-nominal points (20 seconds early and 20 seconds late) because the distribution of possible maneuver execution delays was narrower for vertical resolutions. Table 3 shows the overall results from the vertical maneuver analysis, treating each initial and return pair as a single maneuver. The first thing to note is that there are significantly more cases where the primary conflict reoccurs than a secondary occurs. To understand the reason for this, a further breakdown is required. Table 4 shows the breakdown by primary and secondary conflicts, and whether the conflict came because the aircraft executed a maneuver early or late, for both the initial and return maneuvers. The most striking number from Table 4 is the relatively high number of cases where an aircraft executing the maneuver to return it to its original altitude early causes the primary conflict to reoccur. This is almost certainly an artifact of the way the Autoresolver handles vertical resolutions and the way maneuver execution delay was implemented in this study. While the Autoresolver will account for execution delay when issuing the initial clearance, it assumes the aircraft will begin descending as soon as the conflict is clear. In an operational sense, this means issuing the second half of the clearance 36 seconds before the planned descent time. However, as Autoresolver treats the entire clearance as a single maneuver, there is currently no buffer added to the second portion of the resolution to account for the possibility of an aircraft executing that portion earlier than expected. Thus, an early execution of the return maneuver has an increased likelihood of returning an aircraft to its original altitude before the conflict is completely cleared. This will likely be a danger for any conflict resolution that is probed in a closed form, but transmitted to a pilot in multiple pieces, as two nominal delays will actually need to be built in.
Excluding the early return maneuver cases, Tables 3 and 4 show there are not many cases where executing a maneuver early or late causes conflicts to occur overall. This is partly attributable to the tighter bounds on the distribution maneuver execution delay uncertainty. One would expect uncertainty in the plus or minus 20 second range to see fewer problems than the plus 50 to minus 35 second range for horizontal maneuvers. With that said, executing a vertical maneuver just 20 seconds late still caused the primary conflict to reoccur in a few cases. Figure 10 shows the likelihood of a primary conflict reoccurring during the initial maneuver of a vertical resolution with respect to time until loss of separation. This chart shows much lower percentages than the analogous horizontal chart (Figure 7) , though resolutions issued with less than four minutes until loss still have a higher probability of a primary conflict reoccurring. However, it must be noted that these data are only for maneuver execution delay variation and do not take into account unexpected variations in climb or descent rates. Even with excellent aircraft performance models, climb rate variation combined with maneuver execution delay variations could significantly increase the probability of a primary conflict reoccurring. With that said, it is encouraging to see that maneuver execution delay uncertainty does not affect the probability of a primary conflict reoccurring above four minutes until loss of separation.
3.B.2-10

Figure 10. En Route: Time-Until-Loss Effects on Primary Conflicts in Vertical Resolutions
The results for the effects of increasing horizontal buffers are not included in the results for vertical maneuvers, as they had a relatively minor effect on the probability of the primary conflict reoccurring.
En Route Results Summary
These are a few key points to pull from the en route results.
• The likelihood of a primary conflict reoccurring for horizontal resolutions increases with increasing initial turn-out angle. This effect is somewhat mitigated by increasing horizontal buffer size.
• The probability of a primary conflict occurring in a resolution increases greatly once the time until first loss of separation when the resolution issued falls under three minutes. This is especially true with horizontal resolutions. Mitigation due to increased horizontal buffers is only noticeable above three minutes until loss of separation.
• The likelihood of a primary conflict reoccurring decreases overall as extra horizontal buffer is added to a resolution, but the probability is not driven to zero for the range of buffers studied.
• Vertical resolutions that attempt to bring an aircraft back to its original flight level need to allow a second maneuver execution delay buffer on the return maneuver, as executing the return early produces a very high probability of a primary conflict reoccurring.
• Secondary conflicts have a small but nonzero probability of occurring in all data runs.
Merging-Arrival Results
Merging-arrival conflicts are those that occur between two aircraft headed to the same final fix, with both aircraft less than 20 minutes from that fix. Additionally, because aircraft are near their top of descent points, or are already descending, the types of resolutions available are limited. Vertical resolutions in particular are rare due to the restrictions on aircraft flight path. With that in mind, only horizontal path stretch maneuvers were analyzed in this regime. Table 5 summarizes the merging-arrival conflict data. The first thing to note is that there are far fewer cases to work with in these cases, so trends need to be clearly identifiable. The results shown in Table 6 indicate that there is significant risk of a primary conflict reoccurring if an aircraft executes a resolution maneuver early. This has to do with the situation at the meter fix. Because both aircraft are headed for the same fix, they are guaranteed to have flight plans that converge to the same point. Therefore, separation between aircraft must be maintained by assuring the aircraft pass through that point at times that are at least some minimum time apart. This is similar to a scheduling problem. Therefore, a horizontal conflict resolution achieves separation by delaying one aircraft until there is enough time between the two aircraft for them to fly through the same fix safely. An aircraft executing a path stretch resolution maneuver early actually reduces the delay it accrues while flying along the resolution, as it is flying a shorter distance, as was illustrated in Figure 1 . The end result is an increased probability that there is no longer enough temporal spacing between the aircraft to ensure the positional spacing requirements are met at the meter 3.B.2-11 fix. Fortunately, such things as increased separation requirements and fixed maneuver execution times can significantly mitigate this risk. Figure 11 shows the relationship between turn angle and the probability of a primary conflict reoccurring in merging-arrival cases. It is very similar to the equivalent en route case (Figure 3) , though the slope of the line is slightly steeper in the merging-arrival case. This is largely due to the complications mentioned in the preceding paragraphs, especially that executing a resolution maneuver early can result in less delay than required to ensure safe separation at the meter fix. Simply due to geometry, larger initial turn-out angles will result in a greater time difference at the meter fix due to the aircraft flying shorter or further than required. However, it should be noted that increasing the required separation for resolution maneuvers quickly brings this curve down, as seen in Figure 12 . These values are below those seen in the en route case. This mitigating effect is likely the result of a decreased probability that executing a maneuver early causes the conflicted aircraft to have insufficient spacing at the meter fix. This indicates that merging-arrival path stretch resolutions might benefit more from increased separation buffers than en route path stretches. Interestingly, Figure 14 shows that there is no strong relationship between time until loss of separation and the probability of a primary conflict reoccurring. This is partly attributable to lower sample size and the longer look-ahead times used in merging-arrival conflicts. This allows aircraft to begin maneuvering well before the maneuver execution delay constitutes a significant percentage of the time available to clear the conflict. This leads to one of the possible mitigation strategies mentioned earlier in this paper: fixed maneuver-start waypoints 3.B.2-12 for resolutions. These are generally seen as undesirable since they make maneuvers more complicated. They will also require the maneuver to allow for a delay on the order of two minutes before execution, to ensure that the aircraft has not passed the waypoint before the pilots are ready to execute the maneuver. While this can be constraining for conflicts with 5 minutes until loss of separation, it is not a big issue when there are 18 minutes to loss.
Figure 14. Arrivals: Time-Until-Loss Effects on Primary Conflicts
This point is further bolstered by the data in Figure 15 , which shows the relationship between minimum separation achieved and the likelihood of a primary conflict reoccurring. If the Autoresolver is forced to account for another 0.5 miles of separation at the meter fix when issuing a resolution, the probability of a conflict occurring when the minimum separation is between five and six nmi drops from 70% to under 10%. The reason for this steep decline is most likely the temporal spacing issue that was discussed previously. In merging-arrival conflicts, the minimum spacing achieved by a proposed resolution is almost secondary to the temporal spacing achieved at the meter fix since the physical spacing is guaranteed to be zero there if two aircraft arrive at the same time. Thus, that extra 0.5 miles of separation adds extra temporal spacing. Also, depending on the geometry of the conflict, that seems to be enough to alleviate most of the spacing issues of these types of conflicts. 
Merging-Arrival Results Summary
• For merging-arrival conflicts, executing a horizontal resolution early produces a very high probability of a primary conflict reoccurring. This is because an aircraft that maneuvers earlier than expected does not accrue the delay required during the resolution to maintain separation at the meter fix. This risk is significantly mitigated by horizontal buffers of 0.5 nmi or greater.
• As with en route resolutions, the likelihood of a primary conflict reoccurring for horizontal resolutions increases with increasing initial turn-out angle. Unlike the en route resolutions, this likelihood is significantly mitigated by horizontal buffers.
• There is no strong correlation between time until loss of separation and the probability of a primary conflict reoccurring for merging-arrival resolutions.
• As with en route resolutions, secondary conflicts have a small but non-zero probability of occurring in all data runs.
Conclusions
The first conclusion that can be made is that additional horizontal separation requirements for resolution trajectories will decrease the risk of a primary conflict reoccurring due to maneuver execution delay uncertainty. However, the buffer sizes required to completely remove this risk are 3.B.2-13 larger than the 1.5 nmi examined in this study. Therefore, while buffers are certainly helpful, other means will need to be used to remove the risks of maneuver execution delays uncertainty, such as scanning early and late maneuver trajectories for possible conflicts before issuing a maneuver. One notable point to make in addition to this is that a small amount of buffer can be very helpful in reducing the risks in the arrival environment.
A second conclusion that can be made is that horizontal conflict resolutions should not be a first choice with less than three minutes until loss of separation in the en route environment. This point is important because, barring a big improvement in data link response times to closed-form horizontal trajectories, it is very difficult to mitigate the increased risk in this regime. There is not enough time to issue a fixed maneuver start point, nor is increasing the horizontal separation for a resolution going to provide enough mitigation of this risk. It is more attractive to either issue a vertical resolution, if available, or to perhaps go to voice, or even issue a tactical vectoring solution, which should have a significantly lower response time.
A third conclusion is that it is difficult to reduce the risk of secondary conflicts occurring due to maneuver execution delay uncertainty. The probability of a secondary conflict occurring did not seem to be strongly affected by time until loss of separation, and in the horizontal plane was only slightly affected by initial turn-out angle. This implies that risk mitigation strategies that include fixed start points for resolution maneuvers or scanning a range of airspace instead of a single trajectory might be required to remove the risk of secondary conflicts.

{'paper_id': '10124696', 'title': 'Effect of conflict resolution maneuver execution delay on losses of separation', 's2_url': 'https://api.semanticscholar.org/CorpusID:10124696', 'source': 'S2ORC'}