One of the major drawbacks of noisy observations is that a single observation is unreliable as an estimate of the truth. The qNEHVI algorithm deals with that by integrating out the uncertainty of the GP model when calculating the expected HV improvement. However, if the ultimate goal is to identify the Pareto front correctly, a single observation may not be sufficient. We compared the performance of qNEHVI, qNParEGO, TSEMO, LHS Space Filling and a single-run-based MO-E-EQI algorithm. Even though all five algorithms are implemented for a single run only, unlike qNEHVI, qNParEGO and TSEMO, MO-E-EQI allows repeated observations. This means that an algorithm can choose to return to an existing input location and request the model to be run again. In that case, the model observations will be combined into one observation . Additionally, we compared the performance of algorithms based on multiple model runs. This means that for every input, the simulator is run multiple times and the sample mean is adopted as an 'observation' when a GP is fitted. That also means that, if the algorithm desires to repeat an observation (i.e. run the simulator at the same input again), the simulator will again be run multiple times and the sample mean will be calculated as the secondary observation. The two observations will then be combined according to . The important feature of the MO-E-EQI algorithm is the estimation of the future noise incorporated in the expected improvement calculation. This requires us to predict the level of noise at an unobserved input location. We remind our reader that a direct linear/loglinear relationship between the noise standard deviation and the expected value of the response was assumed. Under that assumption, the future noise on each step of the algorithm was calculated using the current fitted GPs' means as a guide for our mean response and the noise was calculated based on the relationship with a specific noise magnitude. This means replacing f (x) with m(x) in Equations 10 and 11.