Self-Evolutionary Optimization for Pareto Front Learning
  Multi-task learning (MTL), which aims to improve performance by learning
multiple tasks simultaneously, inherently presents an optimization challenge
due to multiple objectives. Hence, multi-objective optimization (MOO)
approaches have been proposed for multitasking problems. Recent MOO methods
approximate multiple optimal solutions (Pareto front) with a single unified
model, which is collectively referred to as Pareto front learning (PFL). In
this paper, we show that PFL can be re-formulated into another MOO problem with
multiple objectives, each of which corresponds to different preference weights
for the tasks. We leverage an evolutionary algorithm (EA) to propose a method
for PFL called self-evolutionary optimization (SEO) by directly maximizing the
hypervolume. By using SEO, the neural network learns to approximate the Pareto
front conditioned on multiple hyper-parameters that drastically affect the
hypervolume. Then, by generating a population of approximations simply by
inferencing the network, the hyper-parameters of the network can be optimized
by EA. Utilizing SEO for PFL, we also introduce self-evolutionary Pareto
networks (SEPNet), enabling the unified model to approximate the entire Pareto
front set that maximizes the hypervolume. Extensive experimental results
confirm that SEPNet can find a better Pareto front than the current
state-of-the-art methods while minimizing the increase in model size and
training cost.
