In this paper, we propose an algorithm for robustly fusing digital surface models (DSMs) with different ground sampling distances and confidences, using explicit surface priors to obtain locally smooth surface models. Robust fusion of the DSMs is achieved by minimizing the L1-distance of each pixel of the solution to each input DSM. This approach is similar to a pixel-wise median, and most outliers are discarded. We further incorporate local planarity assumption as an additional constraint to the optimization problem, thus reducing the noise compared with pixel-wise approaches. The optimization is also inherently able to include weights for the input data, therefore allowing to easily integrate invalid areas, fuse multiresolution DSMs, and to weight the input data. The complete optimization problem is constructed as a variational optimization problem with a convex energy functional, such that the solution is guaranteed to converge toward the global energy minimum. An efficient solver is presented to solve the optimization in reasonable time, e.g., running in real time on standard computer vision camera images. The accuracy of the algorithms and the quality of the resulting fused surface models are evaluated using synthetic data sets and spaceborne data sets from different optical satellite sensors.