1. A computer-implemented method for semantic segmentation of an image, comprising: determining unary energy of each pixel in an image using a first subnetwork; determining pairwise energy of at least some pairs of pixels of the image using a second subnetwork; determining, using a third subnetwork, an inference on a Gaussian random field (GRF) minimizing an energy function including a combination of the unary energy and the pairwise energy to produce a GRF inference defining probabilities of semantic labels for each pixel in the image; and converting the image into a semantically segmented image by assigning to a pixel in the semantically segmented image a semantic label having the highest probability for a corresponding pixel in the image among the probabilities determined by the third subnetwork, wherein the first subnetwork, the second subnetwork, and the third subnetwork are parts of a neural network, and wherein steps of the method are performed by a processor.