1. A computer-readable storage medium having instructions stored thereon for segmenting an image that includes a plurality of pixels, which, when executed by a processor of a computing device cause the computing device to perform actions comprising: receiving an ordered set of tokens that references a first region of the image; generating an image map that represents a correspondence between each of a plurality of image features and a corresponding portion of the plurality of pixels; generating a set of token data elements, wherein each of the token data elements represents semantic features of a corresponding token of the set of tokens; iteratively updating a segmentation map that represents whether each of the plurality of pixels is included in the first region of the image, wherein each of a plurality of iterative updates of the segmentation map is based on a previous version of the segmentation map and a combination of the image map and one of the token data elements that is based on an order of the set of tokens; and generating a segmented image based on the image and the segmentation map.