In this paper we study the problem of image representation learning without human annotation. By following the principles of self-supervision, we build a convolutional neural network (CNN) that can be trained to solve Jigsaw puzzles as a pretext task, which requires no manual labeling, and then later repurposed to solve object classification and detection. To maintain the compatibility across tasks we introduce the context-free network (CFN), a siamese-ennead CNN. The CFN takes image tiles as input and explicitly limits the receptive field (or context) of its early processing units to one tile at a time. We show that the CFN includes fewer parameters than AlexNet while preserving the same semantic learning capabilities. By training the CFN to solve Jigsaw puzzles, we learn both a feature mapping of object parts as well as their correct spatial arrangement. Our experimental evaluations show that the learned features capture semantically relevant content. Our proposed method for learning visual representations outperforms state of the art methods in several transfer learning benchmarks.
@cite_13: 
@cite_7: In the square jigsaw puzzle problem one is required to reconstruct the complete image from a set of non-overlapping, unordered, square puzzle parts. Here we propose a fully automatic solver for this problem, where unlike some previous work, it assumes no clues regarding parts' location and requires no prior knowledge about the original image or its simplified (e.g., lower resolution) versions. To do so, we introduce a greedy solver which combines both informed piece placement and rearrangement of puzzle segments to find the final solution. Among our other contributions are new compatibility metrics which better predict the chances of two given parts to be neighbors, and a novel estimation measure which evaluates the quality of puzzle solutions without the need for ground-truth information. Incorporating these contributions, our approach facilitates solutions that surpass state-of-the-art solvers on puzzles of size larger than ever attempted before.
@cite_20: This paper describes the development of a procedure that enables a digital computer to solve apictorial'' jigsaw puzzles, i.e., puzzles in which all pieces are uniformly gray and the only available information is the shape of the pieces. The problem was selected because it provided an excellent vehicle to develop computer techniques for manipulation of arbitrary geometric patterns, for pattern identification, and for game solving. The kinds of puzzles and their properties are discussed in detail. Methods are described for characterizing and classifying piece contours, for selecting and ordering pieces that are most likely'' to mate with a given piece, for determining likelihood of fit, for overcoming ambiguities, and for evaluation of the progressive puzzle assembly. An illustration of an actual computer solution of a puzzle is given.
There is also a sizeable literature on solving Jigsaw puzzles computationally (see, for example, @cite_13 @cite_20 @cite_7 ). However, these methods rely on the shape of the tiles or on texture especially in the proximity of the borders of the tiles. These are cues that we avoid when training the Jigsaw puzzle solver, as they do not carry useful information when learning a part detector.