# embeddings_queue

created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a401a-6db4-11ee-8b7d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-5", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "that deeper layers may be more domain-speci\ufb01c and therefore potentially less transferrable than shallower layers.\nThe second contribution of this paper is FV-CNN\n(Sect. 3), a pooling method that overcomes these drawbacks . The idea is to regard the convolutional layers of a\nCNN as a \ufb01lter bank and build an orderless representation\nusing FV as a pooling mechanism, as is commonly done\nin the bag-of-words approaches. Although the suggested\nchange is simple, the approach is remarkably \ufb02exible and\neffective. First, pooling is orderless and multi-scale, hence\nsuitable for textures. Second, any image size can be processed by convolutional layers, avoiding costly resizing operations. Third, convolutional \ufb01lters, pooled by FV-CNN,\nare shown to transfer more easily than fully-connected ones\neven without \ufb01ne-tuning. While other authors [15, 18] have\nrecently proposed alternative pooling strategies for CNNs,\nwe show that our method is more natural, faster and often\nsigni\ufb01cantly more accurate.\nThethird contribution of the paper is a thorough evaluation of these descriptors on a variety of benchmarks, from\ntextures to objects (Sect. 4). In textures, we evaluate material and describable attributes recognition and segmentation"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a401b-6db4-11ee-8695-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-6", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "textures to objects (Sect. 4). In textures, we evaluate material and describable attributes recognition and segmentation\non new datasets derived from OpenSurfaces (Sect. 2). When\nused with linear SVMs, FV-CNN improves the state of the\nart on texture recognition by a signi\ufb01cant margin. Like textures, scenes are also weakly structured and a bag-of-words\nrepresentation is effective. FV-CNN obtains 81.1% accuracy on the MIT indoor scenes dataset [34], signi\ufb01cantly\noutperforming the current state-of-the-art of 70.8% [47].\nWhat is remarkable is that, where [47] \ufb01nds that CNNs\ntrained on scene recognition data perform better than CNNs\ntrained on an object domain (ImageNet), when used in FV-CNN not only is there an overall performance improvement, but the domain-speci\ufb01c advantage is entirely removed\n(Tab. 3). This indicates that FV-CNN are in fact better at domain transfer. Our method also matches the previous best in\nPASCAL VOC 2007 classi\ufb01cation dataset providing measurable boost over CNNs and closely approaches competitor methods on CUB 2010-2011 datasets when ground-truth\nobject bounding boxes are given.\nFV-CNN can be used for describing regions by simply"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a401c-6db4-11ee-8d7d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-7", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "object bounding boxes are given.\nFV-CNN can be used for describing regions by simply\npooling across pixels within the region. Combined with\na low-level segmentation algorithm this suf\ufb01ces to localize textures within images. This approach is similar to a\nrecently proposed method called \u201cR-CNN\u201d for localizing\nobjects [14]. However, in contrast to it we do not need repeated evaluations of the CNN since the convolutional features can be computed just once for the entire image and\npooled differently. This makes FV-CNN not only faster,\nbut also as experiments suggest, much more accurate at\ntexture localization. We achieve state of the art results on\nthe MSRC segmentation dataset using a simple scheme of\nclassifying \u201csuperpixels\u201d obtaining an accuracy of 87.0%\n(previous best 86.5%). The corresponding R-CNN obtains\n57.7% accuracy. Segmentation results are promising in the\nOpenSurfaces dataset [4] as well.\nFinally, we analyze the utility of different network layers\nand architectures as \ufb01lter banks, concluding that: SIFT is\ncompetitive only with the \ufb01rst few layers of a CNN (Fig. 4)"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a401d-6db4-11ee-86a6-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-8", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "and architectures as \ufb01lter banks, concluding that: SIFT is\ncompetitive only with the \ufb01rst few layers of a CNN (Fig. 4)\nand that signi\ufb01cant improvement to the underlying CNN architecture, such as the ones achieved by the very deep models of Simonyan and Zisserman [39], directly translate into\nmuch better \ufb01lter banks for texture recognition.\n2. Texture recognition in clutter\nA contribution of this work is the analysis of materials\nand texture attributes in realistic imaging conditions. Earlier datasets such as KTH-TIPS were collected in controlled\nconditions, which makes their applicability to natural images unclear. More recent datasets such as FMD and DTD\nremove this limitation by building on images downloaded\nfrom the Internet, dubbed images \u201cin the wild\u201d. However,\nin these datasets texture always \ufb01ll the \ufb01eld of view of the\ncamera. In this paper we remove this limitation by experimenting for the \ufb01rst time with a large dataset of textures\ncollected in the wild and in cluttered conditions.\nIn particular, we build on the Open Surfaces (OS) dataset\nthat was recently introduced by Bell et al. [4] in computer\ngraphics. OS comprises 25,357 images, each containing a\nnumber of high-quality texture/material segments. Many of"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a401e-6db4-11ee-baac-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-9", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "graphics. OS comprises 25,357 images, each containing a\nnumber of high-quality texture/material segments. Many of\nthese segments are annotated with additional attributes such\nas the material name, the viewpoint, the BRDF, and the object class. Not all segments have a complete set of annotations; the experiments in this paper focus on the 58,928 that\ncontain material names. Since material classes are highly\nunbalanced, only the materials that contain at least 400 examples are considered. This result in 53,915 annotated\nmaterial segments in 10,422 images spanning 22 different\nclasses.1Images are split evenly into training, validation,\nand test subsets with 3,474 images each. Segment sizes are\nhighly variable, with half of them being relatively small,\nwith an area smaller than 64\u000264pixels. While the lack\nof exhaustive annotations makes it impossible to de\ufb01ne a\ncomplete background class, several less common materials\n(including for example segments that annotators could not\nassign to a material) are merged in an \u201cother\u201d class that acts\nas pseudo-background.\nIn order to study perceptual properties as well as materials, we augment the OS dataset with some of the 47\nattributes from the DTD [7]. Since the OS segments do\nnot trigger with suf\ufb01cient frequency all the 47 attributes,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a401f-6db4-11ee-a894-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-10", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "attributes from the DTD [7]. Since the OS segments do\nnot trigger with suf\ufb01cient frequency all the 47 attributes,\nthe evaluation is restricted to eleven of them for which it\nwas possible to identify at least 100 matching segments.2\nThe attributes were manually labelled in the 53,915 segments retained for materials. We refer to this data as\nOSA. The complete list of images, segments, labels, and\nsplits are available at http://www.robots.ox.ac.\nuk/\u02dcvgg/data/wildtex/ .\n3. Method\nThis section describes the methodological contributions\nof this paper: region description and segmentation.\n3.1. Region description\nThis section introduces a number of visual descriptors\nsuitable to model the appearance of image regions. Texture\nis traditionally described by orderless pooling of \ufb01lter bank\nresponses as, unlike in objects, the overall shape information is usually unimportant. However, small under-sampled\ntextures may bene\ufb01t if recognized in the context of an object. Thus, the primacy of orderless pooling may not always\nhold in the recognition of textures in natural conditions.\nIn order to explore the interplay between shape and orderless pooling, we evaluate two corresponding region descriptors: FC-CNN for shape and FV-CNN for texture."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4020-6db4-11ee-a5d9-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-11", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "In order to explore the interplay between shape and orderless pooling, we evaluate two corresponding region descriptors: FC-CNN for shape and FV-CNN for texture.\nBoth descriptors are based on the same CNN features [23]\nobtained from an off-the-shelf CNN pre-trained on the ImageNet ILSVRC 2012 data as suggested in [6, 21, 35]. Since\nthe underlying CNN is the same, it is meaningful to compare FC- and FV-CNN directly.\n1The classes and corresponding number of example segments are:\nbrick (610), cardboard (423), carpet/rug (1,975), ceramic (1,643), concrete (567), fabric/cloth (7,484), food (1,461), glass (4,571), granite/marble (1,596), hair (443), other (2,035), laminate (510), leather (957),\nmetal (4,941), painted (7,870), paper/tissue (1,226), plastic/clear (586),\nplastic/opaque (1,800), stone (417), tile (3,085), wallpaper (483), wood\n(9,232).\n2These are: banded, blotchy, chequered, \ufb02ecked, gauzy, grid, marbled,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4021-6db4-11ee-a08e-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-12", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "(9,232).\n2These are: banded, blotchy, chequered, \ufb02ecked, gauzy, grid, marbled,\npaisley, pleated, strati\ufb01ed, wrinkled.Object descriptor: FC-CNN. The FC-CNN descriptor is\nobtained by extracting as features the output of the penultimate Fully-Connected (FC) layer of a CNN, including\nthe non-linear gating function, applied to the input image.\nThis can be considered an object descriptor because the\nfully connected layers allow FC-CNN to capture the overall\nshape of the object contained in the region. FC-CNN is applied to an image region R(which may be the whole image)\nby warping the bounding box enclosing R(plus a 10% border) to a square of a \ufb01xed size matching the default CNN\ninput geometry, obtaining the same R-CNN descriptor introduced by Girshick et al. [14] as a state-of-the-art object\ndetector in the PASCAL VOC [10] data.\nTexture descriptor: FV-CNN. The FV-CNN descriptor\nis inspired by the state-of-the-art texture descriptors of [7]\nbased on the Fisher Vector (FV). Differently from FC-CNN,\nFV pools local features densely within the described regions"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4022-6db4-11ee-ade0-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-13", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "based on the Fisher Vector (FV). Differently from FC-CNN,\nFV pools local features densely within the described regions\nremoving global spatial information , and is therefore more\napt at describing textures than objects. Here FV is computed\non the output of a single (last) convolutional layer of the\nCNN, but we compared features from other layers as well\n(Sect 4.4). By avoiding the computation of the fully connected layers, the input image does not need to be rescaled\nto a speci\ufb01c size; in fact, the dense convolutional features\nare extracted at multiple scales and pooled into a single FV\njust like for SIFT. The pooled convolutional features are extracted immediately after the last linear \ufb01ltering operator\nand are not otherwise normalised.\nThe FV-CNN descriptor is related to the one proposed\nby Gong et al. [15]. There VLAD pooling, which is similar to FV , is applied to FC-CNN-like descriptors extracted\nfrom densely sampled image windows. A key difference of\nFV-CNN is that dense features are extracted from the convolutional rather than fully-connected layers. This is more\nnatural, signi\ufb01cantly more ef\ufb01cient (as it does not require"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4023-6db4-11ee-baec-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-14", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "natural, signi\ufb01cantly more ef\ufb01cient (as it does not require\nrecomputing the network for each extracted local descriptor) and, as shown in Sect. 4, more accurate.\n3.2. Region segmentation\nThis section discusses our method to automatically partition an image into a number of recognisable regions. Inspired by Cimpoi et al. [7] that successfully ported object\ndescription methods to texture descriptors, here we propose\na segmentation technique inspired by object detection. An\nincreasingly popular method for object detection, followed\nfor example by R-CNN [14], is to \ufb01rst propose a number of\ncandidate object regions using low-level image cues, and\nthen verifying a shortlist of such regions using a powerful classi\ufb01er. Applied to textures, this requires a low-level\nmechanism to generate textured region proposals, followed\nby a region classi\ufb01er. A key advantage of this approach\nis that it allows applying object- (FC-CNN) and texturelike (FV-CNN) descriptors alike. After proposal classi\ufb01cation, each pixel can be assigned more than one label; this is\nsolved with simple voting schemes, also inspired by object\ndetections methods.\nThe paper explores two such region generation methods:"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4024-6db4-11ee-8bc1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-15", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "solved with simple voting schemes, also inspired by object\ndetections methods.\nThe paper explores two such region generation methods:\nthe crisp regions of [19] and the multi-scale combinatorial\ngrouping of [3]. In both cases, region proposals are generated using low-level image cues, such as colour or texture\nconsistency, as speci\ufb01ed by the original methods. It would\nof course be possible to incorporate FC-CNN and FV-CNN\namong these energy terms to potentially strengthen the region generation mechanism itself. However, this contradicts partially the logic of the scheme, which breaks down\nthe problem into cheaply generating tentative segmentations and then verifying them using a more powerful (and\nlikely expensive) model. Furthermore, and more importantly, these cues focus on separating texture instances , as\npresented in each particular image, whereas FC-CNN and\nFV-CNN are meant to identify a texture class. It is reasonable to expect instance-speci\ufb01c cues (say the colour of a\npainted wall) to be better for segmentation.\n4. Results\nThis section evaluates the proposed region recognition\nmethods for classifying and segmenting materials, describable texture properties, and higher-level object categories.\nSect. 4.1 evaluates the classi\ufb01cation task by assessing how\nwell regions can be classi\ufb01ed given that their true extent is"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4025-6db4-11ee-8b3e-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-16", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "Sect. 4.1 evaluates the classi\ufb01cation task by assessing how\nwell regions can be classi\ufb01ed given that their true extent is\nknown and Sect. 4.3 evaluates both classi\ufb01cation and segmentation . The rest of the section introduces the evaluation\nbenchmarks and technical details of the representations.\nDatasets. The evaluation considers three texture recognition benchmarks other than OS (Sect. 2). The \ufb01rst one is the\nFlickr Material Dataset (FMD) [37], a recent benchmark\ncontaining 10 material classes. The second one is the Describable Texture Datasets (DTD) [7], which contains texture images jointly annotated with 47 describable attributes\ndrawn from the psychological literature. Both FMD and\nDTD contain images \u201cin the wild\u201d, i.e. collected in uncontrolled conditions. However, differently from OS, these images are uncluttered. The third texture dataset is KTH-TIPS2b[5, 17], containing a number of example images for each\nof four samples of 11 material categories. For each material, images of one sample are used for training and the\nremaining for testing.\nObject categorisation is evaluated in the PASCAL VOC\n2007 [10] dataset, containing 20 object categories, any\ncombination of which may be associated to any of the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4026-6db4-11ee-995d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-17", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "Object categorisation is evaluated in the PASCAL VOC\n2007 [10] dataset, containing 20 object categories, any\ncombination of which may be associated to any of the\nbenchmark images. Scene categorisation uses the MIT Indoor [34] dataset, containing 67 indoor scene classes. Finegrained categorisation uses the Caltech/UCSD Bird dataset\n(CUB) [45], containing images of 200 bird species.\nNote that some of these datasets come with ground truth\nregion/object localisation. The +R suf\ufb01x will be appendedto a dataset to indicate that this information is used both at\ntraining and testing time. For example, OS means that segmentation is performed automatically at test time, whereas\nOS+R means that ground-truth segmentations are used.\nEvaluation measures. For each dataset the corresponding standard evaluator protocols and accuracy measures are\nused. In particular, for FMD, DTD, MIT Indoor, CUB,\nand OS+R, evaluation uses average classi\ufb01cation accuracy,\nper-segment/image and normalized for each class. When\nevaluating the quality of a segmentation algorithm, however, one must account for the fact that in most datasets,\nand in OS and MSRC in particular, not all pixels are labelled. In this case, accuracy is measured per-pixel rather"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4027-6db4-11ee-964c-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-18", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "and in OS and MSRC in particular, not all pixels are labelled. In this case, accuracy is measured per-pixel rather\nthan per-segment, ignoring all pixels that are unlabelled in\nthe ground truth. For MSRC, furthermore, accuracy is normalised across all pixels regardless of their category. For\nOSA, since some segments may have more than one label,\nwe are reporting mAP, following the standard procedure for\nmulti-label datasets. Finally, PASCAL VOC 2007 classi\ufb01cation uses mean average precision (mAP), computed using\nthe TRECVID 11-point interpolation [10].3\nDescriptor details. FC-CNN and FV-CNN build on the\npre-trained VGG-M [6] model as this performs better than\nother popular models such as [21] while having a similar computational cost. This network results in 4096dimensional FC-CNN features and 512-dimensional local\nfeatures for FV-CNN computation. The latter are pooled\ninto a FV representation with 64 Gaussian components,\nresulting in 65K-dimensional descriptors. While the FVCNN dimensionality is much higher than the 4K dimensions of FC-CNN, the FV is known to be highly redundant and can be typically compressed by one order of magnitude without appreciable reduction in the classi\ufb01cation"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4028-6db4-11ee-a277-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-19", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "performance [31], so the effective dimensionality of FCand FV-CNN is likely comparable. We veri\ufb01ed that by\nPCA-reducing FV to 4096 dimensions and observing only\na marginal reduction in classi\ufb01cation performance in the\nPASCAL VOC object recognition task described below. In\naddition to VGG-M, the recent state-of-the art VGG-VD\n(very deep with 19 layers) model of Simonyan and Zisserman [39] is also evaluated.\nDue to the similarity between FV-CNN and the dense\nSIFT FV descriptors used for texture recognition in [7],\nthe latter is evaluated as well. Since SIFT descriptors\nare smaller (128-dimensional) than the convolutional ones\n(512-dimensional), a larger number of Gaussian components (256) are used to obtain FV descriptors with a comparable dimensionality. The SIFT descriptors support is\n32\u000232pixels at the base scale.\nIn order to make results comparable to [7], we use\nthe same settings whenever possible. FV-CNN and DSIFT compute features after rescaling the image by factors\n3The de\ufb01nition of AP was changed in later versions of the benchmark."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4029-6db4-11ee-95fe-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-20", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "3The de\ufb01nition of AP was changed in later versions of the benchmark.\ndataset meas.IFVVGG-M VGG-VD FV-SIFTSoA(%) FC FV FC+FV FC FV FC+FV FC+FV-VD\n(a)KTH-T2b acc 70.8\u00062.771\u00062.373.3\u00064.773.9\u00064.975.4\u00061.581.8\u00062.581.1\u00062.4 81.5\u00062.0 76.0\u00062.9[40]\nFMD acc 59.8\u00061.670.3\u00061.873.5\u00062.076.6\u00061.977.4\u00061.879.8\u00061.882.4\u00061.5 82.2\u00061.4 57.7\u00061.7[33, 37]\nOS+R acc 39.8 41.3 52.5 54.9 43.4 59.5 60.9 58.7 \u2013\n(b)DTD acc 58.6\u00061.258.8\u00060.866.8\u00061.669.8\u00061.162.9\u00060.872.3\u00061.074.7\u00061.0 75.5\u00060.8 \u2013"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a402a-6db4-11ee-84c9-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-21", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "OSA+R mAP 56.5 54.3 65.2 67.9 49.7 67.2 67.9 68.2 \u2013\n(d)MSRC+R acc 85.7 85.0 95.4 96.9 79.4 97.7 98.8 99.1 \u2013\nMSRC+R msrc-acc 92.0 84.0 97.6 98.1 82.0 99.2 99.6 99.5 \u2013\nVOC07 mAP11 59.9 76.8 76.4 79.5 81.7 84.9 85.1 84.5 85.2 [44]\nMIT Indoor acc 54.9 62.5 74.2 74.4 67.6 81.0 80.3 80.0 70.8 [47]\nCUB acc 17.5 46.1 49.9 54.9 54.6 66.7 67.3 65.4 73.9 (62.8\u0003) [46]\nCUB+R acc 27.7 56.5 65.5 68.1 62.8 73.0 74.9 73.6 76.37 [46]\nTable 1. Evaluation of texture descriptors. The table compares FC-CNN, FV-CNN on two networks trained on ImageNet \u2013 VGG-M and"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a402b-6db4-11ee-835b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-22", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "Table 1. Evaluation of texture descriptors. The table compares FC-CNN, FV-CNN on two networks trained on ImageNet \u2013 VGG-M and\nVGG-VD, and IFV on dense SIFT. We evaluated these descriptors on (a) material datasets (FMD, KTH-T2b, OS+R), (b) texture attributes\n(DTD, OSA+R) and (c) general categorisation datasets (MSRC+R,VOC07,MIT Indoor) and \ufb01ne grained categorisation (CUB, CUB+R).\nFor this experiment the region support is assumed to be known (and equal to the entire image for all the datasets except OS+R and MSRC+R\nand for CUB+R, where it is set to the bounding box of a bird). (\u0003) using a model without parts. Best results are marked in bold.\nVGG-M VGG-VD\ndataset measure (%) FC-CNN FV-CNN FV+FC-CNN FC-CNN FV-CNN FC+FV-CNN SoA\nOS pp-acc 36.3 48.7 (46.9) 50.5 38.8 55.4 (55.7) 55.2 \u2013"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a402c-6db4-11ee-93b1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-23", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "OS pp-acc 36.3 48.7 (46.9) 50.5 38.8 55.4 (55.7) 55.2 \u2013\nMSRC msrc-acc 56.1 82.3 75.5 57.7 87.0 80.4 86.5 [24]\nTable 2. Segmentation and recognition using crisp region proposals of materials (OS) and things & stuff (MSRC). Per-pixel accuracies are\nreported, using the MSRC variant (see text) for the MSRC dataset. Results using MCG proposals [3] are reported in brackets for FV-CNN.\n2s; s=\u00003;\u00002:5; : : :1:5(but, for ef\ufb01ciency, discarding\nscales that would make the image larger than 10242pixels). Before pooling descriptors with a FV , these are usually\nde-correlated by using PCA. Here PCA is applied to SIFT,\nadditionally reducing its dimension to 80, as this was empirically shown to improve the overall recognition performance. However, PCA is not applied to the convolutional\nfeatures in FV-CNN as in this case results were worse.\nLearning details. The region descriptors (FC-CNN, FVCNN, and D-SIFT) are classi\ufb01ed using 1-vs-rest Support"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a402d-6db4-11ee-b620-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-24", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "Learning details. The region descriptors (FC-CNN, FVCNN, and D-SIFT) are classi\ufb01ed using 1-vs-rest Support\nVector Machine (SVM) classi\ufb01ers. Prior to learning, descriptors are L2normalised and the learning constant set\ntoC= 1. This is motivated by the fact that, after data\nnormalisation, the exact choice of Chas a negligible effect\non performance. Furthermore, the accuracy of the 1-vs-rest\nclassi\ufb01cation scheme is improved by recalibrating the SVM\nscores after training, by scaling the SVM weight vector and\nbias such that the median scores of the negative and positive\ntraining samples for each class are mapped respectively to\nthe values\u00001and1.\n4.1. Region recognition: textures\nThis and the following section evaluate region recognition assuming that the ground-truth region Ris known (Table 1), for example because it \ufb01lls the entire image. This\nsection focuses on textures (materials and perceptual attributes), while the next on objects and scenes.\nTexture recognition without clutter. This experimentevaluates the performance of FC-CNN, FV-CNN, D-SIFT,\nand their combinations in standard texture recognition"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a402e-6db4-11ee-b9d1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-25", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "Texture recognition without clutter. This experimentevaluates the performance of FC-CNN, FV-CNN, D-SIFT,\nand their combinations in standard texture recognition\nbenchmarks such as FMD, KTH-TIPS-2, and DTD. FCCNN is roughly equivalent to the DeCAF method used\nin [7] for this data as regions \ufb01ll images; however, while the\nperformance of our FC-CNN is similar in KTH ( \u001870%),\nit is substantially better in FMD ( 60:7%!70:4%accuracy) and DTD ( 54:8%!58:7%). This likely is caused by\nthe improved underlying CNN, an advantage which is more\nobvious in FMD and DTD that are closer to object recognition than KTH. FV-CNN performs within \u00062%in FMD and\nKTH but substantially better for DTD ( 58:7%!66:6%).\nD-SIFT is comparable in performance to FC-CNN in DTD\nand KTH, but substantially worse ( 70:4%!59:2%) in\nFMD. Our conclusion is that, even when textures \ufb01ll the\ninput image as in these benchmarks, orderless pooling in\nFV-CNN and D-SIFT can be either the same or substantially better than the pooling operated in the fully-connected"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a402f-6db4-11ee-b74d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-26", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "input image as in these benchmarks, orderless pooling in\nFV-CNN and D-SIFT can be either the same or substantially better than the pooling operated in the fully-connected\nlayers by FC-CNN.\nCombining FC- and FV-CNN improves performance in\nall datasets by 1\u00003%. While this combination is already signi\ufb01cantly above the state-of-the-art in DTD and\nFMD ( +2:6%=11:2%), the method of [7] still outperforms\nthese descriptors in KTH. However, replacing VGG-M with\nVGG-VD signi\ufb01cantly improves the performance in all\ncases \u2013 a testament to the power of deep features. In particular, the best method FC+FV-CNN-VD, improves the state\nof the art by at least 6%in all datasets. Interestingly, this is\nobtained by using a single low-level feature type as FC- and\nFV-CNN build on the same convolutional features. Adding\nD-SIFT results in at most \u00181%improvement, and in some\ncases it slightly degrades performance.\nTexture recognition in clutter. The advantage of FV-CNN\nover FC-CNN is much larger when textures do not \ufb01ll the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4030-6db4-11ee-b816-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-27", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "cases it slightly degrades performance.\nTexture recognition in clutter. The advantage of FV-CNN\nover FC-CNN is much larger when textures do not \ufb01ll the\nimage but are extracted from clutter. In OS+R (Sect. 2),\nmaterial recognition accuracy starts at about 46% for both\nFC-CNN and D-SIFT; however, FV-CNN improves this by\nmore than 11% (46:5%!58:1%). The combination of FCand FV-CNN improves results further by \u00182%, but adding\nSIFT deteriorates performance. With the very deep CNN\nconclusions are similar; however, switching to VGG-VD\nbarely affects the FC-CNN performance ( 46:5!48:0%),\nbut strongly affects the one of FV-CNN ( 58:1%!65:1%).\nThis con\ufb01rms that FC-CNN, while excellent in object detection, is not a very good descriptor for classifying textured\nregions. Results in OSA+R for texture attribute recognition\n(Sect. 2) and in MSRC+R for semantic segmentation are\nanalogous; it is worth noting that, when ground-truth segments are used in this experiment, the best model achieves\na nearly perfect 99:7%classi\ufb01cation rate in MSRC."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4031-6db4-11ee-979f-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-28", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "a nearly perfect 99:7%classi\ufb01cation rate in MSRC.\n4.2. Region recognition: objects and scenes\nThis section shows that the FV-CNN descriptor, despite\nits orderless nature that make it an excellent texture descriptor, excels at object and scene recognition as well. In the remainder of the section, and unless otherwise noted, region\ndescriptors are applied to images as a whole by considering\nthese single regions.\nFV-CNN vs FC-CNN. As seen in Table 1, in PASCAL\nVOC and MIT Indoor the FC-CNN descriptor performs\nvery well but in line with previous results for this class of\nmethods [6]. FV-CNN performs similarly to FC-CNN in\nPASCAL VOC, but substantially better ( +5% for VGG-M\nand +13% for VGG-VD) in MIT Indoor. As further discussed below, this is an example of the ability of FV-CNN\nto transfer between domains better than FC-CNN. The gap\nbetween FC-CNN and FV-CNN is the highest for the very\ndeep VGG-VD models (68.1% !81.1%), a trend also exhibited by other datasets as seen in Tab. 1. In the CUB\ndataset, FV-CNN signi\ufb01cantly outperforms FC-CNN both"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4032-6db4-11ee-a728-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-29", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "dataset, FV-CNN signi\ufb01cantly outperforms FC-CNN both\nwhether the descriptor is computed from the whole image\n(CUB) or from the bird bounding box (CUB+R). In the latter case, the difference is very large ( +10\u000014%).\nComparison with alternative pooling methods. FV-CNN\nis related to the method of [15], which uses a similar\nunderlying CNN and VLAD instead of FV for pooling.\nNotably, however, FV-CNN results on MIT Indoor are\nmarkedly better than theirs for both VGG-M and VGGVD ( 68:8%!73:5%=81:1%resp.) and marginally better (68:8%!69:1%) when the same CAFFE CNN is used\n(Tab. 3). The key difference is that FV-CNN pools convolu-Accuracy (%)\nCNN FC-CNN FV-CNN FC+FV-CNN\nPLACES 65.0 67.6 73.1\nCAFFE 58.6 69.7 71.6\nVGG-M 62.5 74.2 74.4\nVGG-VD 67.6 81.0 80.3\nTable 3. Accuracy of various CNNs on the MIT indoor dataset."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4033-6db4-11ee-b6c0-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-30", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "VGG-VD 67.6 81.0 80.3\nTable 3. Accuracy of various CNNs on the MIT indoor dataset.\ntional features, whereas [15] pools fully-connected descriptors extracted from square image patches. Thus, even without spatial information as used by [15], FV-CNN is not only\nsubstantially faster, but at least as accurate. Using the same\nsettings, that is, the same net and the same three scales, our\napproach results in an 8.5 \u0002speedup.\nComparison with the state-of-the-art. The best result obtained in PASCAL VOC is comparable to the current stateof-the-art set by the deep learning method of [44] ( 85:2%!\n85:0%), but using a much more straightforward pipeline. In\nMIT Places our best performance is also substantially superior ( +10% ) to the current state-of-the-art using deep convolutional networks learned on the MIT Place dataset [47]\n(see also below). In the CUB dataset, our best performance\nis a little short (\u00183%) of the state-of-the-art results of [46].\nHowever, [46] uses a category-speci\ufb01c part detector and\ncorresponding part descriptor as well as a CNN \ufb01ne-tuned"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4034-6db4-11ee-806e-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-31", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "However, [46] uses a category-speci\ufb01c part detector and\ncorresponding part descriptor as well as a CNN \ufb01ne-tuned\non the CUB data; by contrast, FV-CNN and FC-CNN are\nused here as global image descriptors which, furthermore,\nare the same for all the datasets considered . Compared to\nthe results of [46] without part-based descriptors (but still\nusing a part-based object detector), our global image descriptors perform substantially better ( 62:1%!69:1%).\nWe conclude that FV-CNN is a very strong descriptor.\nResults are usually as good, and often signi\ufb01cantly better,\nthan FC-CNN. In most applications, furthermore, FV-CNN\nis many times faster as it does not require evaluating the\nCNN for each target image region. Finally, FC- and FVCNN can be combined outperforming the state-of-the-art in\nmany benchmarks.\nDomain transfer. So far, the same underlying CNN features, trained on ImageNet\u2019s ILSVCR, were used for all\ndatasets. Here we investigate the effect of using domainspeci\ufb01c features. To do so, we consider the PLACES [47],"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4035-6db4-11ee-99f8-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-32", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "datasets. Here we investigate the effect of using domainspeci\ufb01c features. To do so, we consider the PLACES [47],\ntrained to recognize places on a dataset of about 2.5 million labeled images. [47] showed that, applied to the task of\nscene recognition in MIT Indoor, these features outperform\nsimilar ones trained on ILSVCR (denoted CAFFE [21] below) \u2013 a fact explained by the similarity of domains. Below,\nwe repeat this experiment using FC- and FV-CNN descriptors on top of VGG-M, VGG-VD, PLACES, and CAFFE.\nResults are shown in Table 3. The FC-CNN results are\nin line with those reported in [47] \u2013 in scene recognition\nwith FC-CNN the same CNN architecture performs better\nif trained on the Places dataset instead of the ImageNet data\n(58:6%!65:0%accuracy4). Nevertheless, stronger CNN\narchitectures such as VGG-M and VGG-VD can approach\nand outperform PLACES even if trained on ImageNet data\n(65:0%!63:0%=68:1%).\nHowever, when it comes to using the \ufb01lter banks with\nFV-CNN, conclusions are very different. First, FV-CNN"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4036-6db4-11ee-978e-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-33", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "However, when it comes to using the \ufb01lter banks with\nFV-CNN, conclusions are very different. First, FV-CNN\noutperforms FC-CNN in all cases, with substantial gains\nup to 20% in correspondence of a domain transfer from\nImageNet to MIT Indoor. Second, the advantage of using domain-speci\ufb01c CNNs disappears . In fact, the same\nCAFFE model that is 6.4% worse than PLACES with FCCNN, is actually 1.5% better when used in FV-CNN. The\nconclusion is that FV-CNN appears to be immune, or at\nleast substantially less sensitive, to domain shifts.\nOur tentative explanation of this surprising phenomenon\nis that the convolutional layers are less committed to a speci\ufb01c dataset than the fully ones. Hence, by using those,\nFV-CNN tends to be a more general than FC-CNN.\n4.3. Texture segmentation\nThe previous section considered the problem of region\nrecognition when the region support is known at test time.\nThis section studies the problem of recognising regions\nwhen their extent Risnotknown and also be estimated.\nThe \ufb01rst experiment (Tab. 2) investigates the simplest possible scheme: combining the region descriptors\nof Sect. 4.1 with a general-purpose image segmentation"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4037-6db4-11ee-814e-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-34", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "The \ufb01rst experiment (Tab. 2) investigates the simplest possible scheme: combining the region descriptors\nof Sect. 4.1 with a general-purpose image segmentation\nmethod, namely the crisp regions of [19]. Two datasets\nare evaluated: OS for material recognition and MSRC for\nthings & stuff. Compared to OS+R, classifying crisp regions results in a drop of about 5%points for all descriptors. As this dataset is fairly challenging with best achievable performance is 55:4%, this is a satisfactory result. But\nit also illustrates that there is ample space for future improvements. In MSRC, the best accuracy is 87:0%, just a\nhair above the best published result 86:5%[25]. Remarkably, these algorithms not use any dataset-speci\ufb01c training, nor CRF-regularised semantic inference: they simply\ngreedily classify regions as obtained from a general-purpose\nsegmentation algorithms. Qualitative segmentation results\n(sampled at random) are given in Fig. 2 and 3.\nUnlike crisp regions, the proposals of [3] are overlapping and a typical image contains thousands of them. We\npropose a simple scheme to combine prediction from multiple proposals. For each proposal we set its label to the\nhighest scoring class, and score to the highest score. We\nthen sort the proposals in the increasing order of their score"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4038-6db4-11ee-931a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-35", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "highest scoring class, and score to the highest score. We\nthen sort the proposals in the increasing order of their score\ndivided by their area and paste them one by one. This has\nthe effect of considering larger regions before smaller ones\nand more con\ufb01dent regions after less ones for regions of\nthe same area. Results using FV-CNN shown in Tab. 2 in\n4[47] report 68:3%for PLACES applied to MIT Indoor, a small difference explained by implementation details such as the fact that, for all the\nmethods, we do not perform data augmentation by jittering.\n40444852566064687276Performance (%)\nconv1\n[7 x  7]\n 96x256\nconv2\n[27 x  27]\n256x128\nconv3\n[75 x  75]\n512x 64\nconv4\n[107 x  107]\n512x 64\nconv5\n[139 x  139]\n512x 64(VGG\u2212M) Filterbank Analysis\n  \nVOC07 (mAP11)\nMIT Indoor (acc)\nFMD (acc)\nDTD (acc)\nSIFTFigure 4. CNN \ufb01lterbank analysis for VGG-M. Performance of\n\ufb01lter banks extracted from various layers network are shown on"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4039-6db4-11ee-a492-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-36", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "SIFTFigure 4. CNN \ufb01lterbank analysis for VGG-M. Performance of\n\ufb01lter banks extracted from various layers network are shown on\nvarious datasets. For each layer convf1; : : : ; 5gwe show, the size\nof the receptive \ufb01eld [ N\u0002N], and FB\u0002D, where FBis the size\nof \ufb01lter bank and Dis the dictionary size in the FV representation.\nThe performance using SIFT is shown in black plus (+) marks.\nbrackets (FC-CNN was too slow for our experiments). The\nresults are comparable to those using crisp regions, and we\nobtain 55:7%accuracy on the OS dataset. Our initial attempts at schemes such as non-maximum suppression of\noverlapping regions that are quite successful for object segmentation [16] performed rather poorly. We believe this is\nbecause unlike objects, material information is fairly localized and highly irregularly shaped in an image. However,\nthere is room for improvement by combining evidence from\nmultiple segmentations.\n4.4. Convolutional layer analysis\nWe study the performance of \ufb01lter banks extracted from\ndifferent layers of a CNN in the FV-CNN framework. We"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a403a-6db4-11ee-962f-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-37", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "4.4. Convolutional layer analysis\nWe study the performance of \ufb01lter banks extracted from\ndifferent layers of a CNN in the FV-CNN framework. We\nuse the VGG-M network which has \ufb01ve convolutional layers. Results on various datasets, obtained as in Sect. 4.1\nand 4.2, are shown in Fig. 4. In addition we also show the\nperformance using FVs constructed from dense SIFT using\na number of words such that the resulting FV is roughly the\nsame size of FV-CNN. The CNN \ufb01lter banks from layer\n3 and beyond signi\ufb01cantly outperform SIFT. The performance monotonically improves from layer one to \ufb01ve.\n5. Conclusions\nWe have conducted a range of experiments on material\nand texture attribute recognition in a large dataset of textures in clutter. This benchmark was derived from OpenSurfaces, an earlier contribution of the computer graphics community, highlights the potential for collaboration between\ncomputer graphics and vision communities. We have also\nevaluated a number of state-of-the-art texture descriptors on\nthese and many other benchmarks. Our main \ufb01nding is that\norderless pooling of convolutional neural network features\nis a remarkably good texture descriptor, versatile enough to\ndubbed as a scene and object descriptor, resulting in new"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a403b-6db4-11ee-8b17-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-38", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "orderless pooling of convolutional neural network features\nis a remarkably good texture descriptor, versatile enough to\ndubbed as a scene and object descriptor, resulting in new\nstate-of-the-art performance in several benchmarks.\nbrick cardboard carpet_rug ceramic concrete fabric_cloth food glass\ngranite_marble hair other laminate leather metal painted paper_tissue\nplastic_clear plastic_opaque stone tile wallpaper wood(a) (b) (c) (d) (e) (f)\nFigure 2. OS material recognition results. Example test image with material recognition and segmentation on the OS dataset. (a) original\nimage. (b) ground truth segmentations from the OpenSurfaces repository (note that not all pixels are annotated). (c) FC-CNN and crispregion proposals segmentation results. (d) incorrectly predicted pixels (restricted to the ones annotated). (e-f) the same, but for FV-CNN.\nbuilding grass tree cow sheep sky aeroplane water\nface car bicycle flower sign bird book chair\nroad cat dog body boat\n(a) (b) (c) (d) (e) (f)\nFigure 3. MSRC object segmentation results. (a) image, (b) ground-truth, (c-d) FC-CNN, (d-e) FV-CNN segmentation and errors."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a403c-6db4-11ee-9e54-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-39", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "Acknowledgements M. Cimpoi was supported by ERC\ngrant VisRec no. 228180 and the XRCE UAC grant. S. Maji\nacknowledges a faculty startup grant from UMass, Amherst.References\n[1] E. H. Adelson. On seeing stuff: The perception of materials by humans and machines. SPIE , 4299, 2001. 1\n[2] M. Amadasun and R. King. Textural features corresponding to textural properties. Systems, Man, and Cybernetics , 19(5), 1989. 1\n[3] P. Arbel \u00b4aez, J. Pont-Tuset, J. T. Barron, F. Marques, and J. Malik.\nMultiscale combinatorial grouping. CVPR, 2014. 4, 5, 7\n[4] S. Bell, P. Upchurch, N. Snavely, and K. Bala. Opensurfaces: A\nrichly annotated catalog of surface appearance. In Proc. SIGGRAPH ,\n2013. 2\n[5] B. Caputo, E. Hayman, and P. Mallikarjuna. Class-speci\ufb01c material\ncategorisation. In ICCV , 2005. 1, 4"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a403d-6db4-11ee-a604-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-40", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "categorisation. In ICCV , 2005. 1, 4\n[6] K. Chat\ufb01eld, K. Simonyan, A. Vedaldi, and A. Zisserman. Return\nof the devil in the details: Delving deep into convolutional nets. In\nProc. BMVC , 2014. 2, 3, 4, 6\n[7] M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, , and A. Vedaldi.\nDescribing textures in the wild. In Proc. CVPR , 2014. 1, 3, 4, 5\n[8] K. J. Dana, B. van Ginneken, S. K. Nayar, and J. J. Koenderink.\nRe\ufb02ectance and texture of real world surfaces. ACM Transactions on\nGraphics , 18(1):1\u201334, 1999. 1\n[9] A. Efros and T. Leung. Texture synthesis by non-parametric sampling. In CVPR , volume 2, 1999. 1\n[10] M. Everingham, A. Zisserman, C. Williams, and L. V . Gool. The\nPASCAL visual obiect classes challenge 2007 (VOC2007) results.\nTechnical report, Pascal Challenge, 2007. 3, 4"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a403e-6db4-11ee-ad12-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-41", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "PASCAL visual obiect classes challenge 2007 (VOC2007) results.\nTechnical report, Pascal Challenge, 2007. 3, 4\n[11] V . Ferrari and A. Zisserman. Learning visual attributes. In Proc.\nNIPS , 2007. 1\n[12] D. Forsyth. Shape from texture and integrability. In ICCV , volume 2,\npages 447\u2013452. IEEE, 2001. 1\n[13] J. G \u02daarding. Shape from texture for smooth curved surfaces in perspective projection. Journal of Mathematical Imaging and Vision ,\n2(4):327\u2013350, 1992. 1\n[14] R. B. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature\nhierarchies for accurate object detection and semantic segmentation.\nInProc. CVPR , 2014. 2, 3\n[15] Y . Gong, L. Wang, R. Guo, and S. Lazebnik. Multi-scale orderless\npooling of deep convolutional activation features. In Proc. ECCV ,\n2014. 2, 3, 6\n[16] B. Hariharan, P. Arbel \u00b4aez, R. Girshick, and J. Malik. Simultaneous\ndetection and segmentation. In Computer Vision\u2013ECCV 2014 , pages"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a403f-6db4-11ee-86d6-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-42", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "detection and segmentation. In Computer Vision\u2013ECCV 2014 , pages\n297\u2013312. Springer, 2014. 2, 7\n[17] E. Hayman, B. Caputo, M. Fritz, and J.-O. Eklundh. On the significance of real-world conditions for material classi\ufb01cation. ECCV ,\n2004. 4\n[18] K. He, X. Zhang, S. Ren, and J. Sun. Spatial pyramid pooling in deep\nconvolutional networks for visual recognition. In Proc. ECCV , 2014.\n2\n[19] P. Isola, D. Zoran, D. Krishnan, and E. H. Adelson. Crisp boundary\ndetection using pointwise mutual information. In Proc. ECCV , 2014.\n4, 7\n[20] A. Jain and F. Farrokhnia. Unsupervised texture segmentation using\ngabor \ufb01lters. Pattern recognition , 24(12):1167\u20131186, 1991. 1\n[21] Y . Jia. Caffe: An open source convolutional architecture for fast\nfeature embedding. http://caffe.berkeleyvision.org/ ,\n2013. 3, 4, 6\n[22] B. Julesz and J. R. Bergen. Textons, the fundamental elements in"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4040-6db4-11ee-89fe-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-43", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "2013. 3, 4, 6\n[22] B. Julesz and J. R. Bergen. Textons, the fundamental elements in\npreattentive vision and perception of textures. Bell System Technical\nJournal , 62(6, Pt 3):1619\u20131645, Jul-Aug 1983. 1\n[23] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classi\ufb01cation with deep convolutional neural networks. In Proc. NIPS , 2012.\n2, 3\n[24] L. Ladicky, C. Russell, P. Kohli, and P. Torr. Graph cut based inference with co-occurrence statistics. In Proc. ECCV , pages 239\u2013253,\n2010. 5\n[25] L. Ladicky, P. Sturgess, K. Alahari, C. Russell, and P. Torr. What,\nwhere and how many? combining object detectors and crfs. In Proc.\nECCV , 2010. 7\n[26] T. Leung and J. Malik. Representing and recognizing the visual appearance of materials using three-dimensional textons. International\nJournal of Computer Vision , 43(1):29\u201344, 2001. 1"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4041-6db4-11ee-adf0-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-44", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "Journal of Computer Vision , 43(1):29\u201344, 2001. 1\n[27] L. Liu, L. Wang, and X. Liu. In defense of soft-assignment coding.\nInComputer Vision (ICCV), 2011 IEEE International Conference on ,\npages 2486\u20132493. IEEE, 2011. 1[28] D. G. Lowe. Object recognition from local scale-invariant features.\nInProc. ICCV , 1999. 1\n[29] B. Manjunath and R. Chellappa. Unsupervised texture segmentation\nusing markov random \ufb01eld models. IEEE Transactions on Pattern\nAnalysis and Machine Intelligence , 13(5):478\u2013482, 1991. 1\n[30] M. Oquab, L. Bottou, I. Laptev, and J. Sivic. Learning and Transferring Mid-Level Image Representations using Convolutional Neural\nNetworks. In Proc. CVPR , 2014. 2\n[31] O. M. Parkhi, K. Simonyan, A. Vedaldi, and A. Zisserman. A compact and discriminative face track descriptor. In Proc. CVPR , 2014.\n4"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4042-6db4-11ee-9f10-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-45", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "4\n[32] F. Perronnin, J. S \u00b4anchez, and T. Mensink. Improving the \ufb01sher kernel for large-scale image classi\ufb01cation. In Computer Vision\u2013ECCV\n2010 , pages 143\u2013156. Springer, 2010. 1\n[33] X. Qi, R. Xiao, C. G. Li, Y . Qiao, J. Guo, and X. Tang. Pairwise rotation invariant co-occurrence local binary pattern. PAMI ,\n36(11):2199\u20132213, Nov 2014. 5\n[34] A. Quattoni and A. Torralba. Recognizing indoor scenes. In Proc.\nCVPR , 2009. 2, 4\n[35] A. S. Razavin, H. Azizpour, J. Sullivan, and S. Carlsson. Cnn features off-the-shelf: An astounding baseline for recognition. In DeepVision workshop , 2014. 3\n[36] G. Schwartz and K. Nishino. Visual material traits: Recognizing\nper-pixel material context. In Proc. CVCP , 2013. 1\n[37] L. Sharan, C. Liu, R. Rosenholtz, and E. H. Adelson. Recognizing\nmaterials using perceptually inspired features. International Journal"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4043-6db4-11ee-a30b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-46", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "[37] L. Sharan, C. Liu, R. Rosenholtz, and E. H. Adelson. Recognizing\nmaterials using perceptually inspired features. International Journal\nof Computer Vision , 103(3):348\u2013371, 2013. 1, 4, 5\n[38] L. Sharan, R. Rosenholtz, and E. H. Adelson. Material perceprion:\nWhat can you see in a brief glance? Journal of Vision , 9:784(8),\n2009. 1\n[39] K. Simonyan and A. Zisserman. Very deep convolutional networks\nfor large-scale image recognition. CoRR , abs/1409.1556, 2014. 2, 4\n[40] M. Sulc and J. Matas. Fast features invariant to rotation and scale of\ntexture. Technical report, 2014. 5\n[41] M. Varma and A. Zisserman. Texture classi\ufb01cation: Are \ufb01lter banks\nnecessary? In CVPR , volume 2, pages II\u2013691. IEEE, 2003. 1\n[42] J. Wang, J. Yang, K. Yu, F. Lv, T. Huang, and Y . Gong. Localityconstrained linear coding for image classi\ufb01cation. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on ,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4044-6db4-11ee-8d81-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-47", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "pages 3360\u20133367. IEEE, 2010. 1\n[43] L. Wei and M. Levoy. Fast texture synthesis using treestructured vector quantization. In SIGGRAPH , pages 479\u2013488. ACM\nPress/Addison-Wesley Publishing Co., 2000. 1\n[44] Y . Wei, W. Xia, J. Huang, B. Ni, J. Dong, Y . Zhao, and S. Yan. Cnn:\nSingle-label to multi-label. 2014. 5, 6\n[45] P. Welinder, S. Branson, T. Mita, C. Wah, and F. Schroff. Caltechucsd birds 200. Technical report, Caltech-UCSD, 2010. 4\n[46] N. Zhang, J. Donahue, R. Girshickr, and T. Darrell. Part-based RCNNs for \ufb01ne-grained category detection. In Proc. ECCV , 2014. 5,\n6\n[47] B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva. Learning\ndeep features for scene recognition using places database. In Proc.\nNIPS , 2014. 2, 5, 6, 7"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4045-6db4-11ee-a6b4-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-48", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "deep features for scene recognition using places database. In Proc.\nNIPS , 2014. 2, 5, 6, 7\n[48] X. Zhou, K. Yu, T. Zhang, and T. S. Huang. Image classi\ufb01cation\nusing super-vector coding of local image descriptors. In Computer\nVision\u2013ECCV 2010 , pages 141\u2013154. Springer, 2010. 1"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4046-6db4-11ee-b301-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-0", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "Material Recognition in the Wild with the Materials in Context Database\nSean Bell\u0003Paul Upchurch\u0003Noah Snavely Kavita Bala\nDepartment of Computer Science, Cornell University\nfsbell,paulu,snavely,kb g@cs.cornell.edu\nAbstract\nRecognizing materials in real-world images is a challenging task. Real-world materials have rich surface texture,\ngeometry, lighting conditions, and clutter, which combine\nto make the problem particularly dif\ufb01cult. In this paper, we\nintroduce a new, large-scale, open dataset of materials in\nthe wild, the Materials in Context Database (MINC ), and\ncombine this dataset with deep learning to achieve material\nrecognition and segmentation of images in the wild.\nMINC is an order of magnitude larger than previous material databases, while being more diverse and well-sampled\nacross its 23 categories. Using MINC , we train convolutional neural networks (CNNs) for two tasks: classifying\nmaterials from patches, and simultaneous material recognition and segmentation in full images. For patch-based classi\ufb01cation on MINC we found that the best performing CNN\narchitectures can achieve 85.2% mean class accuracy. We\nconvert these trained CNN classi\ufb01ers into an ef\ufb01cient fully"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4047-6db4-11ee-9f65-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-1", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "architectures can achieve 85.2% mean class accuracy. We\nconvert these trained CNN classi\ufb01ers into an ef\ufb01cient fully\nconvolutional framework combined with a fully connected\nconditional random \ufb01eld (CRF) to predict the material at\nevery pixel in an image, achieving 73.1% mean class accuracy. Our experiments demonstrate that having a large,\nwell-sampled dataset such as MINC is crucial for real-world\nmaterial recognition and segmentation.\n1. Introduction\nMaterial recognition plays a critical role in our understanding of and interactions with the world. To tell whether\na surface is easy to walk on, or what kind of grip to use\nto pick up an object, we must recognize the materials that\nmake up our surroundings. Automatic material recognition\ncan be useful in a variety of applications, including robotics,\nproduct search, and image editing for interior design. But recognizing materials in real-world images is very challenging.\nMany categories of materials, such as fabric or wood, are\nvisually very rich and span a diverse range of appearances.\nMaterials can further vary in appearance due to lighting and\nshape. Some categories, such as plastic and ceramic, are of\u0003Authors contributed equally\nCNN\nSliding\nCNN\u201cwood\u201d\nTransfer weights(trained)\n(discretely"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4048-6db4-11ee-910d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-2", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "CNN\nSliding\nCNN\u201cwood\u201d\nTransfer weights(trained)\n(discretely \noptimized)(fixed)AMT AMT AMT MINC\n3 million\npatches OpenSurfacesFlickr\nHouzz(a) Constructing MINC\nMaterial\nLabels(b) Patch material classification\n(c) Full scene material classificationP(material)\nDense\nCRF\nPatchFigure 1. Overview. (a) We construct a new dataset by combining\nOpenSurfaces [ 1] with a novel three-stage Amazon Mechanical\nTurk (AMT) pipeline. (b)We train various CNNs on patches from\nMINC to predict material labels. (c)We transfer the weights to a\nfully convolutional CNN to ef\ufb01ciently generate a probability map\nacross the image; we then use a fully connected CRF to predict the\nmaterial at every pixel.\nten smooth and featureless, requiring reasoning about subtle\ncues or context to differentiate between them.\nLarge-scale datasets (e.g., ImageNet [ 21], SUN [ 31,19]\nand Places [ 34]) combined with convolutional neural networks (CNNs) have been key to recent breakthroughs in\nobject recognition and scene classi\ufb01cation. Material recognition is similarly poised for advancement through large-scale\ndata and learning. To date, progress in material recognition"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4049-6db4-11ee-a4b3-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-3", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "object recognition and scene classi\ufb01cation. Material recognition is similarly poised for advancement through large-scale\ndata and learning. To date, progress in material recognition\nhas been facilitated by moderate-sized datasets like the Flickr\nMaterial Database (FMD) [ 26]. FMD contains ten material\ncategories, each with 100 samples drawn from Flickr photos.\nThese images were carefully selected to illustrate a wide\nrange of appearances for these categories. FMD has been\nused in research on new features and learning methods for\nmaterial perception and recognition [ 17,10,20,25]. While\nFMD was an important step towards material recognition, it\nis not suf\ufb01cient for classifying materials in real-world im1arXiv:1412.0623v2  [cs.CV]  14 Apr 2015\nagery. This is due to the relatively small set of categories,\nthe relatively small number of images per category, and also\nbecause the dataset has been designed around hand-picked\niconic images of materials. The OpenSurfaces dataset [ 1]\naddresses some of these problems by introducing 105,000\nmaterial segmentations from real-world images, and is signi\ufb01cantly larger than FMD. However, in OpenSurfaces many\nmaterial categories are under-sampled, with only tens of\nimages."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a404a-6db4-11ee-9672-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-4", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "material categories are under-sampled, with only tens of\nimages.\nA major contribution of our paper is a new, well-sampled\nmaterial dataset, called the Materials in Context Database\n(MINC ), with 3 million material samples. MINC is more\ndiverse, has more examples in less common categories, and\nis much larger than existing datasets. MINC draws data from\nboth Flickr images, which include many \u201cregular\u201d scenes,\nas well as Houzz images from professional photographers of\nstaged interiors. These sources of images each have different\ncharacteristics that together increase the range of materials\nthat can be recognized. See Figure 2 for examples of our\ndata. We make our full dataset available online at http:\n//minc.cs.cornell.edu/.\nWe use this data for material recognition by training different CNN architectures on this new dataset. We perform\nexperiments that illustrate the effect of network architecture, image context, and training data size on subregions\n(i.e., patches) of a full scene image. Further, we build on\nour patch classi\ufb01cation results and demonstrate simultaneous material recognition and segmentation of an image by\nperforming dense classi\ufb01cation over the image with a fully\nconnected conditional random \ufb01eld (CRF) model [ 12]. By"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a404b-6db4-11ee-9876-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-5", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "performing dense classi\ufb01cation over the image with a fully\nconnected conditional random \ufb01eld (CRF) model [ 12]. By\nreplacing the fully connected layers of the CNN with convolutional layers [ 24], the computational burden is signi\ufb01cantly\nlower than a naive sliding window approach.\nIn summary, we make two new contributions:\n\u000fWe introduce a new material dataset, MINC , and 3stage crowdsourcing pipeline for ef\ufb01ciently collecting\nmillions of click labels (Section 3.2).\n\u000fOur new semantic segmentation method combines a\nfully-connected CRF with unary predictions based on\nCNN learned features (Section 4.2) for simultaneous\nmaterial recognition and segmentation.\n2. Prior Work\nMaterial Databases. Much of the early work on material\nrecognition focused on classifying speci\ufb01c instances of textures or material samples. For instance, the CUReT [ 4]\ndatabase contains 61 material samples, each captured under\n205 different lighting and viewing conditions. This led to\nresearch on the task of instance-level texture or material classi\ufb01cation [ 15,30], and an appreciation of the challenges of\nbuilding features that are invariant to pose and illumination."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a404c-6db4-11ee-a597-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-6", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "building features that are invariant to pose and illumination.\nLater, databases with more diverse examples from each ma-terial category began to appear, such as KTH-TIPS [ 9,2],\nand led explorations of how to generalize from one example\nof a material to another\u2014from one sample of wood to a completely different sample, for instance. Real-world texture\nattributes have also recently been explored [3].\nIn the domain of categorical material databases, Sharan et\nal. released FMD [ 26] (described above). Subsequently,\nBell et al. released OpenSurfaces [ 1] which contains over\n20,000 real-world scenes labeled with both materials and objects, using a multi-stage crowdsourcing pipeline. Because\nOpenSurfaces images are drawn from consumer photos on\nFlickr, material samples have real-world context, in contrast\nto prior databases (CUReT, KTH-TIPS, FMD) which feature cropped stand-alone samples. While OpenSurfaces is a\ngood starting point for a material database, we substantially\nexpand it with millions of new labels.\nMaterial recognition. Much prior work on material recognition has focused on the classi\ufb01cation problem (categorizing\nan image patch into a set of material categories), often using"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a404d-6db4-11ee-8d15-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-7", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "Material recognition. Much prior work on material recognition has focused on the classi\ufb01cation problem (categorizing\nan image patch into a set of material categories), often using\nhand-designed image features. For FMD, Liu et al. [17] introduced re\ufb02ectance-based edge features in conjunction with\nother general image features. Hu et al. [10] proposed features based on variances of oriented gradients. Qi et al. [20]\nintroduced a pairwise local binary pattern (LBP) feature.\nLiet al. [16] synthesized a dataset based on KTH-TIPS2\nand built a classi\ufb01er from LBP and dense SIFT. Timofte et\nal. [29] proposed a classi\ufb01cation framework with minimal\nparameter optimization. Schwartz and Nishino [ 23] introduced material traits that incorporate learned convolutional\nauto-encoder features. Recently, Cimpoi et al. [3] developed a CNN and improved Fisher vector (IFV) classi\ufb01er that\nachieves state-of-the-art results on FMD and KTH-TIPS2.\nFinally, it has been shown that jointly predicting objects and\nmaterials can improve performance [10, 33].\nConvolutional neural networks. While CNNs have been"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a404e-6db4-11ee-a14d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-8", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "Finally, it has been shown that jointly predicting objects and\nmaterials can improve performance [10, 33].\nConvolutional neural networks. While CNNs have been\naround for a few decades, with early successes such as\nLeNet [ 14], they have only recently led to state-of-theart results in object classi\ufb01cation and detection, leading\nto enormous progress. Driven by the ILSVRC challenge [ 21], we have seen many successful CNN architectures [ 32,24,28,27], led by the work of Krizhevsky et al.\non their SuperVision (a.k.a. AlexNet ) network [ 13], with\nmore recent architectures including GoogLeNet [ 28]. In addition to image classi\ufb01cation, CNNs are the state-of-the-art\nfor detection and localization of objects, with recent work\nincluding R-CNNs [ 7], Overfeat [ 24], and VGG [ 27]. Finally, relevant to our goal of per-pixel material segmentation,\nFarabet et al. [6] use a multi-scale CNN to predict the class\nat every pixel in a segmentation. Oquab et al. [18] employ a\nsliding window approach to localize patch classi\ufb01cation of\nobjects. We build on this body of work in deep learning to"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a404f-6db4-11ee-9b41-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-9", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "sliding window approach to localize patch classi\ufb01cation of\nobjects. We build on this body of work in deep learning to\nsolve our problem of material recognition and segmentation.\n2\nBrick Carpet Ceramic Fabric Foliage Food Glass Hair\nLeather Metal Mirror Other Painted Paper Plastic Pol. stone\nSkin Sky Stone Tile Wallpaper Water Wood\nFigure 2. Example patches from all 23 categories of the Materials in Context Database (MINC ). Note that we sample patches so that the\npatch center is the material in question (and not necessarily the entire patch). See Table 1 for the size of each category.\n3. The Materials in Context Database (MINC)\nWe now describe the methodology that went into building\nour new material database. Why a new database? We needed\na dataset with the following properties:\n\u000fSize: It should be suf\ufb01ciently large that learning methods can generalize beyond the training set.\n\u000fWell-sampled : Rare categories should be represented\nwith a large number of examples.\n\u000fDiversity : Images should span a wide range of appearances of each material in real-world settings.\n\u000fNumber of categories : It should contain many different materials found in the real world.\n3.1. Sources of data\nWe decided to start with the public, crowdsourced\nOpenSurfaces dataset [ 1] as the seed for MINC since it is"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4050-6db4-11ee-bfdf-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-10", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "3.1. Sources of data\nWe decided to start with the public, crowdsourced\nOpenSurfaces dataset [ 1] as the seed for MINC since it is\ndrawn from Flickr imagery of everyday, real-world scenes\nwith reasonable diversity. Furthermore, it has a large number\nof categories and the most samples of all prior databases.\nWhile OpenSurfaces data is a good start, it has a few limitations. Many categories in OpenSurfaces are not well sampled. While the largest category, wood , has nearly 20K samples, smaller categories, such as water , have only tens of examples. This imbalance is due to the way the OpenSurfaces\ndataset was annotated; workers on Amazon Mechanical Turk\n(AMT) were free to choose any material subregion to segment. Workers often gravitated towards certain common\ntypes of materials or salient objects, rather than being encouraged to label a diverse set of materials. Further, the\nimages come from a single source (Flickr).\nWe decided to augment OpenSurfaces with substantially\nmore data, especially for underrepresented material cate-gories, with the initial goal of gathering at least 10K samples\nper material category. We decided to gather this data from\nanother source of imagery, professional photos on the interior design website Houzz (houzz.com). Our motivation for\nusing this different source of data was that, despite Houzz"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4051-6db4-11ee-a946-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-11", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "another source of imagery, professional photos on the interior design website Houzz (houzz.com). Our motivation for\nusing this different source of data was that, despite Houzz\nphotos being more \u201cstaged\u201d (relative to Flickr photos), they\nactually represent a larger variety of materials. For instance,\nHouzz photos contain a wide range of types of polished\nstone. With these sources of image data, we now describe\nhow we gather material annotations.\n3.2. Segments, Clicks, and Patches\nWhat speci\ufb01c kinds of material annotations make for a\ngood database? How should we collect these annotations?\nThe type of annotations to collect is guided in large part by\nthe tasks we wish to generate training data for. For some\ntasks such as scene recognition, whole-image labels can\nsuf\ufb01ce [ 31,34]. For object detection, labeled bounding\nboxes as in PASCAL are often used [ 5]. For segmentation or\nscene parsing tasks, per-pixel segmentations are required [ 22,\n8]. Each style of annotation comes with a cost proportional\nto its complexity. For materials, we decided to focus on two\nproblems, guided by prior work:\n\u000fPatch material classi\ufb01cation. Given an image patch,\nwhat kind of material is it at the center?"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4052-6db4-11ee-9470-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-12", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "problems, guided by prior work:\n\u000fPatch material classi\ufb01cation. Given an image patch,\nwhat kind of material is it at the center?\n\u000fFull scene material classi\ufb01cation. Given a full image, produce a full per-pixel segmentation and labeling.\nAlso known as semantic segmentation orscene parsing\n(but in our case, focused on materials). Note that classi\ufb01cation can be a component of segmentation, e.g., with\nsliding window approaches.\n3\n(a) Which images \ncontain wood ?(b) Click on 3 \npoints of wood(c) What is this \nmaterial?Figure 3. AMT pipeline schematic for collecting clicks. (a)\nWorkers \ufb01lter by images that contain a certain material, (b)workers click on materials, and (c)workers validate click locations by\nre-labeling each point. Example responses are shown in orange.\nSegments. OpenSurfaces contains material segmentations\u2014\ncarefully drawn polygons that enclose same-material regions.\nTo form the basis of MINC, we selected OpenSurfaces segments with high con\ufb01dence (inter-worker agreement) and\nmanually curated segments with low con\ufb01dence, giving a\ntotal of 72K shapes. To better balance the categories, we"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4053-6db4-11ee-be7b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-13", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "manually curated segments with low con\ufb01dence, giving a\ntotal of 72K shapes. To better balance the categories, we\nmanually segmented a few hundred extra samples for sky,\nfoliage andwater .\nSince some of the OpenSurfaces categories are dif\ufb01cult\nfor humans, we consolidated these categories. We found\nthat many AMT workers could not disambiguate stone from\nconcrete ,clear plastic from opaque plastic , and granite\nfrom marble . Therefore, we merged these into stone ,plastic ,\nandpolished stone respectively. Without this merging, many\nground truth examples in these categories would be incorrect.\nThe \ufb01nal list of 23 categories is shown in Table 1. The\ncategory other is different in that it was created by combining\nvarious smaller categories.\nClicks. Since we want to expand our dataset to millions\nof samples, we decided to augment OpenSurfaces segments\nby collecting clicks : single points in an image along with a\nmaterial label, which are much cheaper and faster to collect.\nFigure 3 shows our pipeline for collecting clicks.\nInitially, we tried asking workers to click on examples\nof a given material in a photo. However, we found that\nworkers would get frustrated if the material was absent in\ntoo many of the photos. Thus, we added an initial \ufb01rst"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4054-6db4-11ee-be5d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-14", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "workers would get frustrated if the material was absent in\ntoo many of the photos. Thus, we added an initial \ufb01rst\nstage where workers \ufb01lter out such photos. To increase\nthe accuracy of our labels, we verify the click labels by\nasking different workers to specify the material for each\nclick without providing them with the label from the previous\nstage.\nTo ensure that we obtain high quality annotations and\navoid collecting labels from workers who are not making an\neffort, we include secret known answers (sentinels) in the\n\ufb01rst and third stages, and block workers with an accuracy\nbelow 50% and 85% respectively. We do not use sentinels\nin the second stage since it would require per-pixel ground\ntruth labels, and it turned out not to be necessary. Workers\ngenerally performed all three tasks so we could identify bad\nworkers in the \ufb01rst or third task.Patches Category Patches Category Patches Category\n564,891 Wood 114,085 Polished stone 35,246 Skin\n465,076 Painted 98,891 Carpet 29,616 Stone\n397,982 Fabric 83,644 Leather 28,108 Ceramic\n216,368 Glass 75,084 Mirror 26,103 Hair\n188,491 Metal 64,454 Brick 25,498 Food\n147,346 Tile 55,364 Water 23,779 Paper"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4055-6db4-11ee-b1e7-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-15", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "216,368 Glass 75,084 Mirror 26,103 Hair\n188,491 Metal 64,454 Brick 25,498 Food\n147,346 Tile 55,364 Water 23,779 Paper\n142,150 Sky 39,612 Other 14,954 Wallpaper\n120,957 Foliage 38,975 Plastic\nTable 1. MINC patch counts by category. Patches were created\nfrom both OpenSurfaces segments and our newly collected clicks .\nSee Section 3.2 for details.\nMaterial clicks were collected for both OpenSurfaces\nimages and the new Houzz images. This allowed us to use\nlabels from OpenSurfaces to generate the sentinel data; we\nincluded 4 sentinels per task. With this streamlined pipeline\nwe collected 2,341,473 annotations at an average cost of\n$0.00306 per annotation (stage 1: $0.02 / 40 images, stage\n2: $0.10 / 50 images, 2, stage 3: $0.10 / 50 points).\nPatches. Labeled segments and clicks form the core of\nMINC . For training CNNs and other types of classi\ufb01ers,\nit is useful to have data in the form of \ufb01xed-sized patches.\nWe convert both forms of data into a uni\ufb01ed dataset format:\nsquare image patches. We use a patch center andpatch scale"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4056-6db4-11ee-98b5-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-16", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "We convert both forms of data into a uni\ufb01ed dataset format:\nsquare image patches. We use a patch center andpatch scale\n(a multiplier of the smaller image dimension) to de\ufb01ne the\nimage subregion that makes a patch. For our patch classi\ufb01cation experiments, we use 23.3% of the smaller image\ndimension. Increasing the patch scale provides more context\nbut reduces the spatial resolution. Later in Section 5 we\njustify our choice with experiments that vary the patch scale\nfor AlexNet.\nWe place a patch centered around each click label. For\neach segment , if we were to place a patch at every interior\npixel then we would have a very large and redundant dataset.\nTherefore, we Poisson-disk subsample each segment , separating patch centers by at least 9.1% of the smaller image\ndimension. These segments generated 655,201 patches (an\naverage of 9.05 patches per segment ). In total, we generated 2,996,674 labeled patches from 436,749 images. Patch\ncounts are shown in Table 1, and example patches from\nvarious categories are illustrated in Figure 2.\n4. Material recognition in real-world images\nOur goal is to train a system that recognizes the material\nat every pixel in an image. We split our training procedure\ninto multiple stages and analyze the performance of the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4057-6db4-11ee-ad54-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-17", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "Our goal is to train a system that recognizes the material\nat every pixel in an image. We split our training procedure\ninto multiple stages and analyze the performance of the\nnetwork at each stage. First, we train a CNN that produces a\nsingle prediction for a given input patch. Then, we convert\nthe CNN into a sliding window and predict materials on a\ndense grid across the image. We do this at multiple scales\nand average to obtain a unary term. Finally, a dense CRF\n[12] combines the unary term with fully connected pairwise\nreasoning to output per-pixel material predictions. The entire\nsystem is depicted in Figure 1, and described more below.\n4\nSky\nWater\nFoliageTile\n(a) Multiscale input (b) Probability map (12 of 23 shown) (c) Predicted labelsSliding\nCNNDense\nCRF\nWood\nPlastic Sliding\nCNNSliding\nCNN\nAvg.\nGlass\nHair\nFabric\nPolished\nstone\nStone\n PaperWaterSky\nFoliage\nTilePaperWood\nStoneWood\nTileFigure 4. Pipeline for full scene material classi\ufb01cation. An image (a)is resized to multiple scales [1=p\n2;1;p\n2]. The same sliding CNN"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4058-6db4-11ee-94f4-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-18", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "2;1;p\n2]. The same sliding CNN\npredicts a probability map (b)across the image for each scale; the results are upsampled and averaged. A fully connected CRF predicts a\n\ufb01nal label for each pixel (c). This example shows predictions from a single GoogLeNet converted into a sliding CNN (no average pooling).\n4.1. Training procedure\nMINC contains 3 million patches that we split into training, validation and test sets. Randomly splitting would result\nin nearly identical patches (e.g., from the same OpenSurfaces segment) being put in training and test, thus in\ufb02ating\nthe test score. To prevent correlation, we group photos into\nclusters of near-duplicates, then assign each cluster to one\nof train, validate or test. We make sure that there are at least\n75segments of each category in the test set to ensure there\nare enough segments to evaluate segmentation accuracy. To\ndetect near-duplicates, we compare AlexNet CNN features\ncomputed from each photo (see the supplemental for details).\nFor exact duplicates, we discard all but one of the copies.\nWe train all of our CNNs by \ufb01ne-tuning the network\nstarting from the weights obtained by training on 1.2 million images from ImageNet (ILSVRC2012). When training"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4059-6db4-11ee-bc85-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-19", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "starting from the weights obtained by training on 1.2 million images from ImageNet (ILSVRC2012). When training\nAlexNet , we use stochastic gradient descent with batchsize\n128, dropout rate 0.5, momentum 0.9, and a base learning\nrate of 10\u00003that decreases by a factor of 0.25 every 50,000\niterations. For GoogLeNet , we use batchsize 69, dropout 0.4,\nand learning rate \u000bt= 10\u00004p\n1\u0000t=250000 for iteration t.\nOur training set has a different number of examples per\nclass, so we cycle through the classes and randomly sample\nan example from each class. Failing to properly balance the\nexamples results in a 5.7% drop in mean class accuracy (on\nthe validation set). Further, since it has been shown to reduce\nover\ufb01tting, we randomly augment samples by taking crops\n(227\u0002227out of 256\u0002256), horizontal mirror \ufb02ips, spatial\nscales in the range [1=p\n2;p\n2], aspect ratios from 3:4 to 4:3,\nand amplitude shifts in [0:95;1:05]. Since we are looking at\nlocal regions, we subtract a per-channel mean (R: 124, G:"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a405a-6db4-11ee-b50b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-20", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "and amplitude shifts in [0:95;1:05]. Since we are looking at\nlocal regions, we subtract a per-channel mean (R: 124, G:\n117, B: 104) rather than a mean image [13].\n4.2. Full scene material classi\ufb01cation\nFigure 4 shows an overview of our method for simultaneously segmenting and recognizing materials. Given a\nCNN that can classify individual points in the image, we\nconvert it to a sliding window detector and densely classify\na grid across the image. Speci\ufb01cally, we replace the last\nfully connected layers with convolutional layers, so that the\nnetwork is fully convolutional and can classify images ofany shape. After conversion, the weights are \ufb01xed and not\n\ufb01ne-tuned. With our converted network, the strides of each\nlayer cause the network to output a prediction every 32 pixels. We obtain predictions every 16 pixels by shifting the\ninput image by half-strides (16 pixels). While this appears to\nrequire 4x the computation, Sermanet et al. [24] showed that\nthe convolutions can be reused and only the pool5 through\nfc8 layers need to be recomputed for the half-stride shifts.\nAdding half-strides resulted in a minor 0.2% improvement\nin mean class accuracy across segments (after applying the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a405b-6db4-11ee-a375-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-21", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "Adding half-strides resulted in a minor 0.2% improvement\nin mean class accuracy across segments (after applying the\ndense CRF, described below), and about the same mean class\naccuracy at click locations.\nThe input image is resized so that a patch maps to a\n256x256 square. Thus, for a network trained at patch scale\ns, the resized input has smaller dimension d= 256=s. Note\nthatdis inversely proportional to scale, so increased context\nleads to lower spatial resolution. We then add padding so\nthat the output probability map is aligned with the input\nwhen upsampled. We repeat this at 3 different scales (smaller\ndimensiond=p\n2,d,dp\n2), upsample each output probability\nmap with bilinear interpolation, and average the predictions.\nTo make the next step more ef\ufb01cient, we upsample the output\nto a \ufb01xed smaller dimension of 550.\nWe then use the dense CRF of Kr\u00a8ahenb \u00a8uhlet al. [12] to\npredict a label at every pixel, using the following energy:\nE(xjI) =X\ni i(xi) +X\ni<j ij(xi;xj) (1)\n i(xi) =\u0000logpi(xi) (2)"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a405c-6db4-11ee-bec1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-22", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "i i(xi) +X\ni<j ij(xi;xj) (1)\n i(xi) =\u0000logpi(xi) (2)\n ij(xi;xj) =wp\u000e(xi6=xj)k(fi\u0000fj) (3)\nwhere iis the unary energy (negative log of the aggregated softmax probabilities) and  ijis the pairwise term\nthat connects every pair of pixels in the image. We use a\nsingle pairwise term with a Potts label compatibility term\n\u000eweighted by wpand unit Gaussian kernel k. For the featuresfi, we convert the RGB image to L*a*b*and use color\n(IL\ni;Ia\ni;Ib\ni)and position (px;py)as pairwise features for\neach pixel: fi=h\npx\ni\n\u0012pd;py\ni\n\u0012pd;IL\ni\n\u0012L;Ia\ni\n\u0012ab;Ib\ni\n\u0012abi\n, wheredis the\nsmaller image dimension. Figure 4 shows an example unary\ntermpiand the resulting segmentation x.\n5\nFigure 5. Varying patch scale . We train/test patches of different\nscales (the patch locations do not vary). The optimum is a trade-off"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a405d-6db4-11ee-a542-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-23", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "5\nFigure 5. Varying patch scale . We train/test patches of different\nscales (the patch locations do not vary). The optimum is a trade-off\nbetween context and spatial resolution. CNN: AlexNet.\nArchitecture Validation Test\nAlexNet [13] 82.2% 81.4%\nGoogLeNet [28] 85.9% 85.2%\nVGG-16 [27] 85.6% 84.8%\nTable 2. Patch material classi\ufb01cation results. Mean class accuracy for different CNNs trained on MINC. See Section 5.1.\nSky 97.3% Food 90.4% Wallpaper 83.4% Glass 78.5%\nHair 95.3% Leather 88.2% Tile 82.7% Fabric 77.8%\nFoliage 95.1% Other 87.9% Ceramic 82.7% Metal 77.7%\nSkin 93.9% Pol. stone 85.8% Stone 82.7% Mirror 72.0%\nWater 93.6% Brick 85.1% Paper 81.8% Plastic 70.9%\nCarpet 91.6% Painted 84.2% Wood 81.3%\nTable 3. Patch test accuracy by category. CNN: GoogLeNet . See\nthe supplemental material for a full confusion matrix.\n5. Experiments and Results"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a405e-6db4-11ee-9478-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-24", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "Table 3. Patch test accuracy by category. CNN: GoogLeNet . See\nthe supplemental material for a full confusion matrix.\n5. Experiments and Results\n5.1. Patch material classi\ufb01cation\nIn this section, we evaluate the effect of many different\ndesign decisions for training methods for material classi\ufb01cation and segmentation, including various CNN architectures,\npatch sizes, and amounts of data.\nCNN Architectures. Our ultimate goal is full material segmentation, but we are also interested in exploring which\nCNN architectures give the best results for classifying single patches. Among the networks and parameter variations we tried we found the best performing networks were\nAlexNet [13],VGG-16 [27] and GoogLeNet [28].AlexNet\nandGoogLeNet are re-implementations by BVLC [ 11], and\nVGG-16 is con\ufb01guration D (a 16 layer network) of [ 27].\nAll models were obtained from the Caffe Model Zoo [ 11].\nOur experiments use AlexNet for evaluating material classi\ufb01cation design decisions and combinations of AlexNet and\nGoogLeNet for evaluating material segmentation. Tables 2\nand 3 summarize patch material classi\ufb01cation results on our\ndataset. Figure 10 shows correct and incorrect predictions\nmade with high con\ufb01dence."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a405f-6db4-11ee-aaaf-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-25", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "and 3 summarize patch material classi\ufb01cation results on our\ndataset. Figure 10 shows correct and incorrect predictions\nmade with high con\ufb01dence.\nInput patch scale. To classify a point in an image we must\ndecide how much context to include around it. The context,\nexpressed as a fraction of image size, is the patch scale. A\npriori, it is not clear which scale is best since small patches\nhave better spatial resolution, but large patches have more\n0.5e6 1e6 1.5e6 2e6 2.5e6\nNumber of training patches0.600.650.700.750.800.85Mean class accuracy (test set)66.1%74.8%77.9%79.3%80.2%80.9% 81.2% 81.4%Equal size\n79.6%Figure 6. Varying database size. Patch accuracy when trained on\nrandom subsets of MINC .Equal size is using equal samples per\ncategory (size determined by smallest category). CNN: AlexNet.\nPeak accuracy\nper category\nFigure 7. Accuracy vs patch scale by category . Dots: peak accuracy for each category; colored lines: sky,wallpaper ,mirror ; gray\nlines: other categories. CNN: AlexNet . While most materials are"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6679-6db4-11ee-b99c-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-26", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "lines: other categories. CNN: AlexNet . While most materials are\noptimally recognized at 23.3% or 32% patch scale, recognition of\nsky,wallpaper andmirror improve with increasing context.\ncontextual information. Holding patch centers \ufb01xed we varied scale and evaluated classi\ufb01cation accuracy with AlexNet .\nResults and a visualization of patch scales are shown in Figure 5. Scale 32% performs the best. Individual categories\nhad peaks at middle scale with some exceptions; we \ufb01nd\nthatmirror ,wallpaper andskyimprove with increasing context (Figure 7). We used 23.3% (which has nearly the same\naccuracy but higher spatial resolution) for our experiments.\nDataset size. To measure the effect of size on patch classi\ufb01cation accuracy we trained AlexNet with patches from\nrandomly sampled subsets of all 369,104 training images\nand tested on our full test set (Figure 6). As expected, using\nmore data improved performance. In addition, we still have\nnot saturated performance with 2.5 million training patches;\neven higher accuracies may be possible with more training\ndata (though with diminishing returns).\nDataset balance. Although we\u2019ve shown that more data is\nbetter we also \ufb01nd that a balanced dataset is more effective.\nWe trained AlexNet with all patches of our smallest category"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a667a-6db4-11ee-be0c-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-27", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "better we also \ufb01nd that a balanced dataset is more effective.\nWe trained AlexNet with all patches of our smallest category\n(wallpaper ) and randomly sampled the larger categories\n(the largest, wood , being 40x larger) to be equal size. We\nthen measured mean class accuracy on the same full test\nset. As shown in Figure 6, \u201cEqual size\u201d is more accurate\nthan a dataset of the same size and just 1.7% lower than\nthe full training set (which is 9x larger). This result further\ndemonstrates the value of building up datasets in a balanced\nmanner, focusing on expanding the smallest, least common\ncategories.\n6\nBrick\nWoodCarpet\nWaterWallpaperTileStoneSkySkinPolished stonePlasticPaperPainted\nOtherMirrorMetalLeatherHairGlassFoodFoliageFabricCeramicFigure 8. Full-scene material classi\ufb01cation examples: high-accuracy test set predictions by our method. CNN: GoogLeNet (with the\naverage pooling layer removed). Right: legend for material colors. See Table 4 for quantitative evaluation.\nInput image (a)Labels from CRF (b)Labels from CRF\n(test set) trained on segments trained on clicks"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a667b-6db4-11ee-9931-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-28", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "Input image (a)Labels from CRF (b)Labels from CRF\n(test set) trained on segments trained on clicks\nFigure 9. Optimizing for click accuracy leads to sloppy boundaries. In(a), we optimize for mean class accuracy across segments,\nresulting in high quality boundaries. In (b), we optimize for mean\nclass accuracy at click locations. Since the clicks are not necessarily close to object boundaries, there is no penalty for sloppy\nboundaries. CNN: GoogLeNet (without average pooling).\n5.2. Full scene material segmentation\nThe full test set for our patch dataset contains 41,801\nphotos, but most of them contain only a few labels. Since\nwe want to evaluate the per-pixel classi\ufb01cation performance,\nwe select a subset of 5,000 test photos such that each photo\ncontains a large number of segments and clicks, and small\ncategories are well sampled. We greedily solve for the best\nsuch set of photos. We similarly select 2,500 of 25,844\nvalidation photos. Our splits for all experiments are included\nonline with the dataset. To train the CRF for our model, we\ntry various parameter settings ( \u0012p,\u0012ab,\u0012L,wp) and select the\nmodel that performs best on the validation set. In total, we"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a667c-6db4-11ee-88a5-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-29", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "try various parameter settings ( \u0012p,\u0012ab,\u0012L,wp) and select the\nmodel that performs best on the validation set. In total, we\nevaluate 1799 combinations of CNNs and CRF parameters.\nSee the supplemental material for a detailed breakdown.\nWe evaluate multiple versions of GoogLeNet : both the\noriginal architecture and a version with the average pooling\nlayer (at the end) changed to 5x5, 3x3, and 1x1 (i.e. no\naverage pooling). We evaluate AlexNet trained at multiple\npatch scales (Figure 5). When using an AlexNet trained\nat a different scale, we keep the same scale for testing. We\nalso experiment with ensembles of GoogLeNet andAlexNet ,Architecture(a)Segments only (b)Clicks only\nClass Total Class Total\nAlexNet Scale: 11.6% 64.3% 72.6% 79.9% 77.2%\nAlexNet Scale: 23.3% 69.6% 76.6% 83.3% 81.1%\nAlexNet Scale: 32.0% 70.1% 77.1% 83.2% 80.7%\nAlexNet Scale: 46.5% 69.6% 75.4% 80.8% 77.7%"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a667d-6db4-11ee-b7b4-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-30", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "AlexNet Scale: 46.5% 69.6% 75.4% 80.8% 77.7%\nAlexNet Scale: 66.2% 67.7% 72.0% 77.2% 72.6%\nGoogLeNet 7x7 avg. pool 64.4% 71.6% 63.6% 63.4%\nGoogLeNet 5x5 avg. pool 67.6% 74.6% 70.9% 69.8%\nGoogLeNet 3x3 avg. pool 70.4% 77.7% 76.1% 74.7%\nGoogLeNet No avg. pool 70.4% 78.8% 79.1% 77.4%\nEnsemble 2 CNNs 73.1% 79.8% 84.5% 83.1%\nEnsemble 3 CNNs 73.1% 79.3% 85.9% 83.5%\nEnsemble 4 CNNs 72.1% 78.4% 85.8% 83.2%\nEnsemble 5 CNNs 71.7% 78.3% 85.5% 83.2%\nTable 4. Full scene material classi\ufb01cation results. Mean class\nand total accuracy on the test set. When training, we optimize the\nCRF parameters for mean class accuracy, but report both mean class"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a667e-6db4-11ee-844f-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-31", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "and total accuracy on the test set. When training, we optimize the\nCRF parameters for mean class accuracy, but report both mean class\nand total accuracy (mean accuracy across all examples). In one\nexperiment (a), we train and test only on segments; in a separate\nexperiment (b), we train and test only on clicks. Accuracies for\nsegments are averaged across all pixels that fall in that segment.\ncombined with either arithmetic or geometric mean.\nSince we have two types of data, clicks andsegments , we\nrun two sets of experiments: (a) we train and test only on\nsegments, and in a separate experiment (b) we train and test\nonly on clicks. These two training objectives result in very\ndifferent behavior, as illustrated in Figure 9. In experiment\n(a), the accuracy across segments are optimized, producing\nclean boundaries. In experiment (b), the CRF maximizes\naccuracy only at click locations, thus resulting in sloppy\nboundaries. As shown in Table 4, the numerical scores for\nthe two experiments are also very different: segments are\nmore challenging than clicks. While clicks are suf\ufb01cient to\ntrain a CNN, they are not suf\ufb01cient to train a CRF.\nFocusing on segmentation accuracy, we see from Table 4(a) that our best single model is GoogLeNet without"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a667f-6db4-11ee-b2f3-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-32", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "Focusing on segmentation accuracy, we see from Table 4(a) that our best single model is GoogLeNet without\naverage pooling (6% better than with pooling). The best\nensemble is 2 CNNs: GoogLeNet (no average pooling) and\nAlexNet (patch scale: 46.5%), combined with arithmetic\nmean. Larger ensembles perform worse since we are aver7\nCorrect\nFabric (99%) Foliage (99%) Food (99%) Leather (99%)Correct\nMetal (99%) Mirror (99%) Painted (99%) Plastic (99%)Incorrect\nT: Wood T: Polished stone T: Water T: Fabric\nP: Stone (90%) P: Water (35%) P: Carpet (27%) P: Foliage (67%)\nFigure 10. High con\ufb01dence predictions. Top two rows: correct\npredictions. Bottom row: incorrect predictions ( T: true, P: predicted). Percentages indicate con\ufb01dence (the predictions shown are\nat least this con\ufb01dent). CNN: GoogLeNet.\naging worse CNNs. In Figure 8, we show example labeling\nresults on test images.\n5.3. Comparing MINC to FMD\nCompared to FMD, the size and diversity of MINC is"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6680-6db4-11ee-b1d2-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-33", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "results on test images.\n5.3. Comparing MINC to FMD\nCompared to FMD, the size and diversity of MINC is\nvaluable for classifying real-world imagery. Table 5 shows\nthe effect of training on all of FMD and testing on MINC\n(and vice versa). The results suggests that training on FMD\nalone is not suf\ufb01cient for real-world classi\ufb01cation. Though it\nmay seem that our dataset is \u201ceasy,\u201d since the best classi\ufb01cations scores are lower for FMD than for MINC , we \ufb01nd that\ndif\ufb01culty is in fact closely tied to dataset size (Section 5.1).\nTaking 100 random samples per category, AlexNet achieves\n54.2\u00060.7% on MINC (64.6 \u00061.3% when considering only\nthe 10 FMD categories) and 66.5% on FMD.\n5.4. Comparing CNNs with prior methods\nCimpoi [ 3] is the best prior material classi\ufb01cation method\non FMD. We \ufb01nd that by replacing DeCAF with oversampled AlexNet features we can improve on their FMD results.\nWe then show that on MINC , a \ufb01netuned CNN is even better."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6681-6db4-11ee-9987-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-34", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "We then show that on MINC , a \ufb01netuned CNN is even better.\nTo improve on [ 3], we take their SIFT IFV, combine it\nwith AlexNet fc7 features, and add oversampling [ 13] (see\nsupplemental for details). With a linear SVM we achieve\n69.6\u00060.3% on FMD. Previous results are listed in Table 6.\nHaving found that SIFT IFV+fc7 is the new best on\nFMD, we compare it to a \ufb01netuned CNN on a subset of MINC (2500 patches per category, one patch per\nphoto). Fine-tuning AlexNet achieves 76.0\u00060.2% whereasTest\nFMD MINC\nTrainFMD 66.5% 26.1%\nMINC 41.7% 85.0%(10 categories\nin common)\nTable 5. Cross-dataset experiments. We train on one dataset and\ntest on another dataset. Since MINC contains 23 categories, we\nlimit MINC to the 10 categories in common. CNN: AlexNet.\nMethod Accuracy Trials\nSharan et al. [25] 57.1\u00060.6% 14 splits\nCimpoi et al. [3] 67.1\u00060.4% 14 splits\nFine-tuned AlexNet 66.5\u00061.5% 5 folds"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6682-6db4-11ee-9d02-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-35", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "Cimpoi et al. [3] 67.1\u00060.4% 14 splits\nFine-tuned AlexNet 66.5\u00061.5% 5 folds\nSIFT IFV+fc7 69.6\u00060.3% 10 splits\nTable 6. FMD experiments. By replacing DeCAF features with\noversampled AlexNet features we improve on the best FMD result.\nSIFT IFV+fc7 achieves 67.4 \u00060.5% with a linear SVM\n(oversampling, 5 splits). This experiment shows that\na \ufb01netuned CNN is a better method for MINC than\nSIFT IFV+fc7.\n6. Conclusion\nMaterial recognition is a long-standing, challenging problem. We introduce a new large, open, material database,\nMINC , that includes a diverse range of materials of everyday scenes and staged designed interiors, and is at least an\norder of magnitude larger than prior databases. Using this\nlarge database we conduct an evaluation of recent deep learning algorithms for simultaneous material classi\ufb01cation and\nsegmentation, and achieve results that surpass prior attempts\nat material recognition.\nSome lessons we have learned are:\n\u000fTraining on a dataset which includes the surrounding\ncontext is crucial for real-world material classi\ufb01cation."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6683-6db4-11ee-b8bc-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-36", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "at material recognition.\nSome lessons we have learned are:\n\u000fTraining on a dataset which includes the surrounding\ncontext is crucial for real-world material classi\ufb01cation.\n\u000fLabeled clicks are cheap and suf\ufb01cient to train a CNN\nalone. However, to obtain high quality segmentation\nresults, training a CRF on polygons results in much\nbetter boundaries than training on clicks.\nMany future avenues of work remain. Expanding the\ndataset to a broader range of categories will require new\nways to mine images that have more variety, and new annotation tasks that are cost-effective. Inspired by attributes\nfor textures [ 3], in the future we would like to identify material attributes and expand our database to include them. We\nalso believe that further exploration of joint material and object classi\ufb01cation and segmentation will be fruitful [ 10] and\nlead to improvements in both tasks. Our database, trained\nmodels, and all experimental results are available online at\nhttp://minc.cs.cornell.edu/.\nAcknowledgements. This work was supported in part by\nGoogle, Amazon AWS for Education, a NSERC PGS-D\nscholarship, the National Science Foundation (grants IIS1149393, IIS-1011919, IIS-1161645), and the Intel Science\nand Technology Center for Visual Computing.\n8\nReferences"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6684-6db4-11ee-bdc3-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-37", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "and Technology Center for Visual Computing.\n8\nReferences\n[1]S. Bell, P. Upchurch, N. Snavely, and K. Bala. OpenSurfaces:\nA richly annotated catalog of surface appearance. ACM Trans.\non Graphics (SIGGRAPH) , 32(4), 2013. 1, 2, 3\n[2]B. Caputo, E. Hayman, and P. Mallikarjuna. Class-speci\ufb01c\nmaterial categorisation. In ICCV , pages 1597\u20131604, 2005. 2\n[3]M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, and\nA. Vedaldi. Describing textures in the wild. In CVPR , pages\n3606\u20133613. IEEE, 2014. 2, 8\n[4]K. J. Dana, B. Van Ginneken, S. K. Nayar, and J. J. Koenderink. Re\ufb02ectance and texture of real-world surfaces. ACM\nTransactions on Graphics (TOG) , 18(1):1\u201334, 1999. 2\n[5]M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn,\nand A. Zisserman. The Pascal Visual Object Classes (VOC)"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6685-6db4-11ee-ab50-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-38", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "and A. Zisserman. The Pascal Visual Object Classes (VOC)\nChallenge. IJCV , 88(2):303\u2013338, June 2010. 3\n[6]C. Farabet, C. Couprie, L. Najman, and Y . LeCun. Learning\nhierarchical features for scene labeling. PAMI , 35(8):1915\u2013\n1929, 2013. 2\n[7]R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich\nfeature hierarchies for accurate object detection and semantic\nsegmentation. In CVPR , 2014. 2\n[8]S. Gould, R. Fulton, and D. Koller. Decomposing a scene\ninto geometric and semantically consistent regions. In ICCV ,\n2009. 3\n[9]E. Hayman, B. Caputo, M. Fritz, and J. olof Eklundh. On the\nsigni\ufb01cance of real-world conditions for material classi\ufb01cation. In ECCV , 2004. 2\n[10] D. Hu, L. Bo, and X. Ren. Toward robust material recognition\nfor everyday objects. In BMVC , pages 1\u201311. Citeseer, 2011.\n1, 2, 8"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6686-6db4-11ee-9783-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-39", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "for everyday objects. In BMVC , pages 1\u201311. Citeseer, 2011.\n1, 2, 8\n[11] Y . Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Caffe: Convolutional\narchitecture for fast feature embedding. In Proceedings of\nthe ACM International Conference on Multimedia , pages 675\u2013\n678. ACM, 2014. 6\n[12] P. Kr \u00a8ahenb \u00a8uhl and V . Koltun. Parameter learning and convergent inference for dense random \ufb01elds. In ICML , pages\n513\u2013521, 2013. 2, 4, 5\n[13] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet\nclassi\ufb01cation with deep convolutional neural networks. In\nAdvances in neural information processing systems , pages\n1097\u20131105, 2012. 2, 5, 6, 8\n[14] Y . LeCun, B. Boser, J. S. Denker, D. Henderson, R. E.\nHoward, W. Hubbard, and L. D. Jackel. Backpropagation"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6687-6db4-11ee-afb1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-40", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "Howard, W. Hubbard, and L. D. Jackel. Backpropagation\napplied to handwritten zip code recognition. Neural computation, 1(4):541\u2013551, 1989. 2\n[15] T. Leung and J. Malik. Representing and recognizing the visual appearance of materials using three-dimensional textons.\nIJCV , 43(1):29\u201344, June 2001. 2\n[16] W. Li and M. Fritz. Recognizing materials from virtual examples. In ECCV , pages 345\u2013358. Springer, 2012. 2\n[17] C. Liu, L. Sharan, E. H. Adelson, and R. Rosenholtz. Exploring features in a bayesian framework for material recognition.\nInCVPR , pages 239\u2013246. IEEE, 2010. 1, 2\n[18] M. Oquab, L. Bottou, I. Laptev, and J. Sivic. Learning and\ntransferring mid-level image representations using convolutional neural networks. In CVPR , 2014. 2[19] G. Patterson, C. Xu, H. Su, and J. Hays. The SUN Attribute\nDatabase: Beyond Categories for Deeper Scene Understanding. IJCV , 108(1-2):59\u201381, 2014. 1"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6688-6db4-11ee-9a06-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-41", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "Database: Beyond Categories for Deeper Scene Understanding. IJCV , 108(1-2):59\u201381, 2014. 1\n[20] X. Qi, R. Xiao, J. Guo, and L. Zhang. Pairwise rotation\ninvariant co-occurrence local binary pattern. In ECCV , pages\n158\u2013171. Springer, 2012. 1, 2\n[21] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma,\nZ. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg,\nand L. Fei-Fei. ImageNet Large Scale Visual Recognition\nChallenge, 2014. 1, 2\n[22] B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman. LabelMe: A database and web-based tool for image\nannotation. IJCV , 77(1-3):157\u2013173, May 2008. 3\n[23] G. Schwartz and K. Nishino. Visual material traits: Recognizing per-pixel material context. In Proceedings of the\nInternational Conference on Computer Vision Workshops (ICCVW) , pages 883\u2013890. IEEE, 2013. 2"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6689-6db4-11ee-8af9-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-42", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "International Conference on Computer Vision Workshops (ICCVW) , pages 883\u2013890. IEEE, 2013. 2\n[24] P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus,\nand Y . LeCun. Overfeat: Integrated recognition, localization\nand detection using convolutional networks. In International\nConference on Learning Representations (ICLR2014) . CBLS,\nApril 2014. 2, 5\n[25] L. Sharan, C. Liu, R. Rosenholtz, and E. Adelson. Recognizing materials using perceptually inspired features. IJCV ,\n2013. 1, 8\n[26] L. Sharan, R. Rosenholtz, and E. Adelson. Material perception: What can you see in a brief glance? Journal of Vision ,\n9(8):784\u2013784, 2009. 1, 2\n[27] K. Simonyan and A. Zisserman. Very deep convolutional\nnetworks for large-scale image recognition. arXiv preprint\narXiv:1409.1556 , 2014. 2, 6\n[28] C. Szegedy, W. Liu, Y . Jia, P. Sermanet, S. Reed, D. Anguelov,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a668a-6db4-11ee-93c0-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-43", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "[28] C. Szegedy, W. Liu, Y . Jia, P. Sermanet, S. Reed, D. Anguelov,\nD. Erhan, V . Vanhoucke, and A. Rabinovich. Going deeper\nwith convolutions. CVPR , 2015. 2, 6\n[29] R. Timofte and L. J. Van Gool. A training-free classi\ufb01cation\nframework for textures, writers, and materials. In BMVC ,\npages 1\u201312, 2012. 2\n[30] M. Varma and A. Zisserman. A statistical approach to texture\nclassi\ufb01cation from single images. IJCV , 62(1-2):61\u201381, Apr.\n2005. 2\n[31] J. Xiao, K. A. Ehinger, J. Hays, A. Torralba, and A. Oliva.\nSUN Database: Exploring a large collection of scene categories. IJCV , 2014. 1, 3\n[32] M. D. Zeiler and R. Fergus. Visualizing and understanding\nconvolutional networks. In ECCV , pages 818\u2013833. Springer,\n2014. 2\n[33] S. Zheng, M.-M. Cheng, J. Warrell, P. Sturgess, V . Vineet,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a668b-6db4-11ee-ba4d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.0623-44", "title": "Material Recognition in the Wild with the Materials in Context Database", "source": "http://arxiv.org/pdf/1412.0623", "start_index": 0, "chroma:document": "2014. 2\n[33] S. Zheng, M.-M. Cheng, J. Warrell, P. Sturgess, V . Vineet,\nC. Rother, and P. H. Torr. Dense semantic image segmentation\nwith objects and attributes. In CVPR , pages 3214\u20133221. IEEE,\n2014. 2\n[34] B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva.\nLearning deep features for scene recognition using Places\ndatabase. NIPS , 2014. 1, 3\n9"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a668c-6db4-11ee-8dbb-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-0", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "DeepEdge: A Multi-Scale Bifurcated Deep Network\nfor Top-Down Contour Detection\nGedas Bertasius\nUniversity of Pennsylvania\ngberta@seas.upenn.eduJianbo Shi\nUniversity of Pennsylvania\njshi@seas.upenn.eduLorenzo Torresani\nDartmouth College\nlt@dartmouth.edu\nAbstract\nContour detection has been a fundamental component\nin many image segmentation and object detection systems.\nMost previous work utilizes low-level features such as texture or saliency to detect contours and then use them as cues\nfor a higher-level task such as object detection. However,\nwe claim that recognizing objects and predicting contours\nare two mutually related tasks. Contrary to traditional approaches, we show that we can invert the commonly established pipeline: instead of detecting contours with low-level\ncues for a higher-level recognition task, we exploit objectrelated features as high-level cues for contour detection.\nWe achieve this goal by means of a multi-scale deep network that consists of \ufb01ve convolutional layers and a bifurcated fully-connected sub-network. The section from the input layer to the \ufb01fth convolutional layer is \ufb01xed and directly"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a668d-6db4-11ee-82d4-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-1", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "lifted from a pre-trained network optimized over a largescale object classi\ufb01cation task. This section of the network\nis applied to four different scales of the image input. These\nfour parallel and identical streams are then attached to\na bifurcated sub-network consisting of two independentlytrained branches. One branch learns to predict the contour likelihood (with a classi\ufb01cation objective) whereas the\nother branch is trained to learn the fraction of human labelers agreeing about the contour presence at a given point\n(with a regression criterion).\nWe show that without any feature engineering our multiscale deep learning approach achieves state-of-the-art results in contour detection.\n1. Introduction\nContour detection is typically considered a low-level\nproblem, and used to aid higher-level tasks such as object\ndetection [1, 3, 31, 24]. However, it can be argued that\nthe tasks of detecting objects and predicting contours are\nclosely related. For instance, given the contours we can\neasily infer which objects are present in the image. Con0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 100.10.20.30.40.50.60.70.80.91\nRecallPrecision\n  \n[F=.80] Human"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a668e-6db4-11ee-ac50-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-2", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "RecallPrecision\n  \n[F=.80] Human\n[F=.753] DeepEdge\n[F=.753] N4\u2212fields [10]\n[F=.747] MCG [2]\n[F=.746] SE [8]\n[F=.739] SCG [27]\n[F=.737] PMI [14]\n[F=.727] Sketch Tokens [19]\n[F=.726] gPb\u2212owt\u2212ucm [1]Figure 1: Contour detection accuracy on the BSDS500\ndataset. Our method attains higher average precision compared to prior methods and state-of-the-art F-score. At low\nrecall, DeepEdge achieves nearly 100% precision..\nversely, if we are given exact locations of the objects we\ncould predict contours just as easily. A commonly established pipeline in computer vision starts with with low-level\ncontour prediction and then moves up to higher-level object\ndetection. However, since we claim that these two tasks are\nmutually related, we propose to invert this process. Instead\nof using contours as low-level cues for object detection, we\nwant to use object-speci\ufb01c information as high-level cues\nfor contour detection. Thus, in a sense our scheme can be"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a668f-6db4-11ee-8f83-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-3", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "want to use object-speci\ufb01c information as high-level cues\nfor contour detection. Thus, in a sense our scheme can be\nviewed as a top-down approach where object-level cues inform the low-level contour detection process.\nIn this work, we present a uni\ufb01ed multi-scale deep learning approach that uses higher-level object information to\n1arXiv:1412.1123v3  [cs.CV]  23 Apr 2015\npredict contours. Speci\ufb01cally, we present a front-to-end\nconvolutional architecture where contours are learned directly from raw pixels. Our proposed deep learning architecture reuses features computed by the \ufb01rst \ufb01ve convolutional layers of the network of Krizhevsky et al. [17]. We refer to this network as the KNet . Because the KNet has been\ntrained for object classi\ufb01cation, reusing its features enable\nour method to incorporate object-level information for contour prediction. In the experimental section, we will show\nthat this high-level object information greatly improves contour detection results.\nFurthermore, our de\ufb01ned architecture operates on multiple scales simultaneously and combines local and global\ninformation from the image, which leads to signi\ufb01cantly\nimproved contour detection accuracy rates."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6690-6db4-11ee-af6f-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-4", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "information from the image, which leads to signi\ufb01cantly\nimproved contour detection accuracy rates.\nWe connect the features computed by the convolutional\nlayers of the KNet at four different scales of the input with\nalearned subnetwork that bifurcates into two branches (the\narchitecture of our model is illustrated in Fig. 2).\nWhat should the learning objective be? When a human\nobserver decides if a pixel is a boundary edge, a number\nof supporting evidence is used with object level reasoning.\nWhile it is impossible to record such information, we do\nhave the fraction of observers in agreement for each pixel.\nWe argue that a learning objective that predicts the fraction\nof human labelers in agreement can mimic human reasoning\nbetter.\nThus, in the bifurcated sub-network we optimize the two\nbranches with different learning objectives. The weights in\none branch are optimized with an edge classi\ufb01cation objective, while the other branch is trained to predict the fraction of human labelers in agreement, i.e., using a regression\ncriterion. We show that predictions from the classi\ufb01cation\nbranch yield high edge recall, while the outputs of the regression branch have high precision. Thus, fusing these two\noutputs allows us to obtain excellent results with respect to"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6691-6db4-11ee-acf9-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-5", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "branch yield high edge recall, while the outputs of the regression branch have high precision. Thus, fusing these two\noutputs allows us to obtain excellent results with respect to\nboth metrics and produce state-of-the-art F-score and average precision.\nIn summary, the use of higher-level object features, independent optimization of edge classi\ufb01cation and regression\nobjectives, as well as a uni\ufb01ed multi-scale architecture are\nthe key characteristics that allow our method to achieve the\nstate-of-the-art in contour detection (see Fig. 1).\n2. Related Work\nDeep Learning. In the recent years, deep convolutional\nnetworks have achieved remarkable results in a wide array\nof computer vision tasks [32, 25, 33, 17]. However, thus\nfar, applications of convolutional networks focused on highlevel vision tasks such as face recognition, image classi\ufb01cation, pose estimation or scene labeling [32, 25, 33, 17]. Excellent results in these tasks beg the question whether convolutional networks could perform equally well in lowerFigure 2: Visualization of multi-scale DeepEdge network\narchitecture. To extract candidate contour points, we run the\nCanny edge detector. Then, around each candidate point,\nwe extract patches at four different scales and simultaneously run them through the \ufb01ve convolutional layers of the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6692-6db4-11ee-8ea6-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-6", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "Canny edge detector. Then, around each candidate point,\nwe extract patches at four different scales and simultaneously run them through the \ufb01ve convolutional layers of the\nKNet [17]. We connect these convolutional layers to two\nseparately-trained network branches. The \ufb01rst branch is\ntrained for classi\ufb01cation, while the second branch is trained\nas a regressor. At testing time, the scalar outputs from these\ntwo sub-networks are averaged to produce the \ufb01nal score.\nlevel vision tasks such as contour detection. In this paper,\nwe present a convolutional architecture that achieves stateof-the-art results in a contour detection task, thus demonstrating that convolutional networks can be applied successfully for lower-level vision tasks as well.\nEdge Detection. Most of the contour detection methods\ncan be divided into two branches: local and global methods. Local methods perform contour detection by reasoning about small patches inside the image. Some recent\nlocal methods include sketch tokens [18] and structured\nedges [7], Both of these methods are trained in a supervised fashion using a random forest classi\ufb01er. Sketch tokens [18] pose contour detection as a multi-class classi\ufb01cation task and predicts a label for each of the pixels individually. Structured edges [7], on the other hand, attempt to"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6693-6db4-11ee-97f5-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-7", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "predict the labels of multiple pixels simultaneously.\nGlobal methods predict contours based on the information from the full image. Some of the most successful approaches in this genre are the MCG detector [2], gPb detector [1] and sparse code gradients [26]. While sparse\ncode gradients use supervised SVM learning [4], both gPb\nand MCG rely on some form of spectral methods. Other\nspectral-based methods include Normalized Cuts [30] and\nPMI [13].\nRecently, there have also been attempts to apply deep\nlearning methods to the task of contour detection. While\nSCT [20] is a sparse coding approach, both N4\ufb01elds [9]\nand DeepNet [16] use Convolutional Neural Networks\n(CNNs) to predict contours. N4\ufb01elds rely on dictionary\na) Input Image\n c)2ndlayer\n d)3rdlayer\n e)4thlayer\n e)5thlayer\nFigure 3: Visualization of the activation values at the selected convolutional \ufb01lters of the KNet (\ufb01lters are resized to the\noriginal image dimensions). The \ufb01lters in the second layer \ufb01re on oriented edges inside the image. The third and fourth"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6694-6db4-11ee-9ce9-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-8", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "original image dimensions). The \ufb01lters in the second layer \ufb01re on oriented edges inside the image. The third and fourth\nconvolutional layers produce an outline of the shape of the object. The \ufb01fth layer \ufb01res on the speci\ufb01c parts of the object.\nlearning and the use of the Nearest Neighbor algorithm\nwithin a CNN framework while DeepNet uses a traditional\nCNN architecture to predict contours.\nIn comparison to these prior approaches, our work offers\nseveral contributions. First, we de\ufb01ne a novel multi-scale\nbifurcated CNN architecture that enables our network to\nachieve state-of-the-art contour detection results. Second,\nwe avoid manual feature engineering by learning contours\ndirectly from raw data. Finally, we believe that we are the\n\ufb01rst to propose the use of high-level object features for contour detection, thus inverting the traditional pipeline relying on low-level cues for mid-level feature extraction. Our\nexperiments show that this top-down approach for contour\ndetection yields state-of-the-art results.\n3. The DeepEdge Network\nIn this section, we describe our proposed deep learning approach for contour detection. For simplicity, we \ufb01rst"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6695-6db4-11ee-b25d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-9", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "3. The DeepEdge Network\nIn this section, we describe our proposed deep learning approach for contour detection. For simplicity, we \ufb01rst\npresent our architecture in the single-scale scenario (subsection 3.1) and then discuss how to take advantage of multiple\nscales (subsection 3.2).\n3.1. Single-Scale Architecture\nSelection of Candidate Edge Points. To extract a set\nof candidate contours with high recall we apply the Canny\nedge detector [5] to the input image. For each of these\npoints we then extract a patch of \ufb01xed size such that our\ncandidate point is the center of the patch. Patches that do\nnot \ufb01t into the image boundaries are padded with the mirror\nre\ufb02ections of itself.\nExtraction of High-Level Features. We then resize the\npatch of \ufb01xed size to match the input dimensions of the\nKNet [17] and use this network to extract object-level features. The KNet is an appropriate model for our setting as it\nhas been trained over a large number of object classes (the\n1000 categories of the ImageNet dataset [28]) and thus captures features that are generic and useful for many object\ncategories. While such features have been optimized for the\ntask of object class recognition, they have been shown to"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6696-6db4-11ee-81a1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-10", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "categories. While such features have been optimized for the\ntask of object class recognition, they have been shown to\nbe highly effective for other image-analysis tasks, includ-ing object detection [10], attribute prediction [34], and image style recognition [15]. The network was trained on 1:2\nmillion images and it includes more than 60million parameters. Its architecture consists of 5convolutional layers and\n3fully connected layers. As we intend to use the KNet as\na feature extractor for boundary detection, we utilize only\nthe \ufb01rst 5convolutional layers, which preserve explicit location information before the \u201cspatial scrambling\u201d of the\nfully connection layers (note that a spatially variant representation is crucially necessary to predict the presence of\ncontours at individual pixels). The \ufb01rst two KNet convolutional layers learn low-level information. As we move\ninto the deeper layers, however, we observe that the network\nlearns higher-level object information. The second convolutional layer seems to encode coherent edge structures. The\nthird convolutional layer \ufb01res at locations corresponding to\nprototypical object shapes. The fourth layer appears to generate high responses for full shapes of the object, whereas\nthe \ufb01fth layer \ufb01res on the speci\ufb01c object parts (See Fig. 3)."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6697-6db4-11ee-89dd-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-11", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "the \ufb01fth layer \ufb01res on the speci\ufb01c object parts (See Fig. 3).\nIn order to obtain a representation that captures this hierarchical information, we perform feature extraction at each\nof the \ufb01ve convolutional layers, as shown in Fig. 5. Specifically, we consider a small sub-volume of the feature map\nstack produced at each layer. The sub-volume is centered\nat the center of the patch in order to assess the presence of\na contour in a small area around the candidate point. We\nthen perform max,average , and center pooling on this subvolume. This yields a feature descriptor of size 3\u0002Fwhere\nFis the number of feature maps computed by the convolutional layer. While max and average pooling are well established operations in deep learning, we de\ufb01ne center pooling\nas selecting the center-value from each of the feature maps.\nThe motivation for center pooling is that for each candidate\npoint we want to predict the contour presence at that particular point. Because the candidate point is located at the center of the input patch, center pooling extracts the activation\nvalue from the location that corresponds to our candidate\npoint location.\nA Bifurcated Sub-Network. We connect the feature\nmaps computed via pooling from the \ufb01ve convolutional"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6698-6db4-11ee-9216-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-12", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "point location.\nA Bifurcated Sub-Network. We connect the feature\nmaps computed via pooling from the \ufb01ve convolutional\nlayers to two separately-trained network branches. Each\nFigure 4: A few samples of ground truth data illustrating the difference between the classi\ufb01cation (\ufb01rst row) and the regression\n(second row) objectives. The classi\ufb01cation branch is trained to detect contours that are marked by at least one of the human\nannotators. Conversely, the regression branch is optimized to the contour values that depict the fraction of human annotators\nagreeing on the contour.\nbranch consists of two fully-connected layers. The \ufb01rst\nbranch is trained using binary labels, i.e., to perform contour classi\ufb01cation. This branch is making less selective predictions by classifying whether a given point is a contour or\nnot. In a sense, this classi\ufb01cation branch abstracts details\nrelated to the edge structure (orientation, strength, etc) and\nsimply tries to predict the presence/absence of an edge at\na given point. Due to such abstractions, the classi\ufb01cation\nbranch produces contour predictions with high recall.\nThe second branch is optimized as a regressor to predict"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6699-6db4-11ee-91b1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-13", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "a given point. Due to such abstractions, the classi\ufb01cation\nbranch produces contour predictions with high recall.\nThe second branch is optimized as a regressor to predict\nthe fraction of human labelers agreeing about the contour\npresence at a particular point. Due to a regression objective,\nthis branch is much more selective than the \ufb01rst branch. Intuitively, the second branch is trained to learn the structural\ndifferences between the contours that are marked by a different fraction of human labelers. For instance, the area that\nwas labeled as a contour by 80% of human labelers must\nbe signi\ufb01cantly different than the area that was labeled as\na contour by 20% human labelers. The regression branch\nis trying to learn such differences by predicting the fraction of human labelers who would mark a particular point\nas a contour. Thus, in a sense, we are training the regression branch to implicitly mimic how human labelers reason about the contour presence at a given point. In the experimental section, we demonstrate that due to its selectivity, the regression branch produces contour predictions with\nvery high precision. In Fig. 4, we present several samples\nof ground truth data that illustrate the different properties of\nour two end-objectives.\nThe number of hidden layers in the \ufb01rst and second fully"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a669a-6db4-11ee-97bc-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-14", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "of ground truth data that illustrate the different properties of\nour two end-objectives.\nThe number of hidden layers in the \ufb01rst and second fully\nconnected layers of both branches are 1024 and512, respectively. Both branches optimize the sum of squared difference loss over the (binary or continuous) labels. At test-ing time, the scalar outputs computed from these two subnetworks are averaged to produce a \ufb01nal score indicative of\nthe probability that the candidate point is a contour. Visualization of this architecture is presented in Fig. 5.\nIn order to train our sub-network, we generate patch examples and labels using training images with ground truth\nannotations from multiple human labelers. To generate the\nbinary labels, we \ufb01rst sample 40;000positive examples that\nwere marked as contours by at least one of the labelers. To\ngenerate negative examples we consider the points that were\nselected as candidate contour points by the Canny edge detector but that have not been marked as contours by any of\nthe human labelers. These are essentially false positives.\nFor training, we use a random subset of 40;000 of such\npoints in order to have equal proportion of negative and\npositive examples. These 80;000examples are then used\nto train our classi\ufb01cation sub-network."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a669b-6db4-11ee-96be-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-15", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "points in order to have equal proportion of negative and\npositive examples. These 80;000examples are then used\nto train our classi\ufb01cation sub-network.\nIn addition to the binary labels, we also generate continuous labels that are used to train the regression network.\nFor this purpose, we de\ufb01ne the regression label of a point to\nbe the fraction of human labelers that marked the point as a\ncontour. These 80;000examples with continuous labels are\nthen also used to train our regression sub-network.\n3.2. Multi-Scale Architecture\nIn the previous section we presented a convolutional architecture for contour prediction utilizing a single scale.\nHowever, in practice, we found that a multi-scale approach\nworks much better. In this section, we show how to modify\nthe single-scale architecture so that it can exploit multiple\nscales simultaneously.\nRather than extracting a patch at a single scale as we did\nin the previous section, in a multi-scale setting we extract\n|{z}\n|{z}Fixed WeightsLearned WeightsClassi\ufb01cation Branch\nRegression Branch\n}\n}Figure 5: Detailed illustration of our proposed architecture in a single-scale setting. First, an input patch, centered around the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a669c-6db4-11ee-b769-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-16", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "Regression Branch\n}\n}Figure 5: Detailed illustration of our proposed architecture in a single-scale setting. First, an input patch, centered around the\ncandidate point, goes through \ufb01ve convolutional layers of the KNet . To extract high-level features, at each convolutional layer\nwe extract a small sub-volume of the feature map around the center point, and perform max,average , and center pooling on\nthis sub-volume. The pooled values feed a bifurcated sub-network. At testing time, the scalar outputs computed from the\nbranches of a bifurcated sub-networks are averaged to produce a \ufb01nal contour prediction.\npatches around the candidate point for different patch sizes\nso that they cover different spatial extents of the image. We\nthen resize the patches to \ufb01t the KNet input and pump them\nin parallel through the \ufb01ve convolutional layers. Our highlevel features are then built by performing max, average and\ncenter pooling in a small sub-volume of the feature map at\neach convolutional layer and at each scale. This effectively\nincreases the dimensionality of the feature vector by a factor\nequal to the number of scales compared to the single-scale\nsetting. These pooled features are then connected as before"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a669d-6db4-11ee-9be1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-17", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "increases the dimensionality of the feature vector by a factor\nequal to the number of scales compared to the single-scale\nsetting. These pooled features are then connected as before\nto the two separately-trained network branches. A visualization of our multi-scale architecture is shown in Fig. 2.\n3.3. Implementation Details\nIn this section, we describe additional implementation\ndetails of our model. Our deep network is implemented using the software library Caffe [14].\nWe use four different scales for our patches. The sizes\nof these patches are 64\u000264;128\u0002128;196\u0002196and a\nfull-sized image. All of the patches are then resized to the\nKNet input dimensions of 227\u0002227.\nWhen extracting high-level features from the convolutional layers of KNet , we use sub-volumes of convolutionalfeature maps having spatial sizes 7\u00027;5\u00025;3\u00023;3\u00023,\nand3\u00023for the convolutional layers 1;2;3;4;5, respectively. Note that we shrink the size of the subvolume as we\ngo deeper in the network since the feature maps get smaller\ndue to pooling. Our choice of subvolume sizes is made to\nensure we are roughly considering the same spatial extent\nof the original image at each layer.\nAs illustrated in Fig. 5, during the training the weights"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a669e-6db4-11ee-a198-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-18", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "ensure we are roughly considering the same spatial extent\nof the original image at each layer.\nAs illustrated in Fig. 5, during the training the weights\nin the convolutional layers are \ufb01xed and only the weights in\nthe fully connected layers of the two branches are learned.\nTo train our model we use the learning rate of 0:1, the\ndropout fraction of 0:5,50number of epochs, and the size\nof the batch equal to 100.\nAs described earlier, to train classi\ufb01cation and regression branches we sample 80;000examples with binary labels. We also generate continuous labels for these 80;000\nexamples. In addition, we sample a hold-out dataset of size\n40;000. This hold-out dataset is used for the hard-positive\nmining step [22].\nFor the \ufb01rst 25epochs we train classi\ufb01cation and regression branches independently on the original 80;000sample training dataset. After the \ufb01rst 25epochs, we test both\nbranches on the hold-out dataset and detect false negative\npredictions made by each branch. We then use these false\nnegative examples along with the same number of randomly\nselected true negative examples to augment our original\n80;000training dataset. For the remaining 25epochs, we\ntrain both branches on this augmented dataset."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a669f-6db4-11ee-9093-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-19", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "selected true negative examples to augment our original\n80;000training dataset. For the remaining 25epochs, we\ntrain both branches on this augmented dataset.\nThe motivation for the hard-positive mining step is to reduce the number of false negative predictions produced by\nboth branches. By augmenting the original 80;000sized\ntraining data with false negative examples, we are forcing\nboth branches to focus on hard positive examples, and thus,\neffectively reducing the number of false negative predictions.\n4. Experiments\nIn this section, we present our results on the BSDS500\ndataset [23], which is arguably the most established benchmark for contour detection. This dataset contains 200training images, 100 validation images, and 200 test images.\nContour detection accuracy is evaluated using three standard measures: \ufb01xed contour threshold (ODS), per-image\nbest threshold (OIS), and average precision (AP).\nIn section 4.1 we quantitatively compare our approach\nto the state-of-the-art. In sections 4.2-4.5 we study how the\nperformance of our system changes as we modify some of\nthe architecture choices (number of scales, feature maps,\npooling scheme, training objective). This will cast additional insight into the factors that critically contribute to the\nhigh accuracy of our system.\n4.1. Comparison with Other Methods"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66a0-6db4-11ee-a8f1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-20", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "pooling scheme, training objective). This will cast additional insight into the factors that critically contribute to the\nhigh accuracy of our system.\n4.1. Comparison with Other Methods\nWe compare the results produced by our approach and\npreviously proposed contour detection methods. Table 1\nsummarizes the results. We note that our algorithm achieves\ncontour detection accuracy that is higher or equal to stateof-the-art results according to two of the three metrics.\nFig. 1 shows the precision and recall curve for the methods considered in our comparison. It also lists the F-score\nfor each method (in the legend). We observe that there is the\naccuracy margin separating our approach from prior techniques. In particular, for low-recall our method achieves almost perfect precision rate. It also produces state-of-the-art\nF-score.\n4.2. Single Scale versus Multiple Scales\nIn this section we study the bene\ufb01ts of a multi-scale architecture. Results in Table 2 report accuracy for different numbers and choices of scales. The \ufb01rst four rows in\nthe table illustrate the results achieved using a single-scale\napproach. Speci\ufb01cally, these four cases show performance\nobtained when training and testing our system with an input"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66a1-6db4-11ee-babd-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-21", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "the table illustrate the results achieved using a single-scale\napproach. Speci\ufb01cally, these four cases show performance\nobtained when training and testing our system with an input\npatch of size 64\u000264;128\u0002128;196\u0002196or a full-sized image, respectively. Note that adding information from multiple scales leads to signi\ufb01cantly higher F-scores and higherMethod ODS OIS AP\nFelz., Hutt. [8] 0.610 0.640 0.560\nMean Shift [6] 0.640 0.680 0.560\nNcuts [30] 0.640 0.680 0.450\nSCT [20] 0.710 0.720 0.740\ngPb-owt-ucm [1] 0.726 0.757 0.696\nSketch Tokens [18] 0.727 0.746 0.780\nPMI [13] 0.737 0.771 0.783\nDeepNet [16] 0.738 0.759 0.758\nSCG [26] 0.739 0.758 0.773\nSE [7] 0.746 0.767 0.803\nMCG [2] 0.747 0.779 0.759\nN4-\ufb01elds [9] 0.753 0.769 0.784"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66a2-6db4-11ee-a9af-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-22", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "MCG [2] 0.747 0.779 0.759\nN4-\ufb01elds [9] 0.753 0.769 0.784\nDeepEdge 0.753 0.772 0.807\nTable 1: Edge detection results on the BSDS500 benchmark. Our DeepEdge method achieves state-of-the-art contour detections results according to both F-score and AP\nmetrics.\nScale ODS OIS AP\n64 0.71 0.73 0.76\n128 0.72 0.74 0.78\n196 0.71 0.73 0.76\nFull Image 0.67 0.69 0.57\n64, 128 0.72 0.75 0.78\n64, 128,196 0.72 0.75 0.78\n64,128,196,Full Image 0.75 0.77 0.81\nTable 2: Results illustrating the effect of using a multi-scale\narchitecture. Considering multiple scales for contour detection yields signi\ufb01cantly higher accuracy relative to a single\nscale approach.\naverage precisions. Thus, these results suggest that a multiscale approach is highly advantageous in comparison to a\nsingle scale setting.\n4.3. Advantages of Higher-Level Features\nIn this section, we examine the validity of our earlier"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66a3-6db4-11ee-8602-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-23", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "single scale setting.\n4.3. Advantages of Higher-Level Features\nIn this section, we examine the validity of our earlier\nclaim that higher-level object-features enhance contour detection accuracy. In Table 3, we present individual contour\ndetection results using features from the different convolutional layers of KNet . Note that the 4thconvolutional layer\nproduces the most effective features when considering one\nlayer at a time. From our earlier discussion we know that\nthe4thconvolutional layer encodes higher-level object information related to shape and speci\ufb01c object parts. This\nindicates that object speci\ufb01c cues are particularly bene\ufb01cial\nfor contour detection accuracy.\nWe also observe that by incorporating features from all\nInput Image\n Canny Edges\n Raw DeepEdges\n Thresholded DeepEdges\n Ground Truth Edges\nFigure 6: Qualitative results produced by our method. Notice how our method learns to distinguish between strong and\nweak contours. For instance, in the last row of predictions, contours corresponding to zebra stripes are assigned much lower\nprobabilities than contours that correspond to the actual object boundaries separating the zebras from the background.\nConv. Layers ODS OIS AP\n1st0.66 0.68 0.69\n2nd0.71 0.74 0.76"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66a4-6db4-11ee-b1fe-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-24", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "Conv. Layers ODS OIS AP\n1st0.66 0.68 0.69\n2nd0.71 0.74 0.76\n3rd0.74 0.75 0.79\n4th0.74 0.76 0.79\n5th0.73 0.74 0.77\nAll 0.75 0.77 0.81\nTable 3: This table shows the advantage of using higherlevel features from the KNet convolutional layers. Individually, the 4thconvolutional layer produces the best contour\nprediction results, which implies that higher-level object information is indeed bene\ufb01cial for contour detection. Combining the features from all convolutional layers leads to\nstate-of-the-art results.\nthe convolutional layers, our method achieves state-of-theart contour detection results. This suggests that the features\ncomputed by different layers are complementary and that\nconsidering information from the entire hierarchy is advantageous.4.4. Pooling Scheme\nWhen presenting the architecture of our model, we discussed three different types of pooling: max,average , and\ncenter pooling. These three techniques were used to pool\nthe values from the sub-volumes extracted around the center point in each convolutional \ufb01lter as illustrated in Fig. 5.\nWe now show how each type of pooling affects contour"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66a5-6db4-11ee-a46c-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-25", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "We now show how each type of pooling affects contour\ndetection results. Table 4 illustrates that, individually, center pooling yields the best contour detection results. This is\nexpected because the candidate point for which we are trying to predict a contour probability is located at the center\nof the input patch.\nHowever, we note that combining all three types of pooling, achieves better contour detection results than any single\npooling technique alone.\n4.5. Bifurcation and Training Objective\nNext, we want to show that that the two independentlytrained classi\ufb01cation and regression branches in the bifurcated sub-network provide complementary information that\nyields improved contour detection accuracy. In Table 5, we\npresent contour detection results achieved by using predictions from the individual branches of the bifurcated subnetwork.\nPooling Type ODS OIS AP\nAverage 0.73 0.75 0.78\nMax 0.69 0.72 0.73\nCenter 0.74 0.76 0.8\nAvg+Max+Cen 0.75 0.77 0.81\nTable 4: Effect of different pooling schemes on contour detection results. Center pooling produces better results than\nmax or average pooling. Combining all three types of pooling further improves the results.\nBranch ODS OIS AP"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66a6-6db4-11ee-806c-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-26", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "max or average pooling. Combining all three types of pooling further improves the results.\nBranch ODS OIS AP\nClassi\ufb01cation 0.75 0.76 0.78\nRegression 0.74 0.76 0.80\nClassi\ufb01cation+Regression 0.75 0.77 0.81\nTable 5: Contour detection accuracy of the two branches\nin our bifurcated sub-network. The classi\ufb01cation branch\nyields solid F-score results whereas the regression branch\nachieves high average precision. Averaging the outputs\nfrom these two branches further improve the results.\nFrom these results, we observe that using predictions\njust from the classi\ufb01cation branch produces high F-score\nwhereas using predictions only from the regression branch\nyields high average precision. Combining the predictions\nfrom both branches improves the results according to both\nmetrics thus, supporting our claim that separately optimizing edge classi\ufb01cation and regression objectives is bene\ufb01cial to contour detection.\n4.6. Qualitative Results\nFinally, we present qualitative results produced by our\nmethod. In Figure 6 we show for each input image example, the set of candidate points produced by the Canny edge\ndetector, the un-thresholded predictions of our method, the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66a7-6db4-11ee-b0ac-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-27", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "method. In Figure 6 we show for each input image example, the set of candidate points produced by the Canny edge\ndetector, the un-thresholded predictions of our method, the\nthresholded predictions, and the ground truth contour map\ncomputed as an average of the multiple manual annotations.\nTo generate the thresholded predictions, we use a probability threshold of 0:5.\nNote that our method successfully distinguishes between\nstrong and weak contours. Speci\ufb01cally, observe that in the\nlast row of Figure 6, our method assigns lower probability\nto contours corresponding to zebra stripes compared to the\ncontours of the actual object boundary separating the zebras\nfrom the background. Thus, in the thresholded version of\nthe prediction, the weak contours inside the zebra bodies\nare removed and we obtain contour predictions that look\nvery similar to the ground truth.\nDue to locality of our method, it may be bene\ufb01cial\nto apply spectral methods [21, 30] or conditional random\ufb01elds [27] on top of our method to further improve its performance.\n4.7. Computational Cost\nIn its current form, our method requires about 60KKNet\nevaluations ( 15Kper scale) to extract the features. Based\non the runtimes reported in [14], if executed on a GPU our"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66a8-6db4-11ee-a027-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-28", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "evaluations ( 15Kper scale) to extract the features. Based\non the runtimes reported in [14], if executed on a GPU our\nmethod would take about 5 minutes and could be made\nfaster using the approach described in [12].\nAn alternative way to dramatically reduce the runtime of\nDeepEdge is to interpolate the entries of the feature maps\nproduced by applying the KNet to the full image rather than\nto individual patches. Such an approach would reduce the\nnumber of CNN evaluations needed from 60Kto4(one\nfor each scale), which would allow our method to run in\nreal time even on CPUs. We note that interpolation of features in deep layers has been used successfully in several\nrecent vision papers [29, 11, 19]. Thus, we believe that such\nan approach could yield nearly equivalent contour detection\naccuracy, up to a small possible degradation caused by interpolation.\nSince in this work we were primarily interested in studying the effective advantage enabled by object-level features\nin contour detection, we have not invested any effort in optimizing the implementation of our method. This will be one\nof our immediate goals in the future.\n5. Conclusions\nIn this work, we presented a multi-scale bifurcated deep\nnetwork for top-down contour detection. In the past, contour detection has been approached as a bottom-up task"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66a9-6db4-11ee-b2eb-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-29", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "network for top-down contour detection. In the past, contour detection has been approached as a bottom-up task\nwhere low-level features are engineered \ufb01rst, then contour\ndetection is performed, and \ufb01nally contours may be used\nas cues for object detection. However, due to a close relationship between object and contour detection tasks, we\nproposed to invert this pipeline and perform contour detection in a top-down fashion. We demonstrated how to use\nhigher-level object cues to predict contours and showed that\nconsidering higher-level object-features leads to a substantial gain in contour detection accuracy.\nAdditionally, we demonstrated that our multi-scale architecture is bene\ufb01cial to contour prediction as well. By\nconsidering multiple scales, our method incorporates local\nand global information around the candidate contour points,\nwhich leads to signi\ufb01cantly better contour detection results.\nFurthermore, we showed that independent optimization of\ncontour classi\ufb01cation and regression objectives improves\ncontour prediction accuracy as well. As our experiments indicate, DeepEdge achieves higher average precision results\ncompared to any prior or concurrent work.\nIn conclusion, our results suggest that pure CNN systems can be applied successfully to contour detection and\npossibly to many other low-level vision tasks.\n6. Acknowledgements"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66aa-6db4-11ee-8c88-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-30", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "In conclusion, our results suggest that pure CNN systems can be applied successfully to contour detection and\npossibly to many other low-level vision tasks.\n6. Acknowledgements\nWe thank Piotr Teterwak, Du Tran and Mohammad Haris\nBaig for helpful discussions. This research was funded in\npart by NSF award CNS-1205521.\nReferences\n[1] P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik. Contour\ndetection and hierarchical image segmentation. IEEE Trans.\nPattern Anal. Mach. Intell. , 33(5):898\u2013916, May 2011. 1, 2,\n6\n[2] P. Arbelaez, J. Pont-Tuset, J. Barron, F. Marqu \u00b4es, and J. Malik. Multiscale combinatorial grouping. In Computer Vision\nand Pattern Recognition (CVPR) , 2014. 2, 6\n[3] E. Borenstein. Combining top-down and bottom-up segmentation. In In Proceedings IEEE workshop on Perceptual Organization in Computer Vision, CVPR , page 46, 2004. 1\n[4] C. J. C. Burges. A tutorial on support vector machines for\npattern recognition. Data Mining and Knowledge Discovery ,\n2:121\u2013167, 1998. 2"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66ab-6db4-11ee-bc62-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-31", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "[4] C. J. C. Burges. A tutorial on support vector machines for\npattern recognition. Data Mining and Knowledge Discovery ,\n2:121\u2013167, 1998. 2\n[5] J. Canny. A computational approach to edge detection. IEEE\nTrans. Pattern Anal. Mach. Intell. , 8(6):679\u2013698, June 1986.\n3\n[6] D. Comaniciu, P. Meer, and S. Member. Mean shift: A robust\napproach toward feature space analysis. IEEE Transactions\non Pattern Analysis and Machine Intelligence , 24:603\u2013619,\n2002. 6\n[7] P. Doll \u00b4ar and C. L. Zitnick. Fast edge detection using structured forests. PAMI , 2015. 2, 6\n[8] P. F. Felzenszwalb and D. P. Huttenlocher. Ef\ufb01cient\ngraph-based image segmentation. Int. J. Comput. Vision ,\n59(2):167\u2013181, Sept. 2004. 6\n[9] Y . Ganin and V . S. Lempitsky. N4-\ufb01elds: Neural network\nnearest neighbor \ufb01elds for image transforms. ACCV , 2014.\n2, 6"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66ac-6db4-11ee-856e-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-32", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "nearest neighbor \ufb01elds for image transforms. ACCV , 2014.\n2, 6\n[10] R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature hierarchies for accurate object detection and semantic\nsegmentation. In Proceedings of the IEEE Conference on\nComputer Vision and Pattern Recognition (CVPR) , 2014. 3\n[11] B. Hariharan, P. A. Arbel \u00b4aez, R. B. Girshick, and J. Malik.\nHypercolumns for object segmentation and \ufb01ne-grained localization. CoRR , abs/1411.5752, 2014. 8\n[12] F. N. Iandola, M. W. Moskewicz, S. Karayev, R. B. Girshick,\nT. Darrell, and K. Keutzer. Densenet: Implementing ef\ufb01cient\nconvnet descriptor pyramids. CoRR , abs/1404.1869, 2014. 8\n[13] P. Isola, D. Zoran, D. Krishnan, and E. H. Adelson. Crisp\nboundary detection using pointwise mutual information. In\nECCV , 2014. 2, 6"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66ad-6db4-11ee-9b7e-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-33", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "boundary detection using pointwise mutual information. In\nECCV , 2014. 2, 6\n[14] Y . Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. arXiv preprint\narXiv:1408.5093 , 2014. 5, 8\n[15] S. Karayev, A. Hertzmann, H. Winnemoeller, A. Agarwala, and T. Darrell. Recognizing image style. CoRR ,\nabs/1311.3715, 2013. 3[16] J. J. Kivinen, C. K. Williams, N. Heess, and D. Technologies. Visual boundary prediction: A deep neural prediction\nnetwork and quality dissection. AISTATS , 1(2):9, 2014. 2, 6\n[17] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet\nclassi\ufb01cation with deep convolutional neural networks. In\nF. Pereira, C. Burges, L. Bottou, and K. Weinberger, editors, Advances in Neural Information Processing Systems 25 ,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66ae-6db4-11ee-9e34-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-34", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "F. Pereira, C. Burges, L. Bottou, and K. Weinberger, editors, Advances in Neural Information Processing Systems 25 ,\npages 1097\u20131105. Curran Associates, Inc., 2012. 2, 3\n[18] J. Lim, C. L. Zitnick, and P. Doll \u00b4ar. Sketch tokens: A learned\nmid-level representation for contour and object detection. In\nCVPR , 2013. 2, 6\n[19] J. Long, E. Shelhamer, and T. Darrell. Fully convolutional\nnetworks for semantic segmentation. CoRR , abs/1411.4038,\n2014. 8\n[20] M. Maire, S. X. Yu, and P. Perona. Reconstructive sparse\ncode transfer for contour detection and semantic labeling. In\nAsian Conference on Computer Vision (ACCV) , 2014. 2, 6\n[21] J. Malik, S. Belongie, T. Leung, and J. Shi. Contour and texture analysis for image segmentation. Int. J. Comput. Vision ,\n43(1):7\u201327, June 2001. 8\n[22] T. Malisiewicz, A. Gupta, and A. A. Efros. Ensemble of\nexemplar-svms for object detection and beyond. In ICCV ,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66af-6db4-11ee-ab22-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-35", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "exemplar-svms for object detection and beyond. In ICCV ,\n2011. 5\n[23] D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database\nof human segmented natural images and its application to\nevaluating segmentation algorithms and measuring ecological statistics. In Proc. 8th Int\u2019l Conf. Computer Vision , volume 2, pages 416\u2013423, July 2001. 6\n[24] A. Opelt and A. Zisserman. A boundary-fragment-model for\nobject detection. In In ECCV , pages 575\u2013588, 2006. 1\n[25] P. H. O. Pinheiro and R. Collobert. Recurrent convolutional\nneural networks for scene parsing. CoRR , abs/1306.2795,\n2013. 2\n[26] X. Ren and L. Bo. Discriminatively Trained Sparse Code\nGradients for Contour Detection. In Advances in Neural Information Processing Systems , December 2012. 2, 6\n[27] X. Ren, C. C. Fowlkes, and J. Malik. Scale-invariant contour\ncompletion using condition random \ufb01elds. Technical report.\n8"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66b0-6db4-11ee-9c80-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-36", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "completion using condition random \ufb01elds. Technical report.\n8\n[28] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh,\nS. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein,\nA. C. Berg, and L. Fei-Fei. ImageNet Large Scale Visual\nRecognition Challenge, 2014. 3\n[29] P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus,\nand Y . LeCun. Overfeat: Integrated recognition, localization and detection using convolutional networks. CoRR ,\nabs/1312.6229, 2013. 8\n[30] J. Shi and J. Malik. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine\nIntelligence , 22:888\u2013905, 1997. 2, 6, 8\n[31] J. Shotton. Contour-based learning for object detection. In\nIn Proc. ICCV , pages 503\u2013510, 2005. 1\n[32] Y . Taigman, M. Yang, M. Ranzato, and L. Wolf. Deepface:"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66b1-6db4-11ee-9b59-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.1123-37", "title": "DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection", "source": "http://arxiv.org/pdf/1412.1123", "start_index": 0, "chroma:document": "[32] Y . Taigman, M. Yang, M. Ranzato, and L. Wolf. Deepface:\nClosing the gap to human-level performance in face veri\ufb01cation. In Conference on Computer Vision and Pattern Recognition (CVPR) , 2014. 2\n[33] A. Toshev and C. Szegedy. Deeppose: Human pose estimation via deep neural networks. CoRR , abs/1312.4659, 2013.\n2\n[34] N. Zhang, M. Paluri, M. Ranzato, T. Darrell, and L. Bourdev.\nPanda: Pose aligned networks for deep attribute modeling.\nCoRR , abs/1311.5591, 2013. 3"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66b2-6db4-11ee-8c2e-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-0", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "Published as a conference paper at ICLR 2015\nSEMANTIC IMAGE SEGMENTATION WITH DEEP CONVOLUTIONAL NETS AND FULLY CONNECTED CRF S\nLiang-Chieh Chen\nUniv. of California, Los Angeles\nlcchen@cs.ucla.edu\nGeorge Papandreou\u0003\nGoogle Inc.\ngpapan@google.com\nIasonas Kokkinos\nCentraleSup \u00b4elec and INRIA\niasonas.kokkinos@ecp.fr\nKevin Murphy\nGoogle Inc.\nkpmurphy@google.com\nAlan L. Yuille\nUniv. of California, Los Angeles\nyuille@stat.ucla.edu\nABSTRACT\nDeep Convolutional Neural Networks (DCNNs) have recently shown state of the\nart performance in high level vision tasks, such as image classi\ufb01cation and object detection. This work brings together methods from DCNNs and probabilistic\ngraphical models for addressing the task of pixel-level classi\ufb01cation (also called\n\u201dsemantic image segmentation\u201d). We show that responses at the \ufb01nal layer of\nDCNNs are not suf\ufb01ciently localized for accurate object segmentation. This is"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66b3-6db4-11ee-8137-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-1", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "DCNNs are not suf\ufb01ciently localized for accurate object segmentation. This is\ndue to the very invariance properties that make DCNNs good for high level tasks.\nWe overcome this poor localization property of deep networks by combining the\nresponses at the \ufb01nal DCNN layer with a fully connected Conditional Random\nField (CRF). Qualitatively, our \u201cDeepLab\u201d system is able to localize segment\nboundaries at a level of accuracy which is beyond previous methods. Quantitatively, our method sets the new state-of-art at the PASCAL VOC-2012 semantic\nimage segmentation task, reaching 71.6% IOU accuracy in the test set. We show\nhow these results can be obtained ef\ufb01ciently: Careful network re-purposing and a\nnovel application of the \u2019hole\u2019 algorithm from the wavelet community allow dense\ncomputation of neural net responses at 8 frames per second on a modern GPU.\n1 I NTRODUCTION\nDeep Convolutional Neural Networks (DCNNs) had been the method of choice for document recognition since LeCun et al. (1998), but have only recently become the mainstream of high-level vision"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66b4-6db4-11ee-bbac-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-2", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "research. Over the past two years DCNNs have pushed the performance of computer vision systems to soaring heights on a broad array of high-level problems, including image classi\ufb01cation\n(Krizhevsky et al., 2013; Sermanet et al., 2013; Simonyan & Zisserman, 2014; Szegedy et al., 2014;\n\u0003Work initiated when G.P. was with the Toyota Technological Institute at Chicago. The \ufb01rst two authors\ncontributed equally to this work.\n1arXiv:1412.7062v4  [cs.CV]  7 Jun 2016\nPublished as a conference paper at ICLR 2015\nPapandreou et al., 2014), object detection (Girshick et al., 2014), \ufb01ne-grained categorization (Zhang\net al., 2014), among others. A common theme in these works is that DCNNs trained in an end-to-end\nmanner deliver strikingly better results than systems relying on carefully engineered representations,\nsuch as SIFT or HOG features. This success can be partially attributed to the built-in invariance of\nDCNNs to local image transformations, which underpins their ability to learn hierarchical abstractions of data (Zeiler & Fergus, 2014). While this invariance is clearly desirable for high-level vision"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66b5-6db4-11ee-996f-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-3", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "tasks, it can hamper low-level tasks, such as pose estimation (Chen & Yuille, 2014; Tompson et al.,\n2014) and semantic segmentation - where we want precise localization, rather than abstraction of\nspatial details.\nThere are two technical hurdles in the application of DCNNs to image labeling tasks: signal downsampling, and spatial \u2018insensitivity\u2019 (invariance). The \ufb01rst problem relates to the reduction of signal\nresolution incurred by the repeated combination of max-pooling and downsampling (\u2018striding\u2019) performed at every layer of standard DCNNs (Krizhevsky et al., 2013; Simonyan & Zisserman, 2014;\nSzegedy et al., 2014). Instead, as in Papandreou et al. (2014), we employ the \u2018atrous\u2019 (with holes)\nalgorithm originally developed for ef\ufb01ciently computing the undecimated discrete wavelet transform\n(Mallat, 1999). This allows ef\ufb01cient dense computation of DCNN responses in a scheme substantially simpler than earlier solutions to this problem (Giusti et al., 2013; Sermanet et al., 2013)."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66b6-6db4-11ee-bd81-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-4", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "The second problem relates to the fact that obtaining object-centric decisions from a classi\ufb01er requires invariance to spatial transformations, inherently limiting the spatial accuracy of the DCNN\nmodel. We boost our model\u2019s ability to capture \ufb01ne details by employing a fully-connected Conditional Random Field (CRF). Conditional Random Fields have been broadly used in semantic segmentation to combine class scores computed by multi-way classi\ufb01ers with the low-level information\ncaptured by the local interactions of pixels and edges (Rother et al., 2004; Shotton et al., 2009) or\nsuperpixels (Lucchi et al., 2011). Even though works of increased sophistication have been proposed\nto model the hierarchical dependency (He et al., 2004; Ladicky et al., 2009; Lempitsky et al., 2011)\nand/or high-order dependencies of segments (Delong et al., 2012; Gonfaus et al., 2010; Kohli et al.,\n2009; Chen et al., 2013; Wang et al., 2015), we use the fully connected pairwise CRF proposed by\nKr\u00a8ahenb \u00a8uhl & Koltun (2011) for its ef\ufb01cient computation, and ability to capture \ufb01ne edge details"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66b7-6db4-11ee-983a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-5", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "Kr\u00a8ahenb \u00a8uhl & Koltun (2011) for its ef\ufb01cient computation, and ability to capture \ufb01ne edge details\nwhile also catering for long range dependencies. That model was shown in Kr \u00a8ahenb \u00a8uhl & Koltun\n(2011) to largely improve the performance of a boosting-based pixel-level classi\ufb01er, and in our work\nwe demonstrate that it leads to state-of-the-art results when coupled with a DCNN-based pixel-level\nclassi\ufb01er.\nThe three main advantages of our \u201cDeepLab\u201d system are (i) speed: by virtue of the \u2018atrous\u2019 algorithm, our dense DCNN operates at 8 fps, while Mean Field Inference for the fully-connected CRF\nrequires 0.5 second, (ii) accuracy: we obtain state-of-the-art results on the PASCAL semantic segmentation challenge, outperforming the second-best approach of Mostajabi et al. (2014) by a margin\nof 7.2 %and (iii) simplicity: our system is composed of a cascade of two fairly well-established modules, DCNNs and CRFs.\n2 R ELATED WORK"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66b8-6db4-11ee-a4d4-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-6", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "of 7.2 %and (iii) simplicity: our system is composed of a cascade of two fairly well-established modules, DCNNs and CRFs.\n2 R ELATED WORK\nOur system works directly on the pixel representation, similarly to Long et al. (2014). This is in contrast to the two-stage approaches that are now most common in semantic segmentation with DCNNs:\nsuch techniques typically use a cascade of bottom-up image segmentation and DCNN-based region\nclassi\ufb01cation, which makes the system commit to potential errors of the front-end segmentation system. For instance, the bounding box proposals and masked regions delivered by (Arbel \u00b4aez et al.,\n2014; Uijlings et al., 2013) are used in Girshick et al. (2014) and (Hariharan et al., 2014b) as inputs\nto a DCNN to introduce shape information into the classi\ufb01cation process. Similarly, the authors of\nMostajabi et al. (2014) rely on a superpixel representation. A celebrated non-DCNN precursor to\nthese works is the second order pooling method of (Carreira et al., 2012) which also assigns labels\nto the regions proposals delivered by (Carreira & Sminchisescu, 2012). Understanding the perils"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66b9-6db4-11ee-a296-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-7", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "to the regions proposals delivered by (Carreira & Sminchisescu, 2012). Understanding the perils\nof committing to a single segmentation, the authors of Cogswell et al. (2014) build on (Yadollahpour et al., 2013) to explore a diverse set of CRF-based segmentation proposals, computed also by\n(Carreira & Sminchisescu, 2012). These segmentation proposals are then re-ranked according to a\nDCNN trained in particular for this reranking task. Even though this approach explicitly tries to\nhandle the temperamental nature of a front-end segmentation algorithm, there is still no explicit ex2\nPublished as a conference paper at ICLR 2015\nploitation of the DCNN scores in the CRF-based segmentation algorithm: the DCNN is only applied\npost-hoc, while it would make sense to directly try to use its results during segmentation.\nMoving towards works that lie closer to our approach, several other researchers have considered\nthe use of convolutionally computed DCNN features for dense image labeling. Among the \ufb01rst\nhave been Farabet et al. (2013) who apply DCNNs at multiple image resolutions and then employ a\nsegmentation tree to smooth the prediction results; more recently, Hariharan et al. (2014a) propose to"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66ba-6db4-11ee-aa02-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-8", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "segmentation tree to smooth the prediction results; more recently, Hariharan et al. (2014a) propose to\nconcatenate the computed inter-mediate feature maps within the DCNNs for pixel classi\ufb01cation, and\nDai et al. (2014) propose to pool the inter-mediate feature maps by region proposals. Even though\nthese works still employ segmentation algorithms that are decoupled from the DCNN classi\ufb01er\u2019s\nresults, we believe it is advantageous that segmentation is only used at a later stage, avoiding the\ncommitment to premature decisions.\nMore recently, the segmentation-free techniques of (Long et al., 2014; Eigen & Fergus, 2014) directly apply DCNNs to the whole image in a sliding window fashion, replacing the last fully connected layers of a DCNN by convolutional layers. In order to deal with the spatial localization\nissues outlined in the beginning of the introduction, Long et al. (2014) upsample and concatenate\nthe scores from inter-mediate feature maps, while Eigen & Fergus (2014) re\ufb01ne the prediction result\nfrom coarse to \ufb01ne by propagating the coarse results to another DCNN.\nThe main difference between our model and other state-of-the-art models is the combination of"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66bb-6db4-11ee-a9da-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-9", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "from coarse to \ufb01ne by propagating the coarse results to another DCNN.\nThe main difference between our model and other state-of-the-art models is the combination of\npixel-level CRFs and DCNN-based \u2018unary terms\u2019. Focusing on the closest works in this direction,\nCogswell et al. (2014) use CRFs as a proposal mechanism for a DCNN-based reranking system,\nwhile Farabet et al. (2013) treat superpixels as nodes for a local pairwise CRF and use graph-cuts for\ndiscrete inference; as such their results can be limited by errors in superpixel computations, while ignoring long-range superpixel dependencies. Our approach instead treats every pixel as a CRF node,\nexploits long-range dependencies, and uses CRF inference to directly optimize a DCNN-driven cost\nfunction. We note that mean \ufb01eld had been extensively studied for traditional image segmentation/edge detection tasks, e.g., (Geiger & Girosi, 1991; Geiger & Yuille, 1991; Kokkinos et al.,\n2008), but recently Kr \u00a8ahenb \u00a8uhl & Koltun (2011) showed that the inference can be very ef\ufb01cient for\nfully connected CRF and particularly effective in the context of semantic segmentation."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66bc-6db4-11ee-affc-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-10", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "fully connected CRF and particularly effective in the context of semantic segmentation.\nAfter the \ufb01rst version of our manuscript was made publicly available, it came to our attention that\ntwo other groups have independently and concurrently pursued a very similar direction, combining\nDCNNs and densely connected CRFs (Bell et al., 2014; Zheng et al., 2015). There are several\ndifferences in technical aspects of the respective models. Bell et al. (2014) focus on the problem\nof material classi\ufb01cation, while Zheng et al. (2015) unroll the CRF mean-\ufb01eld inference steps to\nconvert the whole system into an end-to-end trainable feed-forward network.\nWe have updated our proposed \u201cDeepLab\u201d system with much improved methods and results in our\nlatest work (Chen et al., 2016). We refer the interested reader to the paper for details.\n3 C ONVOLUTIONAL NEURAL NETWORKS FOR DENSE IMAGE LABELING\nHerein we describe how we have re-purposed and \ufb01netuned the publicly available Imagenetpretrained state-of-art 16-layer classi\ufb01cation network of (Simonyan & Zisserman, 2014) (VGG-16)\ninto an ef\ufb01cient and effective dense feature extractor for our dense semantic image segmentation"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66bd-6db4-11ee-9bc6-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-11", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "into an ef\ufb01cient and effective dense feature extractor for our dense semantic image segmentation\nsystem.\n3.1 E FFICIENT DENSE SLIDING WINDOW FEATURE EXTRACTION WITH THE HOLE\nALGORITHM\nDense spatial score evaluation is instrumental in the success of our dense CNN feature extractor. As\na \ufb01rst step to implement this, we convert the fully-connected layers of VGG-16 into convolutional\nones and run the network in a convolutional fashion on the image at its original resolution. However\nthis is not enough as it yields very sparsely computed detection scores (with a stride of 32 pixels). To\ncompute scores more densely at our target stride of 8 pixels, we develop a variation of the method\npreviously employed by Giusti et al. (2013); Sermanet et al. (2013). We skip subsampling after\nthe last two max-pooling layers in the network of Simonyan & Zisserman (2014) and modify the\nconvolutional \ufb01lters in the layers that follow them by introducing zeros to increase their length ( 2\u0002in\n3\nPublished as a conference paper at ICLR 2015\nInput stride\nOutput stride\nFigure 1: Illustration of the hole algorithm in 1-D, when kernel size = 3 ,input stride = 2 , and"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66be-6db4-11ee-9ea3-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-12", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "Input stride\nOutput stride\nFigure 1: Illustration of the hole algorithm in 1-D, when kernel size = 3 ,input stride = 2 , and\noutput stride = 1 .\nthe last three convolutional layers and 4\u0002in the \ufb01rst fully connected layer). We can implement this\nmore ef\ufb01ciently by keeping the \ufb01lters intact and instead sparsely sample the feature maps on which\nthey are applied on using an input stride of 2 or 4 pixels, respectively. This approach, illustrated\nin Fig. 1 is known as the \u2018hole algorithm\u2019 (\u2018atrous algorithm\u2019) and has been developed before for\nef\ufb01cient computation of the undecimated wavelet transform (Mallat, 1999). We have implemented\nthis within the Caffe framework (Jia et al., 2014) by adding to the im2col function (it converts multichannel feature maps to vectorized patches) the option to sparsely sample the underlying feature\nmap. This approach is generally applicable and allows us to ef\ufb01ciently compute dense CNN feature\nmaps at any target subsampling rate without introducing any approximations.\nWe \ufb01netune the model weights of the Imagenet-pretrained VGG-16 network to adapt it to the image"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66bf-6db4-11ee-9802-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-13", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "We \ufb01netune the model weights of the Imagenet-pretrained VGG-16 network to adapt it to the image\nclassi\ufb01cation task in a straightforward fashion, following the procedure of Long et al. (2014). We\nreplace the 1000-way Imagenet classi\ufb01er in the last layer of VGG-16 with a 21-way one. Our\nloss function is the sum of cross-entropy terms for each spatial position in the CNN output map\n(subsampled by 8 compared to the original image). All positions and labels are equally weighted in\nthe overall loss function. Our targets are the ground truth labels (subsampled by 8). We optimize the\nobjective function with respect to the weights at all network layers by the standard SGD procedure\nof Krizhevsky et al. (2013).\nDuring testing, we need class score maps at the original image resolution. As illustrated in Figure 2\nand further elaborated in Section 4.1, the class score maps (corresponding to log-probabilities) are\nquite smooth, which allows us to use simple bilinear interpolation to increase their resolution by a\nfactor of 8 at a negligible computational cost. Note that the method of Long et al. (2014) does not\nuse the hole algorithm and produces very coarse scores (subsampled by a factor of 32) at the CNN"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66c0-6db4-11ee-9a88-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-14", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "use the hole algorithm and produces very coarse scores (subsampled by a factor of 32) at the CNN\noutput. This forced them to use learned upsampling layers, signi\ufb01cantly increasing the complexity\nand training time of their system: Fine-tuning our network on PASCAL VOC 2012 takes about 10\nhours, while they report a training time of several days (both timings on a modern GPU).\n3.2 C ONTROLLING THE RECEPTIVE FIELD SIZE AND ACCELERATING DENSE\nCOMPUTATION WITH CONVOLUTIONAL NETS\nAnother key ingredient in re-purposing our network for dense score computation is explicitly controlling the network\u2019s receptive \ufb01eld size. Most recent DCNN-based image recognition methods\nrely on networks pre-trained on the Imagenet large-scale classi\ufb01cation task. These networks typically have large receptive \ufb01eld size: in the case of the VGG-16 net we consider, its receptive \ufb01eld\nis224\u0002224(with zero-padding) and 404\u0002404pixels if the net is applied convolutionally. After"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66c1-6db4-11ee-88cc-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-15", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "is224\u0002224(with zero-padding) and 404\u0002404pixels if the net is applied convolutionally. After\nconverting the network to a fully convolutional one, the \ufb01rst fully connected layer has 4,096 \ufb01lters of large 7\u00027spatial size and becomes the computational bottleneck in our dense score map\ncomputation.\nWe have addressed this practical problem by spatially subsampling (by simple decimation) the \ufb01rst\nFC layer to 4\u00024(or3\u00023) spatial size. This has reduced the receptive \ufb01eld of the network down to\n128\u0002128(with zero-padding) or 308\u0002308(in convolutional mode) and has reduced computation time\nfor the \ufb01rst FC layer by 2\u00003times. Using our Caffe-based implementation and a Titan GPU, the\nresulting VGG-derived network is very ef\ufb01cient: Given a 306\u0002306input image, it produces 39\u000239\n4\nPublished as a conference paper at ICLR 2015\ndense raw feature scores at the top of the network at a rate of about 8 frames/sec during testing. The\nspeed during training is 3 frames/sec. We have also successfully experimented with reducing the\nnumber of channels at the fully connected layers from 4,096 down to 1,024, considerably further"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66c2-6db4-11ee-a26a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-16", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "speed during training is 3 frames/sec. We have also successfully experimented with reducing the\nnumber of channels at the fully connected layers from 4,096 down to 1,024, considerably further\ndecreasing computation time and memory footprint without sacri\ufb01cing performance, as detailed in\nSection 5. Using smaller networks such as Krizhevsky et al. (2013) could allow video-rate test-time\ndense feature computation even on light-weight GPUs.\n4 D ETAILED BOUNDARY RECOVERY : FULLY -CONNECTED CONDITIONAL\nRANDOM FIELDS AND MULTI -SCALE PREDICTION\n4.1 D EEPCONVOLUTIONAL NETWORKS AND THE LOCALIZATION CHALLENGE\nAs illustrated in Figure 2, DCNN score maps can reliably predict the presence and rough position\nof objects in an image but are less well suited for pin-pointing their exact outline. There is a natural\ntrade-off between classi\ufb01cation accuracy and localization accuracy with convolutional networks:\nDeeper models with multiple max-pooling layers have proven most successful in classi\ufb01cation tasks,\nhowever their increased invariance and large receptive \ufb01elds make the problem of inferring position\nfrom the scores at their top output levels more challenging."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66c3-6db4-11ee-98a0-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-17", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "however their increased invariance and large receptive \ufb01elds make the problem of inferring position\nfrom the scores at their top output levels more challenging.\nRecent work has pursued two directions to address this localization challenge. The \ufb01rst approach is\nto harness information from multiple layers in the convolutional network in order to better estimate\nthe object boundaries (Long et al., 2014; Eigen & Fergus, 2014). The second approach is to employ\na super-pixel representation, essentially delegating the localization task to a low-level segmentation\nmethod. This route is followed by the very successful recent method of Mostajabi et al. (2014).\nIn Section 4.2, we pursue a novel alternative direction based on coupling the recognition capacity\nof DCNNs and the \ufb01ne-grained localization accuracy of fully connected CRFs and show that it is\nremarkably successful in addressing the localization challenge, producing accurate semantic segmentation results and recovering object boundaries at a level of detail that is well beyond the reach\nof existing methods.\n4.2 F ULLY -CONNECTED CONDITIONAL RANDOM FIELDS FOR ACCURATE LOCALIZATION\nImage/G.T. DCNN output CRF Iteration 1 CRF Iteration 2 CRF Iteration 10"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66c4-6db4-11ee-80a4-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-18", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "Image/G.T. DCNN output CRF Iteration 1 CRF Iteration 2 CRF Iteration 10\nFigure 2: Score map (input before softmax function) and belief map (output of softmax function) for\nAeroplane. We show the score (1st row) and belief (2nd row) maps after each mean \ufb01eld iteration.\nThe output of last DCNN layer is used as input to the mean \ufb01eld inference. Best viewed in color.\nTraditionally, conditional random \ufb01elds (CRFs) have been employed to smooth noisy segmentation\nmaps (Rother et al., 2004; Kohli et al., 2009). Typically these models contain energy terms that\ncouple neighboring nodes, favoring same-label assignments to spatially proximal pixels. Qualitatively, the primary function of these short-range CRFs has been to clean up the spurious predictions\nof weak classi\ufb01ers built on top of local hand-engineered features.\nCompared to these weaker classi\ufb01ers, modern DCNN architectures such as the one we use in this\nwork produce score maps and semantic label predictions which are qualitatively different. As illustrated in Figure 2, the score maps are typically quite smooth and produce homogeneous classi\ufb01cation"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66c5-6db4-11ee-9c7a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-19", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "work produce score maps and semantic label predictions which are qualitatively different. As illustrated in Figure 2, the score maps are typically quite smooth and produce homogeneous classi\ufb01cation\nresults. In this regime, using short-range CRFs can be detrimental, as our goal should be to recover\ndetailed local structure rather than further smooth it. Using contrast-sensitive potentials (Rother\n5\nPublished as a conference paper at ICLR 2015\nDeep \nConvolutional \nNeural \nNetwork\nInputAeroplane\nCoarse Score map\nBi-linear Interpolation Fully Connected CRF Final Output\nFigure 3: Model Illustration. The coarse score map from Deep Convolutional Neural Network (with\nfully convolutional layers) is upsampled by bi-linear interpolation. A fully connected CRF is applied\nto re\ufb01ne the segmentation result. Best viewed in color.\net al., 2004) in conjunction to local-range CRFs can potentially improve localization but still miss\nthin-structures and typically requires solving an expensive discrete optimization problem.\nTo overcome these limitations of short-range CRFs, we integrate into our system the fully connected\nCRF model of Kr \u00a8ahenb \u00a8uhl & Koltun (2011). The model employs the energy function\nE(x) =X\ni\u0012i(xi) +X"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66c6-6db4-11ee-bb26-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-20", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "E(x) =X\ni\u0012i(xi) +X\nij\u0012ij(xi;xj) (1)\nwhere xis the label assignment for pixels. We use as unary potential \u0012i(xi) =\u0000logP(xi), where\nP(xi)is the label assignment probability at pixel ias computed by DCNN. The pairwise potential\nis\u0012ij(xi;xj) =\u0016(xi;xj)PK\nm=1wm\u0001km(fi;fj), where\u0016(xi;xj) = 1 ifxi6=xj, and zero\notherwise ( i.e., Potts Model). There is one pairwise term for each pair of pixels iandjin the image\nno matter how far from each other they lie, i.e. the model\u2019s factor graph is fully connected. Each km\nis the Gaussian kernel depends on features (denoted as f) extracted for pixel iandjand is weighted\nby parameter wm. We adopt bilateral position and color terms, speci\ufb01cally, the kernels are\nw1exp\u0010\n\u0000jjpi\u0000pjjj2\n2\u001b2\u000b\u0000jjIi\u0000Ijjj2\n2\u001b2\n\f\u0011\n+w2exp\u0010\n\u0000jjpi\u0000pjjj2\n2\u001b2\r\u0011\n(2)"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66c7-6db4-11ee-8c7b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-21", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "2\u001b2\n\f\u0011\n+w2exp\u0010\n\u0000jjpi\u0000pjjj2\n2\u001b2\r\u0011\n(2)\nwhere the \ufb01rst kernel depends on both pixel positions (denoted as p) and pixel color intensities\n(denoted asI), and the second kernel only depends on pixel positions. The hyper parameters \u001b\u000b,\u001b\f\nand\u001b\rcontrol the \u201cscale\u201d of the Gaussian kernels.\nCrucially, this model is amenable to ef\ufb01cient approximate probabilistic inference (Kr \u00a8ahenb \u00a8uhl &\nKoltun, 2011). The message passing updates under a fully decomposable mean \ufb01eld approximationb(x) =Q\nibi(xi)can be expressed as convolutions with a Gaussian kernel in feature space.\nHigh-dimensional \ufb01ltering algorithms (Adams et al., 2010) signi\ufb01cantly speed-up this computation\nresulting in an algorithm that is very fast in practice, less that 0.5 sec on average for Pascal VOC\nimages using the publicly available implementation of (Kr \u00a8ahenb \u00a8uhl & Koltun, 2011).\n4.3 M ULTI -SCALE PREDICTION"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66c8-6db4-11ee-8d49-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-22", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "images using the publicly available implementation of (Kr \u00a8ahenb \u00a8uhl & Koltun, 2011).\n4.3 M ULTI -SCALE PREDICTION\nFollowing the promising recent results of (Hariharan et al., 2014a; Long et al., 2014) we have also\nexplored a multi-scale prediction method to increase the boundary localization accuracy. Specifically, we attach to the input image and the output of each of the \ufb01rst four max pooling layers a\ntwo-layer MLP (\ufb01rst layer: 128 3x3 convolutional \ufb01lters, second layer: 128 1x1 convolutional \ufb01lters) whose feature map is concatenated to the main network\u2019s last layer feature map. The aggregate\nfeature map fed into the softmax layer is thus enhanced by 5 * 128 = 640 channels. We only adjust\nthe newly added weights, keeping the other network parameters to the values learned by the method\nof Section 3. As discussed in the experimental section, introducing these extra direct connections\nfrom \ufb01ne-resolution layers improves localization performance, yet the effect is not as dramatic as\nthe one obtained with the fully-connected CRF.\n6\nPublished as a conference paper at ICLR 2015\nMethod mean IOU (%)\nDeepLab 59.80\nDeepLab-CRF 63.74"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66c9-6db4-11ee-ba90-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-23", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "6\nPublished as a conference paper at ICLR 2015\nMethod mean IOU (%)\nDeepLab 59.80\nDeepLab-CRF 63.74\nDeepLab-MSc 61.30\nDeepLab-MSc-CRF 65.21\nDeepLab-7x7 64.38\nDeepLab-CRF-7x7 67.64\nDeepLab-LargeFOV 62.25\nDeepLab-CRF-LargeFOV 67.64\nDeepLab-MSc-LargeFOV 64.21\nDeepLab-MSc-CRF-LargeFOV 68.70Method mean IOU (%)\nMSRA-CFM 61.8\nFCN-8s 62.2\nTTI-Zoomout-16 64.4\nDeepLab-CRF 66.4\nDeepLab-MSc-CRF 67.1\nDeepLab-CRF-7x7 70.3\nDeepLab-CRF-LargeFOV 70.3\nDeepLab-MSc-CRF-LargeFOV 71.6\n(a) (b)\nTable 1: (a) Performance of our proposed models on the PASCAL VOC 2012 \u2018val\u2019 set (with training\nin the augmented \u2018train\u2019 set). The best performance is achieved by exploiting both multi-scale"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66ca-6db4-11ee-8c30-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-24", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "in the augmented \u2018train\u2019 set). The best performance is achieved by exploiting both multi-scale\nfeatures and large \ufb01eld-of-view. (b) Performance of our proposed models (with training in the\naugmented \u2018trainval\u2019 set) compared to other state-of-art methods on the PASCAL VOC 2012 \u2018test\u2019\nset.\n5 E XPERIMENTAL EVALUATION\nDataset We test our DeepLab model on the PASCAL VOC 2012 segmentation benchmark (Everingham et al., 2014), consisting of 20 foreground object classes and one background class. The\noriginal dataset contains 1;464,1;449, and 1;456images for training, validation, and testing, respectively. The dataset is augmented by the extra annotations provided by Hariharan et al. (2011),\nresulting in 10;582training images. The performance is measured in terms of pixel intersectionover-union (IOU) averaged across the 21 classes.\nTraining We adopt the simplest form of piecewise training, decoupling the DCNN and CRF training stages, assuming the unary terms provided by the DCNN are \ufb01xed during CRF training.\nFor DCNN training we employ the VGG-16 network which has been pre-trained on ImageNet. We"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66cb-6db4-11ee-8223-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-25", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "For DCNN training we employ the VGG-16 network which has been pre-trained on ImageNet. We\n\ufb01ne-tuned the VGG-16 network on the VOC 21-way pixel-classi\ufb01cation task by stochastic gradient\ndescent on the cross-entropy loss function, as described in Section 3.1. We use a mini-batch of 20\nimages and initial learning rate of 0:001(0:01for the \ufb01nal classi\ufb01er layer), multiplying the learning\nrate by 0.1 at every 2000 iterations. We use momentum of 0:9and a weight decay of 0:0005 .\nAfter the DCNN has been \ufb01ne-tuned, we cross-validate the parameters of the fully connected CRF\nmodel in Eq. (2) along the lines of Kr \u00a8ahenb \u00a8uhl & Koltun (2011). We use the default values of\nw2= 3 and\u001b\r= 3 and we search for the best values of w1,\u001b\u000b, and\u001b\fby cross-validation on a\nsmall subset of the validation set (we use 100 images). We employ coarse-to-\ufb01ne search scheme."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66cc-6db4-11ee-9dbc-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-26", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "small subset of the validation set (we use 100 images). We employ coarse-to-\ufb01ne search scheme.\nSpeci\ufb01cally, the initial search range of the parameters are w12[5;10],\u001b\u000b2[50 : 10 : 100] and\n\u001b\f2[3 : 1 : 10] (MATLAB notation), and then we re\ufb01ne the search step sizes around the \ufb01rst\nround\u2019s best values. We \ufb01x the number of mean \ufb01eld iterations to 10 for all reported experiments.\nEvaluation on Validation set We conduct the majority of our evaluations on the PASCAL \u2018val\u2019\nset, training our model on the augmented PASCAL \u2018train\u2019 set. As shown in Tab. 1 (a), incorporating\nthe fully connected CRF to our model (denoted by DeepLab-CRF) yields a substantial performance\nboost, about 4% improvement over DeepLab. We note that the work of Kr \u00a8ahenb \u00a8uhl & Koltun\n(2011) improved the 27:6%result of TextonBoost (Shotton et al., 2009) to 29:1%, which makes the\nimprovement we report here (from 59:8%to63:7%) all the more impressive."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66cd-6db4-11ee-9e9c-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-27", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "improvement we report here (from 59:8%to63:7%) all the more impressive.\nTurning to qualitative results, we provide visual comparisons between DeepLab and DeepLab-CRF\nin Fig. 7. Employing a fully connected CRF signi\ufb01cantly improves the results, allowing the model\nto accurately capture intricate object boundaries.\nMulti-Scale features We also exploit the features from the intermediate layers, similar to Hariharan et al. (2014a); Long et al. (2014). As shown in Tab. 1 (a), adding the multi-scale features to our\n7\nPublished as a conference paper at ICLR 2015\nMethod kernel size input stride receptive \ufb01eld # parameters mean IOU (%) Training speed (img/sec)\nDeepLab-CRF-7x7 7\u00027 4 224 134.3M 67.64 1.44\nDeepLab-CRF 4\u00024 4 128 65.1M 63.74 2.90\nDeepLab-CRF-4x4 4\u00024 8 224 65.1M 67.14 2.90\nDeepLab-CRF-LargeFOV 3\u00023 12 224 20.5M 67.64 4.84\nTable 2: Effect of Field-Of-View. We show the performance (after CRF) and training speed on the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66ce-6db4-11ee-98f7-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-28", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "Table 2: Effect of Field-Of-View. We show the performance (after CRF) and training speed on the\nPASCAL VOC 2012 \u2018val\u2019 set as the function of (1) the kernel size of \ufb01rst fully connected layer, (2)\nthe input stride value employed in the atrous algorithm.\nDeepLab model (denoted as DeepLab-MSc) improves about 1:5%performance, and further incorporating the fully connected CRF (denoted as DeepLab-MSc-CRF) yields about 4% improvement.\nThe qualitative comparisons between DeepLab and DeepLab-MSc are shown in Fig. 4. Leveraging\nthe multi-scale features can slightly re\ufb01ne the object boundaries.\nField of View The \u2018atrous algorithm\u2019 we employed allows us to arbitrarily control the Field-ofView (FOV) of the models by adjusting the input stride, as illustrated in Fig. 1. In Tab. 2, we\nexperiment with several kernel sizes and input strides at the \ufb01rst fully connected layer. The method,\nDeepLab-CRF-7x7, is the direct modi\ufb01cation from VGG-16 net, where the kernel size = 7\u00027and\ninput stride = 4. This model yields performance of 67:64% on the \u2018val\u2019 set, but it is relatively slow"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66cf-6db4-11ee-bdc5-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-29", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "input stride = 4. This model yields performance of 67:64% on the \u2018val\u2019 set, but it is relatively slow\n(1:44images per second during training). We have improved model speed to 2:9images per second\nby reducing the kernel size to 4\u00024. We have experimented with two such network variants with\ndifferent FOV sizes, DeepLab-CRF and DeepLab-CRF-4x4; the latter has large FOV ( i.e., large\ninput stride) and attains better performance. Finally, we employ kernel size 3\u00023and input stride =\n12, and further change the \ufb01lter sizes from 4096 to 1024 for the last two layers. Interestingly, the\nresulting model, DeepLab-CRF-LargeFOV , matches the performance of the expensive DeepLabCRF-7x7. At the same time, it is 3:36times faster to run and has signi\ufb01cantly fewer parameters\n(20.5M instead of 134.3M).\nThe performance of several model variants is summarized in Tab. 1, showing the bene\ufb01t of exploiting\nmulti-scale features and large FOV .\nFigure 4: Incorporating multi-scale features improves the boundary segmentation. We show the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66d0-6db4-11ee-a011-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-30", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "multi-scale features and large FOV .\nFigure 4: Incorporating multi-scale features improves the boundary segmentation. We show the\nresults obtained by DeepLab and DeepLab-MSc in the \ufb01rst and second row, respectively. Best\nviewed in color.\nMean Pixel IOU along Object Boundaries To quantify the accuracy of the proposed model near\nobject boundaries, we evaluate the segmentation accuracy with an experiment similar to Kohli et al.\n(2009); Kr \u00a8ahenb \u00a8uhl & Koltun (2011). Speci\ufb01cally, we use the \u2018void\u2019 label annotated in val set,\nwhich usually occurs around object boundaries. We compute the mean IOU for those pixels that\nare located within a narrow band (called trimap) of \u2018void\u2019 labels. As shown in Fig. 5, exploiting\nthe multi-scale features from the intermediate layers and re\ufb01ning the segmentation results by a fully\nconnected CRF signi\ufb01cantly improve the results around object boundaries.\nComparison with State-of-art In Fig. 6, we qualitatively compare our proposed model, DeepLabCRF, with two state-of-art models: FCN-8s (Long et al., 2014) and TTI-Zoomout-16 (Mostajabi"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66d1-6db4-11ee-a774-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-31", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "et al., 2014) on the \u2018val\u2019 set (the results are extracted from their papers). Our model is able to\ncapture the intricate object boundaries.\n8\nPublished as a conference paper at ICLR 2015\n0 510 15 20 25 30 35 405560657075808590Pixelwise Accuracy (%)\nTrimap Width (pixels)  \nDL\u2212MSc\u2212CRF\nDeepLab\u2212CRF\nDeepLab\u2212MSc\nDeepLab\n0 510 15 20 25 30 35 4035404550556065mean IOU (%)\nTrimap Width (pixels)  \nDL\u2212MSc\u2212CRF\nDeepLab\u2212CRF\nDeepLab\u2212MSc\nDeepLab\n(a) (b) (c)\nFigure 5: (a) Some trimap examples (top-left: image. top-right: ground-truth. bottom-left: trimap\nof 2 pixels. bottom-right: trimap of 10 pixels). Quality of segmentation result within a band around\nthe object boundaries for the proposed methods. (b) Pixelwise accuracy. (c) Pixel mean IOU.\n(a) FCN-8s vs. DeepLab-CRF (b) TTI-Zoomout-16 vs. DeepLab-CRF"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66d2-6db4-11ee-b1c3-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-32", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "(a) FCN-8s vs. DeepLab-CRF (b) TTI-Zoomout-16 vs. DeepLab-CRF\nFigure 6: Comparisons with state-of-the-art models on the val set. First row: images. Second row:\nground truths. Third row: other recent models (Left: FCN-8s, Right: TTI-Zoomout-16). Fourth\nrow: our DeepLab-CRF. Best viewed in color.\nReproducibility We have implemented the proposed methods by extending the excellent Caffe\nframework (Jia et al., 2014). We share our source code, con\ufb01guration \ufb01les, and trained models that\nallow reproducing the results in this paper at a companion web site https://bitbucket.org/\ndeeplab/deeplab-public .\nTest set results Having set our model choices on the validation set, we evaluate our model variants\non the PASCAL VOC 2012 of\ufb01cial \u2018test\u2019 set. As shown in Tab. 3, our DeepLab-CRF and DeepLabMSc-CRF models achieve performance of 66:4%and67:1%mean IOU1, respectively. Our models"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66d3-6db4-11ee-882e-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-33", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "outperform all the other state-of-the-art models (speci\ufb01cally, TTI-Zoomout-16 (Mostajabi et al.,\n2014), FCN-8s (Long et al., 2014), and MSRA-CFM (Dai et al., 2014)). When we increase the FOV\nof the models, DeepLab-CRF-LargeFOV yields performance of 70:3%, the same as DeepLab-CRF7x7, while its training speed is faster. Furthermore, our best model, DeepLab-MSc-CRF-LargeFOV ,\nattains the best performance of 71:6%by employing both multi-scale features and large FOV .\n1http://host.robots.ox.ac.uk:8080/leaderboard/displaylb.php?\nchallengeid=11&compid=6\n9\nPublished as a conference paper at ICLR 2015\nFigure 7: Visualization results on VOC 2012-val. For each row, we show the input image, the\nsegmentation result delivered by the DCNN (DeepLab), and the re\ufb01ned segmentation result of the\nFully Connected CRF (DeepLab-CRF). We show our failure modes in the last three rows. Best\nviewed in color.\n10"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66d4-6db4-11ee-bea0-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-34", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "Fully Connected CRF (DeepLab-CRF). We show our failure modes in the last three rows. Best\nviewed in color.\n10\nPublished as a conference paper at ICLR 2015\nMethod bkg aero bike bird boat bottle bus car cat chair cow table dog horse mbike person plant sheep sofa train tv mean\nMSRA-CFM -75.7 26.7 69.5 48.8 65.6 81.0 69.2 73.3 30.0 68.7 51.5 69.1 68.1 71.7 67.5 50.4 66.5 44.4 58.9 53.5 61.8\nFCN-8s -76.8 34.2 68.9 49.4 60.3 75.3 74.7 77.6 21.4 62.5 46.8 71.8 63.9 76.5 73.9 45.2 72.4 37.4 70.9 55.1 62.2\nTTI-Zoomout-16 89.8 81.9 35.1 78.2 57.4 56.5 80.5 74.0 79.8 22.4 69.6 53.7 74.0 76.0 76.6 68.8 44.3 70.2 40.2 68.9 55.3 64.4"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66d5-6db4-11ee-b73f-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-35", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "DeepLab-CRF 92.1 78.4 33.1 78.2 55.6 65.3 81.3 75.5 78.6 25.3 69.2 52.7 75.2 69.0 79.1 77.6 54.7 78.3 45.1 73.3 56.2 66.4\nDeepLab-MSc-CRF 92.6 80.4 36.8 77.4 55.2 66.4 81.5 77.5 78.9 27.1 68.2 52.7 74.3 69.6 79.4 79.0 56.9 78.8 45.2 72.7 59.3 67.1\nDeepLab-CRF-7x7 92.8 83.9 36.6 77.5 58.4 68.0 84.6 79.7 83.1 29.5 74.6 59.3 78.9 76.0 82.1 80.6 60.3 81.7 49.2 78.0 60.7 70.3\nDeepLab-CRF-LargeFOV 92.6 83.5 36.6 82.5 62.3 66.5 85.4 78.5 83.7 30.4 72.9 60.4 78.5 75.5 82.1 79.7 58.2 82.0 48.8 73.7 63.3 70.3"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66d6-6db4-11ee-b08f-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-36", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "DeepLab-MSc-CRF-LargeFOV 93.1 84.4 54.5 81.5 63.6 65.9 85.1 79.1 83.4 30.7 74.1 59.8 79.0 76.1 83.2 80.8 59.7 82.2 50.4 73.1 63.7 71.6\nTable 3: Labeling IOU (%) on the PASCAL VOC 2012 test set, using the trainval set for training.\n6 D ISCUSSION\nOur work combines ideas from deep convolutional neural networks and fully-connected conditional\nrandom \ufb01elds, yielding a novel method able to produce semantically accurate predictions and detailed segmentation maps, while being computationally ef\ufb01cient. Our experimental results show that\nthe proposed method signi\ufb01cantly advances the state-of-art in the challenging PASCAL VOC 2012\nsemantic image segmentation task.\nThere are multiple aspects in our model that we intend to re\ufb01ne, such as fully integrating its two\nmain components (CNN and CRF) and train the whole system in an end-to-end fashion, similar to\nKr\u00a8ahenb \u00a8uhl & Koltun (2013); Chen et al. (2014); Zheng et al. (2015). We also plan to experiment"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66d7-6db4-11ee-a9ff-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-37", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "Kr\u00a8ahenb \u00a8uhl & Koltun (2013); Chen et al. (2014); Zheng et al. (2015). We also plan to experiment\nwith more datasets and apply our method to other sources of data such as depth maps or videos. Recently, we have pursued model training with weakly supervised annotations, in the form of bounding\nboxes or image-level labels (Papandreou et al., 2015).\nAt a higher level, our work lies in the intersection of convolutional neural networks and probabilistic\ngraphical models. We plan to further investigate the interplay of these two powerful classes of\nmethods and explore their synergistic potential for solving challenging computer vision tasks.\nACKNOWLEDGMENTS\nThis work was partly supported by ARO 62250-CS, NIH Grant 5R01EY022247-03, EU Project\nRECONFIG FP7-ICT-600825 and EU Project MOBOT FP7-ICT-2011-600796. We also gratefully\nacknowledge the support of NVIDIA Corporation with the donation of GPUs used for this research.\nWe would like to thank the anonymous reviewers for their detailed comments and constructive feedback.\nPAPER REVISIONS\nHere we present the list of major paper revisions for the convenience of the readers.\nv1 Submission to ICLR 2015. Introduces the model DeepLab-CRF, which attains the performance"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66d8-6db4-11ee-850b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-38", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "v1 Submission to ICLR 2015. Introduces the model DeepLab-CRF, which attains the performance\nof66:4%on PASCAL VOC 2012 test set.\nv2 Rebuttal for ICLR 2015. Adds the model DeepLab-MSc-CRF, which incorporates multi-scale\nfeatures from the intermediate layers. DeepLab-MSc-CRF yields the performance of 67:1%on\nPASCAL VOC 2012 test set.\nv3 Camera-ready for ICLR 2015. Experiments with large Field-Of-View. On PASCAL VOC 2012\ntest set, DeepLab-CRF-LargeFOV achieves the performance of 70:3%. When exploiting both mutliscale features and large FOV , DeepLab-MSc-CRF-LargeFOV attains the performance of 71:6%.\nv4 Reference to our updated \u201cDeepLab\u201d system (Chen et al., 2016) with much improved results.\nREFERENCES\nAdams, A., Baek, J., and Davis, M. A. Fast high-dimensional \ufb01ltering using the permutohedral\nlattice. In Computer Graphics Forum , 2010."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66d9-6db4-11ee-9d0b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-39", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "lattice. In Computer Graphics Forum , 2010.\nArbel \u00b4aez, P., Pont-Tuset, J., Barron, J. T., Marques, F., and Malik, J. Multiscale combinatorial\ngrouping. In CVPR , 2014.\n11\nPublished as a conference paper at ICLR 2015\nBell, S., Upchurch, P., Snavely, N., and Bala, K. Material recognition in the wild with the materials\nin context database. arXiv:1412.0623 , 2014.\nCarreira, J. and Sminchisescu, C. Cpmc: Automatic object segmentation using constrained parametric min-cuts. PAMI , 2012.\nCarreira, J., Caseiro, R., Batista, J., and Sminchisescu, C. Semantic segmentation with second-order\npooling. In ECCV , 2012.\nChen, L.-C., Papandreou, G., and Yuille, A. Learning a dictionary of shape epitomes with applications to image labeling. In ICCV , 2013.\nChen, L.-C., Schwing, A., Yuille, A., and Urtasun, R. Learning deep structured models.\narXiv:1407.2538 , 2014."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66da-6db4-11ee-aaed-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-40", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "arXiv:1407.2538 , 2014.\nChen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A. L. Deeplab: Semantic\nimage segmentation with deep convolutional nets, atrous convolution, and fully connected crfs.\narXiv:1606.00915 , 2016.\nChen, X. and Yuille, A. L. Articulated pose estimation by a graphical model with image dependent\npairwise relations. In NIPS , 2014.\nCogswell, M., Lin, X., Purushwalkam, S., and Batra, D. Combining the best of graphical models\nand convnets for semantic segmentation. arXiv:1412.4313 , 2014.\nDai, J., He, K., and Sun, J. Convolutional feature masking for joint object and stuff segmentation.\narXiv:1412.1283 , 2014.\nDelong, A., Osokin, A., Isack, H. N., and Boykov, Y . Fast approximate energy minimization with\nlabel costs. IJCV , 2012.\nEigen, D. and Fergus, R. Predicting depth, surface normals and semantic labels with a common"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66db-6db4-11ee-b2e4-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-41", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "label costs. IJCV , 2012.\nEigen, D. and Fergus, R. Predicting depth, surface normals and semantic labels with a common\nmulti-scale convolutional architecture. arXiv:1411.4734 , 2014.\nEveringham, M., Eslami, S. M. A., Gool, L. V ., Williams, C. K. I., Winn, J., and Zisserma, A. The\npascal visual object classes challenge a retrospective. IJCV , 2014.\nFarabet, C., Couprie, C., Najman, L., and LeCun, Y . Learning hierarchical features for scene labeling.PAMI , 2013.\nGeiger, D. and Girosi, F. Parallel and deterministic algorithms from mrfs: Surface reconstruction.\nPAMI , 13(5):401\u2013412, 1991.\nGeiger, D. and Yuille, A. A common framework for image segmentation. IJCV , 6(3):227\u2013243,\n1991.\nGirshick, R., Donahue, J., Darrell, T., and Malik, J. Rich feature hierarchies for accurate object\ndetection and semantic segmentation. In CVPR , 2014."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66dc-6db4-11ee-8d5c-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-42", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "detection and semantic segmentation. In CVPR , 2014.\nGiusti, A., Ciresan, D., Masci, J., Gambardella, L., and Schmidhuber, J. Fast image scanning with\ndeep max-pooling convolutional neural networks. In ICIP , 2013.\nGonfaus, J. M., Boix, X., Van de Weijer, J., Bagdanov, A. D., Serrat, J., and Gonzalez, J. Harmony\npotentials for joint classi\ufb01cation and segmentation. In CVPR , 2010.\nHariharan, B., Arbel \u00b4aez, P., Bourdev, L., Maji, S., and Malik, J. Semantic contours from inverse\ndetectors. In ICCV , 2011.\nHariharan, B., Arbel \u00b4aez, P., Girshick, R., and Malik, J. Hypercolumns for object segmentation and\n\ufb01ne-grained localization. arXiv:1411.5752 , 2014a.\nHariharan, B., Arbel \u00b4aez, P., Girshick, R., and Malik, J. Simultaneous detection and segmentation.\nInECCV , 2014b.\n12\nPublished as a conference paper at ICLR 2015"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66dd-6db4-11ee-a52a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-43", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "InECCV , 2014b.\n12\nPublished as a conference paper at ICLR 2015\nHe, X., Zemel, R. S., and Carreira-Perpindn, M. Multiscale conditional random \ufb01elds for image\nlabeling. In CVPR , 2004.\nJia, Y ., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., and Darrell,\nT. Caffe: Convolutional architecture for fast feature embedding. arXiv:1408.5093 , 2014.\nKohli, P., Ladicky, L., and Torr, P. H. Robust higher order potentials for enforcing label consistency.\nIJCV , 2009.\nKokkinos, I., Deriche, R., Faugeras, O., and Maragos, P. Computational analysis and learning for a\nbiologically motivated model of boundary detection. Neurocomputing , 71(10):1798\u20131812, 2008.\nKr\u00a8ahenb \u00a8uhl, P. and Koltun, V . Ef\ufb01cient inference in fully connected crfs with gaussian edge potentials. In NIPS , 2011."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66de-6db4-11ee-a9bb-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-44", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "Kr\u00a8ahenb \u00a8uhl, P. and Koltun, V . Parameter learning and convergent inference for dense random \ufb01elds.\nInICML , 2013.\nKrizhevsky, A., Sutskever, I., and Hinton, G. E. Imagenet classi\ufb01cation with deep convolutional\nneural networks. In NIPS , 2013.\nLadicky, L., Russell, C., Kohli, P., and Torr, P. H. Associative hierarchical crfs for object class image\nsegmentation. In ICCV , 2009.\nLeCun, Y ., Bottou, L., Bengio, Y ., and Haffner, P. Gradient-based learning applied to document\nrecognition. In Proc. IEEE , 1998.\nLempitsky, V ., Vedaldi, A., and Zisserman, A. Pylon model for semantic segmentation. In NIPS ,\n2011.\nLong, J., Shelhamer, E., and Darrell, T. Fully convolutional networks for semantic segmentation.\narXiv:1411.4038 , 2014.\nLucchi, A., Li, Y ., Boix, X., Smith, K., and Fua, P. Are spatial and global constraints really necessary"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66df-6db4-11ee-afeb-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-45", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "Lucchi, A., Li, Y ., Boix, X., Smith, K., and Fua, P. Are spatial and global constraints really necessary\nfor segmentation? In ICCV , 2011.\nMallat, S. A Wavelet Tour of Signal Processing . Acad. Press, 2 edition, 1999.\nMostajabi, M., Yadollahpour, P., and Shakhnarovich, G. Feedforward semantic segmentation with\nzoom-out features. arXiv:1412.0774 , 2014.\nPapandreou, G., Kokkinos, I., and Savalle, P.-A. Untangling local and global deformations in deep\nconvolutional networks for image classi\ufb01cation and sliding window detection. arXiv:1412.0296 ,\n2014.\nPapandreou, G., Chen, L.-C., Murphy, K., and Yuille, A. L. Weakly- and semi-supervised learning\nof a DCNN for semantic image segmentation. arXiv:1502.02734 , 2015.\nRother, C., Kolmogorov, V ., and Blake, A. Grabcut: Interactive foreground extraction using iterated\ngraph cuts. In SIGGRAPH , 2004."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66e0-6db4-11ee-90db-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-46", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "Rother, C., Kolmogorov, V ., and Blake, A. Grabcut: Interactive foreground extraction using iterated\ngraph cuts. In SIGGRAPH , 2004.\nSermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., and LeCun, Y . Overfeat: Integrated\nrecognition, localization and detection using convolutional networks. arXiv:1312.6229 , 2013.\nShotton, J., Winn, J., Rother, C., and Criminisi, A. Textonboost for image understanding: Multiclass object recognition and segmentation by jointly modeling texture, layout, and context. IJCV ,\n2009.\nSimonyan, K. and Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 , 2014.\nSzegedy, C., Liu, W., Jia, Y ., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V ., and\nRabinovich, A. Going deeper with convolutions. arXiv:1409.4842 , 2014.\n13\nPublished as a conference paper at ICLR 2015"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66e1-6db4-11ee-a1d0-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-47", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "Rabinovich, A. Going deeper with convolutions. arXiv:1409.4842 , 2014.\n13\nPublished as a conference paper at ICLR 2015\nTompson, J., Jain, A., LeCun, Y ., and Bregler, C. Joint Training of a Convolutional Network and a\nGraphical Model for Human Pose Estimation. In NIPS , 2014.\nUijlings, J., van de Sande, K., Gevers, T., and Smeulders, A. Selective search for object recognition.\nIJCV , 2013.\nWang, P., Shen, X., Lin, Z., Cohen, S., Price, B., and Yuille, A. Towards uni\ufb01ed depth and semantic\nprediction from a single image. In CVPR , 2015.\nYadollahpour, P., Batra, D., and Shakhnarovich, G. Discriminative re-ranking of diverse segmentations. In CVPR , 2013.\nZeiler, M. D. and Fergus, R. Visualizing and understanding convolutional networks. In ECCV , 2014.\nZhang, N., Donahue, J., Girshick, R., and Darrell, T. Part-based r-cnns for \ufb01ne-grained category\ndetection. In ECCV , 2014."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66e2-6db4-11ee-b536-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1412.7062-48", "title": "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", "source": "http://arxiv.org/pdf/1412.7062", "start_index": 0, "chroma:document": "detection. In ECCV , 2014.\nZheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V ., Su, Z., Du, D., Huang, C., and Torr, P.\nConditional random \ufb01elds as recurrent neural networks. arXiv:1502.03240 , 2015.\n14"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66e3-6db4-11ee-ad78-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-0", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "arXiv:1502.03167v3  [cs.LG]  2 Mar 2015BatchNormalization: AcceleratingDeepNetworkTrainingb y\nReducingInternalCovariateShift\nSergey Ioffe\nGoogleInc., sioffe@google.comChristianSzegedy\nGoogleInc., szegedy@google.com\nAbstract\nTrainingDeepNeuralNetworksiscomplicatedbythefact\nthat the distributionofeach layer\u2019sinputschangesduring\ntraining, as the parametersof the previouslayers change.\nThis slows down the training by requiringlower learning\nratesandcarefulparameterinitialization,andmakesitno toriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate\nshift, and address the problem by normalizing layer inputs. Ourmethoddrawsitsstrengthfrommakingnormalizationapartofthemodelarchitectureandperformingthe\nnormalization for each training mini-batch . Batch Normalizationallowsustousemuchhigherlearningratesand\nbe less careful about initialization. It also acts as a regularizer, in some cases eliminating the need for Dropout.\nApplied to a state-of-the-art image classi\ufb01cation model,\nBatch Normalizationachievesthe same accuracy with 14\ntimes fewer training steps, and beats the original model"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66e4-6db4-11ee-85b8-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-1", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "Batch Normalizationachievesthe same accuracy with 14\ntimes fewer training steps, and beats the original model\nby a signi\ufb01cant margin. Using an ensemble of batchnormalizednetworks,weimproveuponthebestpublished\nresult on ImageNet classi\ufb01cation: reaching 4.9% top-5\nvalidation error (and 4.8% test error), exceeding the accuracyofhumanraters.\n1 Introduction\nDeep learning has dramatically advanced the state of the\nart in vision, speech, and many other areas. Stochastic gradient descent (SGD) has proved to be an effective way of training deep networks, and SGD variants\nsuch as momentum (Sutskeveret al., 2013) and Adagrad\n(Duchiet al.,2011)havebeenusedtoachievestate ofthe\nart performance. SGD optimizes the parameters \u0398of the\nnetwork,soasto minimizetheloss\n\u0398 = argmin\n\u03981\nNN/summationdisplay\ni=1\u2113(xi,\u0398)"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66e5-6db4-11ee-8ce6-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-2", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "network,soasto minimizetheloss\n\u0398 = argmin\n\u03981\nNN/summationdisplay\ni=1\u2113(xi,\u0398)\nwherex1...Nisthetrainingdataset. With SGD,thetrainingproceedsinsteps,andateachstepweconsidera minibatchx1...mofsizem. The mini-batchis usedtoapproximate the gradient of the loss functionwith respect to the\nparameters,bycomputing\n1\nm\u2202\u2113(xi,\u0398)\n\u2202\u0398.Usingmini-batchesofexamples,asopposedtooneexampleatatime,ishelpfulinseveralways. First,thegradient\nofthelossoveramini-batchisanestimateofthegradient\noverthetrainingset, whose qualityimprovesas thebatch\nsize increases. Second, computation over a batch can be\nmuch more ef\ufb01cient than mcomputations for individual\nexamples, due to the parallelism afforded by the modern\ncomputingplatforms.\nWhile stochastic gradient is simple and effective, it\nrequires careful tuning of the model hyper-parameters,\nspeci\ufb01callythelearningrateusedinoptimization,aswell"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66e6-6db4-11ee-b6ac-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-3", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "requires careful tuning of the model hyper-parameters,\nspeci\ufb01callythelearningrateusedinoptimization,aswell\nas the initial values for the model parameters. The trainingiscomplicatedbythefactthattheinputstoeachlayer\nareaffectedbytheparametersofallprecedinglayers\u2013so\nthat small changes to the network parameters amplify as\nthenetworkbecomesdeeper.\nThe change in the distributions of layers\u2019 inputs\npresents a problem because the layers need to continuously adapt to the new distribution. When the input distributiontoalearningsystemchanges,itissaidtoexperiencecovariateshift (Shimodaira, 2000). This is typically\nhandled via domain adaptation (Jiang, 2008). However,\nthe notion of covariate shift can be extended beyond the\nlearningsystemasawhole,toapplytoitsparts,suchasa\nsub-networkora layer. Considera networkcomputing\n\u2113=F2(F1(u,\u03981),\u03982)\nwhereF1andF2are arbitrary transformations, and the\nparameters \u03981,\u03982are to be learned so as to minimize\nthe loss\u2113. Learning \u03982can be viewed as if the inputs\nx =F1(u,\u03981)arefedintothesub-network"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66e7-6db4-11ee-964a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-4", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "the loss\u2113. Learning \u03982can be viewed as if the inputs\nx =F1(u,\u03981)arefedintothesub-network\n\u2113=F2(x,\u03982).\nForexample,agradientdescentstep\n\u03982\u2190\u03982\u2212\u03b1\nmm/summationdisplay\ni=1\u2202F2(xi,\u03982)\n\u2202\u03982\n(forbatchsize mandlearningrate \u03b1)isexactlyequivalent\nto that for a stand-alone network F2with input x. Therefore, the input distribution properties that make training\nmore ef\ufb01cient \u2013 such as having the same distribution between the training and test data \u2013 apply to training the\nsub-network as well. As such it is advantageous for the\ndistributionof xtoremain\ufb01xedovertime. Then, \u03982does\n1\nnot have to readjust to compensate for the change in the\ndistributionof x.\nFixed distribution of inputs to a sub-network would\nhavepositiveconsequencesforthelayers outsidethesubnetwork,as well. Consider a layer with a sigmoid activation function z =g(Wu+b)whereuis the layer input,\nthe weight matrix Wand bias vector bare the layer parameters to be learned, and g(x) =1"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66e8-6db4-11ee-bb3b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-5", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "the weight matrix Wand bias vector bare the layer parameters to be learned, and g(x) =1\n1+exp(\u2212x). As|x|\nincreases, g\u2032(x)tends to zero. This means that for all dimensionsof x =Wu+bexceptthosewithsmallabsolute\nvalues,thegradient\ufb02owingdownto uwillvanishandthe\nmodel will train slowly. However, since xis affected by\nW,band the parameters of all the layers below, changes\ntothoseparametersduringtrainingwilllikelymovemany\ndimensions of xinto the saturated regime of the nonlinearity and slow down the convergence. This effect is\nampli\ufb01ed as the network depth increases. In practice,\nthe saturation problem and the resulting vanishing gradientsareusuallyaddressedbyusingRecti\ufb01edLinearUnits\n(Nair&Hinton, 2010) ReLU(x) = max( x,0), careful\ninitialization (Bengio&Glorot, 2010; Saxeet al., 2013),\nand small learning rates. If, however, we could ensure\nthat the distribution of nonlinearity inputs remains more\nstable as the network trains, then the optimizer would be\nless likely to get stuck in the saturated regime, and the\ntrainingwouldaccelerate.\nWe refer to the change in the distributions of internal"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66e9-6db4-11ee-afbd-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-6", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "less likely to get stuck in the saturated regime, and the\ntrainingwouldaccelerate.\nWe refer to the change in the distributions of internal\nnodes of a deep network, in the course of training, as Internal Covariate Shift . Eliminating it offers a promise of\nfaster training. We propose a new mechanism, which we\ncallBatch Normalization , that takes a step towards reducing internal covariate shift, and in doing so dramatically accelerates the training of deep neural nets. It accomplishes this via a normalization step that \ufb01xes the\nmeansandvariancesoflayerinputs. BatchNormalization\nalso has a bene\ufb01cial effect on the gradient \ufb02ow through\nthe network, by reducing the dependence of gradients\non the scale of the parameters or of their initial values.\nThis allows us to use much higher learning rates without the risk of divergence. Furthermore, batch normalization regularizes the model and reduces the need for\nDropout(Srivastavaet al., 2014). Finally, Batch Normalization makes it possible to use saturating nonlinearities\nby preventingthe network from getting stuck in the saturatedmodes.\nIn Sec. 4.2, we apply Batch Normalization to the bestperforming ImageNet classi\ufb01cation network, and show\nthat we can match its performance using only 7% of the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66ea-6db4-11ee-8373-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-7", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "that we can match its performance using only 7% of the\ntraining steps, and can further exceed its accuracy by a\nsubstantial margin. Using an ensemble of such networks\ntrained with Batch Normalization, we achieve the top-5\nerror rate that improves upon the best known results on\nImageNetclassi\ufb01cation.2 Towards Reducing Internal\nCovariateShift\nWe de\ufb01ne Internal Covariate Shift as the change in the\ndistribution of network activations due to the change in\nnetworkparametersduringtraining. Toimprovethetraining, we seek to reduce the internal covariate shift. By\n\ufb01xingthe distributionof the layer inputs xas the training\nprogresses,weexpecttoimprovethetrainingspeed. Ithas\nbeen long known (LeCunetal., 1998b; Wiesler &Ney,\n2011) that the network training convergesfaster if its inputsarewhitened\u2013i.e.,linearlytransformedtohavezero\nmeansandunitvariances,anddecorrelated. Aseachlayer\nobservestheinputsproducedbythelayersbelow,itwould\nbe advantageousto achieve the same whiteningof the inputsof each layer. By whitening the inputsto each layer,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66eb-6db4-11ee-beb1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-8", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "be advantageousto achieve the same whiteningof the inputsof each layer. By whitening the inputsto each layer,\nwe would take a step towards achieving the \ufb01xed distributions of inputs that would remove the ill effects of the\ninternalcovariateshift.\nWe couldconsiderwhiteningactivationsat everytraining step or at some interval, either by modifying the\nnetwork directly or by changing the parameters of the\noptimization algorithm to depend on the network activation values (Wiesleret al., 2014; Raikoetal., 2012;\nPoveyet al., 2014; Desjardins&Kavukcuoglu). However, if these modi\ufb01cations are interspersed with the optimization steps, then the gradient descent step may attempt to update the parameters in a way that requires\nthe normalization to be updated, which reduces the effect of the gradient step. For example, consider a layer\nwith the input uthat addsthe learned bias b, and normalizes the result by subtracting the mean of the activation\ncomputed over the training data: /hatwidex=x\u2212E[x]where\nx=u+b,X={x1...N}is the set of values of xover\nthe training set, and E [x] =1\nN/summationtextN\ni=1xi. If a gradient"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66ec-6db4-11ee-9c25-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-9", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "the training set, and E [x] =1\nN/summationtextN\ni=1xi. If a gradient\ndescent step ignores the dependence of E [x]onb, then it\nwill update b\u2190b+ \u2206b, where\u2206b\u221d\u2212\u2202\u2113/\u2202/hatwidex. Then\nu+ (b+ \u2206b)\u2212E[u+ (b+ \u2206b)] =u+b\u2212E[u+b].\nThus, the combination of the update to band subsequent\nchange in normalization led to no change in the output\nof the layer nor, consequently, the loss. As the training\ncontinues, bwill grow inde\ufb01nitely while the loss remains\n\ufb01xed. Thisproblemcangetworseifthenormalizationnot\nonly centers but also scales the activations. We have observed this empirically in initial experiments, where the\nmodel blows up when the normalization parameters are\ncomputedoutsidethe gradientdescentstep.\nThe issue with the above approach is that the gradient\ndescent optimization does not take into account the fact\nthat the normalization takes place. To address this issue,\nwe would like to ensure that, for any parameter values,\nthe network alwaysproducesactivationswith the desired\ndistribution. Doing so would allow the gradient of the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66ed-6db4-11ee-a4f2-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-10", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "we would like to ensure that, for any parameter values,\nthe network alwaysproducesactivationswith the desired\ndistribution. Doing so would allow the gradient of the\nloss with respect to the model parameters to account for\nthe normalization, and for its dependence on the model\nparameters \u0398. Let again xbe a layer input, treated as a\n2\nvector, andXbe the set of these inputs over the training\ndataset. Thenormalizationcanthenbewrittenasatransformation\n/hatwidex =Norm(x,X)\nwhich depends not only on the given training example x\nbut on all examples X\u2013 each of which depends on \u0398if\nxis generatedby anotherlayer. For backpropagation,we\nwouldneedtocomputetheJacobians\n\u2202Norm(x,X)\n\u2202xand\u2202Norm(x,X)\n\u2202X;\nignoring the latter term would lead to the explosion describedabove. Withinthisframework,whiteningthelayer\ninputs is expensive, as it requires computing the covariance matrix Cov [x] =Ex\u2208X[xxT]\u2212E[x]E[x]Tand its\ninverse square root, to produce the whitened activations\nCov[x]\u22121/2(x\u2212E[x]), as well as the derivatives of these"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66ee-6db4-11ee-9894-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-11", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "inverse square root, to produce the whitened activations\nCov[x]\u22121/2(x\u2212E[x]), as well as the derivatives of these\ntransformsforbackpropagation.Thismotivatesustoseek\nan alternative that performs input normalization in a way\nthat is differentiable and does not require the analysis of\ntheentiretrainingset aftereveryparameterupdate.\nSome of the previous approaches (e.g.\n(Lyu&Simoncelli, 2008)) use statistics computed\nover a single training example, or, in the case of image\nnetworks, over differentfeature maps at a given location.\nHowever, this changes the representation ability of a\nnetwork by discarding the absolute scale of activations.\nWe want to a preservethe informationin the network,by\nnormalizing the activations in a training example relative\ntothe statisticsoftheentiretrainingdata.\n3 Normalization via Mini-Batch\nStatistics\nSince the full whitening of each layer\u2019s inputs is costly\nand not everywhere differentiable, we make two necessary simpli\ufb01cations. The \ufb01rst is that instead of whitening\nthe features in layer inputs and outputs jointly, we will\nnormalizeeachscalarfeatureindependently,bymakingit\nhave the mean of zero and the variance of 1. For a layer"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66ef-6db4-11ee-a7d3-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-12", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "normalizeeachscalarfeatureindependently,bymakingit\nhave the mean of zero and the variance of 1. For a layer\nwithd-dimensionalinput x = (x(1)...x(d)),wewillnormalizeeachdimension\n/hatwidex(k)=x(k)\u2212E[x(k)]/radicalbig\nVar[x(k)]\nwheretheexpectationandvariancearecomputedoverthe\ntrainingdataset. Asshownin(LeCunetal.,1998b),such\nnormalizationspeedsupconvergence,evenwhenthefeaturesarenotdecorrelated.\nNotethatsimplynormalizingeachinputofalayermay\nchange what the layer can represent. For instance, normalizing the inputsof a sigmoid wouldconstrain them to\nthe linear regime of the nonlinearity. To address this, we\nmakesurethat thetransformationinsertedin thenetwork\ncan represent the identity transform . To accomplish this,weintroduce,foreachactivation x(k),apairofparameters\n\u03b3(k),\u03b2(k),whichscale andshift thenormalizedvalue:\ny(k)=\u03b3(k)/hatwidex(k)+\u03b2(k).\nThese parameters are learned along with the original\nmodel parameters, and restore the representation power"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66f0-6db4-11ee-8b21-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-13", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "y(k)=\u03b3(k)/hatwidex(k)+\u03b2(k).\nThese parameters are learned along with the original\nmodel parameters, and restore the representation power\nofthenetwork. Indeed,bysetting \u03b3(k)=/radicalbig\nVar[x(k)]and\n\u03b2(k)=E[x(k)], we couldrecoverthe originalactivations,\nifthatwerethe optimalthingto do.\nInthebatchsettingwhereeachtrainingstepisbasedon\ntheentire trainingset, we woulduse the wholeset to normalize activations. However,this is impracticalwhen using stochastic optimization. Therefore, we make the secondsimpli\ufb01cation: since we use mini-batchesin stochastic gradient training, each mini-batch produces estimates\nofthemeanandvariance ofeachactivation. Thisway,the\nstatistics used for normalization can fully participate in\nthe gradient backpropagation. Note that the use of minibatchesis enabledbycomputationof per-dimensionvariances rather than joint covariances; in the joint case, regularizationwouldbe requiredsince the mini-batchsize is\nlikely to be smaller than the number of activations being\nwhitened,resultinginsingularcovariancematrices."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66f1-6db4-11ee-aaed-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-14", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "likely to be smaller than the number of activations being\nwhitened,resultinginsingularcovariancematrices.\nConsider a mini-batch Bof sizem. Since the normalization is applied to each activation independently, let us\nfocusonaparticularactivation x(k)andomitkforclarity.\nWe havemvaluesofthisactivationinthemini-batch,\nB={x1...m}.\nLetthenormalizedvaluesbe /hatwidex1...m,andtheirlineartransformationsbe y1...m. We referto thetransform\nBN\u03b3,\u03b2:x1...m\u2192y1...m\nas theBatch Normalizing Transform . We present the BN\nTransforminAlgorithm1. Inthealgorithm, \u01ebisaconstant\naddedtothemini-batchvariancefornumericalstability.\nInput:Valuesof xovera mini-batch: B={x1...m};\nParametersto belearned: \u03b3,\u03b2\nOutput:{yi=BN\u03b3,\u03b2(xi)}\n\u00b5B\u21901\nmm/summationdisplay\ni=1xi // mini-batchmean\n\u03c32\nB\u21901\nmm/summationdisplay\ni=1(xi\u2212\u00b5B)2// mini-batchvariance"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66f2-6db4-11ee-a9e2-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-15", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "\u03c32\nB\u21901\nmm/summationdisplay\ni=1(xi\u2212\u00b5B)2// mini-batchvariance\n/hatwidexi\u2190xi\u2212\u00b5B/radicalbig\n\u03c32\nB+\u01eb// normalize\nyi\u2190\u03b3/hatwidexi+\u03b2\u2261BN\u03b3,\u03b2(xi) // scale andshift\nAlgorithm 1: Batch Normalizing Transform, applied to\nactivation xoveramini-batch.\nTheBNtransformcanbeaddedtoanetworktomanipulate any activation. In the notation y=BN\u03b3,\u03b2(x), we\n3\nindicate that the parameters \u03b3and\u03b2are to be learned,\nbut it should be noted that the BN transform does not\nindependently process the activation in each training example. Rather, BN \u03b3,\u03b2(x)depends both on the training\nexampleand the other examples in the mini-batch . The\nscaled and shifted values yare passed to other network\nlayers. The normalized activations /hatwidexare internal to our\ntransformation, but their presence is crucial. The distributions of values of any /hatwidexhas the expected value of 0\nand the variance of 1, as long as the elements of each\nmini-batch are sampled from the same distribution, and"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66f3-6db4-11ee-ac3c-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-16", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "and the variance of 1, as long as the elements of each\nmini-batch are sampled from the same distribution, and\nif we neglect \u01eb. This can be seen by observing that/summationtextm\ni=1/hatwidexi= 0and1\nm/summationtextm\ni=1/hatwidex2\ni= 1, and taking expectations. Eachnormalizedactivation /hatwidex(k)canbeviewedas\nan input to a sub-network composed of the linear transformy(k)=\u03b3(k)/hatwidex(k)+\u03b2(k), followed by the other processing doneby the originalnetwork. Thesesub-network\ninputs all have \ufb01xed means and variances, and although\nthe jointdistributionofthese normalized /hatwidex(k)canchange\nover the course of training, we expect that the introduction of normalized inputs accelerates the training of the\nsub-networkand,consequently,thenetworkasawhole.\nDuring training we need to backpropagate the gradient of loss \u2113through this transformation,as well as compute the gradients with respect to the parameters of the\nBN transform. We use chainrule,as follows(beforesimpli\ufb01cation):\n\u2202\u2113\n\u2202/hatwidexi=\u2202\u2113\n\u2202yi\u00b7\u03b3"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66f4-6db4-11ee-acd2-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-17", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "\u2202\u2113\n\u2202/hatwidexi=\u2202\u2113\n\u2202yi\u00b7\u03b3\n\u2202\u2113\n\u2202\u03c32\nB=/summationtextm\ni=1\u2202\u2113\n\u2202/hatwidexi\u00b7(xi\u2212\u00b5B)\u00b7\u22121\n2(\u03c32\nB+\u01eb)\u22123/2\n\u2202\u2113\n\u2202\u00b5B=/parenleftbigg/summationtextm\ni=1\u2202\u2113\n\u2202/hatwidexi\u00b7\u22121\u221a\n\u03c32\nB+\u01eb/parenrightbigg\n+\u2202\u2113\n\u2202\u03c32\nB\u00b7/summationtextm\ni=1\u22122(xi\u2212\u00b5B)\nm\n\u2202\u2113\n\u2202xi=\u2202\u2113\n\u2202/hatwidexi\u00b71\u221a\n\u03c32\nB+\u01eb+\u2202\u2113\n\u2202\u03c32\nB\u00b72(xi\u2212\u00b5B)\nm+\u2202\u2113\n\u2202\u00b5B\u00b71\nm\n\u2202\u2113\n\u2202\u03b3=/summationtextm\ni=1\u2202\u2113\n\u2202yi\u00b7/hatwidexi\n\u2202\u2113"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66f5-6db4-11ee-ab5c-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-18", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "\u2202\u2113\n\u2202\u03b3=/summationtextm\ni=1\u2202\u2113\n\u2202yi\u00b7/hatwidexi\n\u2202\u2113\n\u2202\u03b2=/summationtextm\ni=1\u2202\u2113\n\u2202yi\nThus,BNtransformisadifferentiabletransformationthat\nintroduces normalized activations into the network. This\nensures that as the model is training, layers can continue\nlearningoninputdistributionsthatexhibitlessinternal covariate shift, thus accelerating the training. Furthermor e,\nthe learned af\ufb01ne transform applied to these normalized\nactivationsallowstheBNtransformtorepresenttheidentity transformationandpreservesthenetworkcapacity.\n3.1 Training and Inference with BatchNormalizedNetworks\nToBatch-Normalize anetwork,wespecifyasubsetofactivations and insert the BN transform for each of them,\naccording to Alg. 1. Any layer that previously received\nxas the input, now receives BN (x). A model employing\nBatch Normalization can be trained using batch gradient\ndescent,orStochasticGradientDescentwithamini-batch\nsizem >1, or with any of its variants such as Adagrad(Duchiet al.,2011). Thenormalizationofactivationsthat"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66f6-6db4-11ee-938c-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-19", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "sizem >1, or with any of its variants such as Adagrad(Duchiet al.,2011). Thenormalizationofactivationsthat\ndependsonthemini-batchallowsef\ufb01cienttraining,but is\nneithernecessarynordesirableduringinference;wewant\nthe output to depend only on the input, deterministically.\nFor this, once the network has been trained, we use the\nnormalization\n/hatwidex=x\u2212E[x]/radicalbig\nVar[x]+\u01eb\nusing the population, rather than mini-batch, statistics.\nNeglecting \u01eb, these normalized activations have the same\nmean0 and variance1 as duringtraining. We use the unbiased variance estimate Var [x] =m\nm\u22121\u00b7EB[\u03c32\nB], where\ntheexpectationisovertrainingmini-batchesofsize mand\n\u03c32\nBaretheirsamplevariances. Usingmovingaveragesinstead, we can track the accuracy of a model as it trains.\nSincethemeansandvariancesare\ufb01xedduringinference,\nthe normalization is simply a linear transform applied to\neachactivation. Itmayfurtherbecomposedwiththescaling by\u03b3and shift by \u03b2, to yield a single linear transform"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66f7-6db4-11ee-82dd-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-20", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "the normalization is simply a linear transform applied to\neachactivation. Itmayfurtherbecomposedwiththescaling by\u03b3and shift by \u03b2, to yield a single linear transform\nthat replacesBN (x). Algorithm 2 summarizesthe procedurefortrainingbatch-normalizednetworks.\nInput:NetworkNwith trainableparameters \u0398;\nsubsetofactivations {x(k)}K\nk=1\nOutput: Batch-normalizednetworkforinference, Ninf\nBN\n1:Ntr\nBN\u2190N// TrainingBN network\n2:fork= 1...Kdo\n3:Add transformation y(k)=BN\u03b3(k),\u03b2(k)(x(k))to\nNtr\nBN(Alg.1)\n4:Modify each layer in Ntr\nBNwith input x(k)to take\ny(k)instead\n5:end for\n6:TrainNtr\nBNto optimize the parameters \u0398\u222a\n{\u03b3(k),\u03b2(k)}K\nk=1\n7:Ninf\nBN\u2190Ntr\nBN// InferenceBN networkwithfrozen\n// parameters\n8:fork= 1...Kdo\n9:// Forclarity, x\u2261x(k),\u03b3\u2261\u03b3(k),\u00b5B\u2261\u00b5(k)\nB, etc."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66f8-6db4-11ee-ac39-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-21", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "9:// Forclarity, x\u2261x(k),\u03b3\u2261\u03b3(k),\u00b5B\u2261\u00b5(k)\nB, etc.\n10:Process multiple training mini-batches B, each of\nsizem,andaverageoverthem:\nE[x]\u2190EB[\u00b5B]\nVar[x]\u2190m\nm\u22121EB[\u03c32\nB]\n11:InNinf\nBN, replace the transform y=BN\u03b3,\u03b2(x)with\ny=\u03b3\u221a\nVar[x]+\u01eb\u00b7x+/parenleftbig\n\u03b2\u2212\u03b3E[x]\u221a\nVar[x]+\u01eb/parenrightbig\n12:end for\nAlgorithm2: Traininga Batch-NormalizedNetwork\n3.2 Batch-Normalized Convolutional Networks\nBatch Normalization can be applied to any set of activations in the network. Here, we focus on transforms\n4\nthat consist of an af\ufb01ne transformation followed by an\nelement-wisenonlinearity:\nz =g(Wu+b)\nwhereWandbare learned parametersof the model, and\ng(\u00b7)isthenonlinearitysuchassigmoidorReLU.Thisformulation covers both fully-connected and convolutional\nlayers. We add the BN transform immediately before the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66f9-6db4-11ee-9051-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-22", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "layers. We add the BN transform immediately before the\nnonlinearity,bynormalizing x =Wu+b. Wecouldhave\nalso normalized the layer inputs u, but since uis likely\nthe output of another nonlinearity, the shape of its distributionislikelytochangeduringtraining,andconstrainin g\nits \ufb01rst and second moments would not eliminate the covariate shift. In contrast, Wu + bis more likely to have\na symmetric,non-sparsedistribution,that is \u201cmoreGaussian\u201d(Hyv\u00a8 arinen&Oja,2000);normalizingitislikelyto\nproduceactivationswithastable distribution.\nNotethat,sincewenormalize Wu+b,thebiasbcanbe\nignoredsinceitseffectwillbecanceledbythesubsequent\nmeansubtraction(theroleofthebiasissubsumedby \u03b2in\nAlg.1). Thus, z =g(Wu+b)is replacedwith\nz =g(BN(Wu))\nwhere the BN transformis applied independentlyto each\ndimension of x =Wu, with a separate pair of learned\nparameters \u03b3(k),\u03b2(k)perdimension.\nForconvolutionallayers,we additionallywant the normalization to obey the convolutional property \u2013 so that"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66fa-6db4-11ee-a2b3-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-23", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "parameters \u03b3(k),\u03b2(k)perdimension.\nForconvolutionallayers,we additionallywant the normalization to obey the convolutional property \u2013 so that\ndifferent elements of the same feature map, at different\nlocations, are normalized in the same way. To achieve\nthis, we jointly normalize all the activations in a minibatch, overall locations. In Alg. 1, we let Bbe the set of\nall values in a feature map across both the elements of a\nmini-batch and spatial locations \u2013 so for a mini-batch of\nsizemand feature maps of size p\u00d7q, we use the effective mini-batch of size m\u2032=|B|=m\u00b7pq. We learn a\npair of parameters \u03b3(k)and\u03b2(k)per feature map, rather\nthan per activation. Alg. 2 is modi\ufb01ed similarly, so that\nduringinferencetheBNtransformappliesthesamelinear\ntransformationtoeachactivationina givenfeaturemap.\n3.3 Batch Normalization enables higher\nlearning rates\nIn traditional deep networks, too-high learning rate may\nresult in the gradients that explode or vanish, as well as\ngetting stuck in poor local minima. Batch Normalization helps address these issues. By normalizing activations throughout the network, it prevents small changes"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66fb-6db4-11ee-b6d1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-24", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "getting stuck in poor local minima. Batch Normalization helps address these issues. By normalizing activations throughout the network, it prevents small changes\nto the parameters from amplifying into larger and suboptimal changes in activations in gradients; for instance, it\nprevents the training from getting stuck in the saturated\nregimesofnonlinearities.\nBatchNormalizationalsomakestrainingmoreresilient\ntotheparameterscale. Normally,largelearningratesmay\nincreasethescaleoflayerparameters,whichthenamplifythegradientduringbackpropagationandleadtothemodel\nexplosion. However, with Batch Normalization, backpropagation through a layer is unaffected by the scale of\nitsparameters. Indeed,fora scalar a,\nBN(Wu) =BN((aW)u)\nandwe canshowthat\n\u2202BN((aW)u)\n\u2202u=\u2202BN(Wu)\n\u2202u\n\u2202BN((aW)u)\n\u2202(aW)=1\na\u00b7\u2202BN(Wu)\n\u2202W\nThe scale does not affect the layer Jacobian nor, consequently, the gradient propagation. Moreover, larger\nweights lead to smallergradients, and Batch Normalizationwill stabilize theparametergrowth.\nWe further conjecture that Batch Normalization may"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66fc-6db4-11ee-81f7-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-25", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "weights lead to smallergradients, and Batch Normalizationwill stabilize theparametergrowth.\nWe further conjecture that Batch Normalization may\nleadthelayerJacobianstohavesingularvaluescloseto1,\nwhich is known to be bene\ufb01cial for training (Saxeet al.,\n2013). Consider two consecutive layers with normalized\ninputs, and the transformation between these normalized\nvectors:/hatwidez =F(/hatwidex). Ifweassumethat /hatwidexand/hatwidezareGaussian\nanduncorrelated,andthat F(/hatwidex)\u2248J/hatwidexisalineartransformationforthe givenmodelparameters,thenboth /hatwidexand/hatwidez\nhave unit covariances, and I=Cov[/hatwidez] =JCov[/hatwidex]JT=\nJJT. Thus,JJT=I, and so all singular values of J\nare equal to 1, which preserves the gradient magnitudes\nduring backpropagation. In reality, the transformation is\nnotlinear,andthenormalizedvaluesarenotguaranteedto\nbe Gaussian nor independent, but we nevertheless expect\nBatch Normalization to help make gradient propagation\nbetter behaved. The precise effect of Batch Normalization on gradient propagation remains an area of further\nstudy."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66fd-6db4-11ee-b226-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-26", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "Batch Normalization to help make gradient propagation\nbetter behaved. The precise effect of Batch Normalization on gradient propagation remains an area of further\nstudy.\n3.4 Batch Normalization regularizes the\nmodel\nWhen training with Batch Normalization, a training example is seen in conjunction with other examples in the\nmini-batch, and the training network no longer producing deterministic values for a given training example. In\nour experiments,we foundthis effect to be advantageous\nto the generalization of the network. Whereas Dropout\n(Srivastavaet al., 2014) is typically used to reduce over\ufb01tting,inabatch-normalizednetworkwefoundthatitcan\nbeeitherremovedorreducedinstrength.\n4 Experiments\n4.1 Activationsovertime\nTo verify the effects of internal covariate shift on training, and the ability of Batch Normalization to combat it,\nweconsideredtheproblemofpredictingthedigitclasson\ntheMNISTdataset(LeCunetal.,1998a). Weusedavery\nsimple network, with a 28x28binary image as input, and\n5\n10K20K30K40K50K0.70.80.91\n  \nWithout BN\nWith BN\n\u2212202\n\u2212202\n(a) (b)WithoutBN (c)With BN"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66fe-6db4-11ee-83c6-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-27", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "Without BN\nWith BN\n\u2212202\n\u2212202\n(a) (b)WithoutBN (c)With BN\nFigure 1: (a) The test accuracy of the MNIST network\ntrained with and without Batch Normalization, vs. the\nnumber of training steps. Batch Normalization helps the\nnetwork train faster and achieve higher accuracy. (b,\nc)The evolution of input distributions to a typical sigmoid,overthecourseoftraining,shownas {15,50,85}th\npercentiles. Batch Normalization makes the distribution\nmorestableandreducestheinternalcovariateshift.\n3fully-connectedhiddenlayerswith100activationseach.\nEachhiddenlayercomputes y =g(Wu+b)withsigmoid\nnonlinearity, and the weights Winitialized to small random Gaussian values. The last hidden layer is followed\nby a fully-connected layer with 10 activations (one per\nclass) and cross-entropyloss. We trained the network for\n50000steps, with 60 examplespermini-batch. We added\nBatchNormalizationtoeachhiddenlayerofthenetwork,\nas in Sec. 3.1. We were interested in the comparison betweenthebaselineandbatch-normalizednetworks,rather\nthanachievingthestateoftheartperformanceonMNIST"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a66ff-6db4-11ee-98ac-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-28", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "thanachievingthestateoftheartperformanceonMNIST\n(whichthe describedarchitecturedoesnot).\nFigure 1(a) shows the fraction of correct predictions\nby the two networks on held-out test data, as training\nprogresses. The batch-normalized network enjoys the\nhigher test accuracy. To investigate why, we studied inputs to the sigmoid, in the original network Nand batchnormalizednetwork Ntr\nBN(Alg.2)overthecourseoftraining. InFig.1(b,c)weshow,foronetypicalactivationfrom\nthe last hidden layer of each network, how its distribution evolves. The distributions in the original network\nchange signi\ufb01cantly over time, both in their mean and\nthe variance, which complicates the training of the subsequent layers. In contrast, the distributions in the batch normalizednetworkaremuchmorestableastrainingprogresses,whichaidsthe training.\n4.2 ImageNetclassi\ufb01cation\nWe applied Batch Normalization to a new variant of the\nInception network (Szegedyetal., 2014), trained on the\nImageNet classi\ufb01cation task (Russakovskyet al., 2014).\nThe network has a large number of convolutional and\npooling layers, with a softmax layer to predict the image\nclass, out of 1000 possibilities. Convolutional layers use"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6700-6db4-11ee-b03c-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-29", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "The network has a large number of convolutional and\npooling layers, with a softmax layer to predict the image\nclass, out of 1000 possibilities. Convolutional layers use\nReLU asthenonlinearity. Themaindifferenceto thenetwork described in (Szegedyet al., 2014) is that the 5\u00d75\nconvolutionallayersare replacedby two consecutivelayers of3\u00d73convolutionswith up to 128\ufb01lters. The network contains 13.6\u00b7106parameters, and, other than the\ntop softmax layer, has no fully-connected layers. Moredetails are given in the Appendix. We refer to this model\nasInception intherestofthetext. Themodelwastrained\nusing a version of Stochastic Gradient Descent with momentum(Sutskeveretal.,2013),usingthemini-batchsize\nof32. Thetrainingwasperformedusingalarge-scale,distributed architecture (similar to (Deanet al., 2012)). All\nnetworksare evaluatedastrainingprogressesbycomputing the validation accuracy @1, i.e. the probability of\npredicting the correct label out of 1000 possibilities, on\naheld-outset,usinga singlecropperimage.\nInourexperiments,weevaluatedseveralmodi\ufb01cations"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6701-6db4-11ee-b1dd-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-30", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "aheld-outset,usinga singlecropperimage.\nInourexperiments,weevaluatedseveralmodi\ufb01cations\nofInceptionwithBatchNormalization. Inallcases,Batch\nNormalizationwasappliedtotheinputofeachnonlinearity, in a convolutional way, as described in section 3.2,\nwhilekeepingtherestofthearchitectureconstant.\n4.2.1 AcceleratingBN Networks\nSimplyaddingBatchNormalizationtoanetworkdoesnot\ntake full advantage of our method. To do so, we further\nchanged the network and its training parameters, as follows:\nIncrease learning rate. In a batch-normalized model,\nwe have been able to achieve a training speedup from\nhigherlearningrates,with noill sideeffects(Sec.3.3).\nRemoveDropout. As describedin Sec. 3.4, Batch Normalizationful\ufb01llssomeofthesamegoalsasDropout. Removing Dropout from Modi\ufb01ed BN-Inception speeds up\ntraining,withoutincreasingover\ufb01tting.\nReduce the L2weight regularization. While in Inception anL2loss on the model parameters controls over\ufb01tting, in Modi\ufb01ed BN-Inception the weight of this loss is"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6702-6db4-11ee-a99f-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-31", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "reduced by a factor of 5. We \ufb01nd that this improves the\naccuracyontheheld-outvalidationdata.\nAccelerate the learning rate decay. In training Inception, learning rate was decayed exponentially. Because\nour network trains faster than Inception, we lower the\nlearningrate 6timesfaster.\nRemove Local Response Normalization While Inception and other networks (Srivastavaet al., 2014) bene\ufb01t\nfrom it, we found that with Batch Normalization it is not\nnecessary.\nShuf\ufb02etrainingexamplesmorethoroughly. Weenabled\nwithin-shardshuf\ufb02ingofthetrainingdata,whichprevents\nthesameexamplesfromalwaysappearinginamini-batch\ntogether. This led to about 1% improvements in the validation accuracy, which is consistent with the view of\nBatch Normalization as a regularizer (Sec. 3.4): the randomization inherent in our method should be most bene\ufb01cialwhenitaffectsanexampledifferentlyeachtimeitis\nseen.\nReduce the photometric distortions. Because batchnormalized networks train faster and observe each trainingexamplefewertimes,weletthetrainerfocusonmore\n\u201creal\u201dimagesbydistortingthemless.\n6"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6703-6db4-11ee-828a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-32", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "\u201creal\u201dimagesbydistortingthemless.\n6\n5M 10M 15M 20M 25M 30M0.40.50.60.70.8\nInception\nBN\u2212Baseline\nBN\u2212x5\nBN\u2212x30\nBN\u2212x5\u2212Sigmoid\nSteps to match Inception\nFigure 2: Single crop validation accuracy of Inception\nand its batch-normalized variants, vs. the number of\ntrainingsteps.Model Stepsto72.2% Maxaccuracy\nInception 31.0\u00b710672.2%\nBN-Baseline 13.3\u00b710672.7%\nBN-x5 2.1\u00b710673.0%\nBN-x30 2.7\u00b710674.8%\nBN-x5-Sigmoid 69.8%\nFigure 3: For Inception and the batch-normalized\nvariants, the number of training steps required to\nreach the maximum accuracy of Inception(72.2%),\nand the maximum accuracy achieved by the network.\n4.2.2 Single-NetworkClassi\ufb01cation\nWe evaluated the following networks, all trained on the\nLSVRC2012 training data, and tested on the validation\ndata:\nInception : the network described at the beginning of\nSection4.2,trainedwiththeinitiallearningrateof0.001 5."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6704-6db4-11ee-ae39-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-33", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "data:\nInception : the network described at the beginning of\nSection4.2,trainedwiththeinitiallearningrateof0.001 5.\nBN-Baseline : Same as Inception with Batch Normalizationbeforeeachnonlinearity.\nBN-x5: Inception with Batch Normalization and the\nmodi\ufb01cations in Sec. 4.2.1. The initial learning rate was\nincreased by a factor of 5, to 0.0075. The same learning\nrateincreasewithoriginalInceptioncausedthemodelparameterstoreachmachinein\ufb01nity.\nBN-x30: LikeBN-x5, but with the initial learning rate\n0.045(30timesthatofInception).\nBN-x5-Sigmoid : LikeBN-x5, but with sigmoid nonlinearityg(t) =1\n1+exp(\u2212x)instead of ReLU. We also attempted to train the original Inception with sigmoid, but\nthemodelremainedat theaccuracyequivalenttochance.\nIn Figure 2, we show the validation accuracy of the\nnetworks, as a function of the number of training steps.\nInception reached the accuracy of 72.2% after 31\u00b7106\ntraining steps. The Figure 3 shows, for each network,\nthe number of training steps required to reach the same"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6705-6db4-11ee-8671-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-34", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "Inception reached the accuracy of 72.2% after 31\u00b7106\ntraining steps. The Figure 3 shows, for each network,\nthe number of training steps required to reach the same\n72.2%accuracy,aswellasthemaximumvalidationaccuracy reached by the network and the number of steps to\nreachit.\nBy onlyusingBatch Normalization( BN-Baseline ),we\nmatchtheaccuracyofInceptioninlessthanhalfthenumber of training steps. By applying the modi\ufb01cations in\nSec. 4.2.1, we signi\ufb01cantly increase the training speed of\nthe network. BN-x5needs 14 times fewer steps than Inception to reach the 72.2% accuracy. Interestingly, increasing the learning rate further ( BN-x30) causes the\nmodel to train somewhat slowerinitially, but allows it to\nreachahigher\ufb01nalaccuracy. Itreaches74.8%after 6\u00b7106\nsteps, i.e. 5 times fewer steps than required by Inception\ntoreach72.2%.\nWe also veri\ufb01ed that the reduction in internal covariate shift allows deep networks with Batch Normalizationto be trained when sigmoid is used as the nonlinearity,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6706-6db4-11ee-8c6d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-35", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "We also veri\ufb01ed that the reduction in internal covariate shift allows deep networks with Batch Normalizationto be trained when sigmoid is used as the nonlinearity,\ndespite the well-known dif\ufb01culty of training such networks. Indeed, BN-x5-Sigmoid achieves the accuracy of\n69.8%. WithoutBatchNormalization,Inceptionwithsigmoidneverachievesbetterthan 1/1000accuracy.\n4.2.3 Ensemble Classi\ufb01cation\nThe current reported best results on the ImageNet Large\nScale Visual RecognitionCompetitionare reachedby the\nDeep Image ensemble of traditional models (Wuet al.,\n2015) and the ensemble model of (Heet al., 2015). The\nlatterreportsthetop-5errorof4.94%,asevaluatedbythe\nILSVRCserver. Herewereportatop-5validationerrorof\n4.9%, and test error of 4.82% (according to the ILSVRC\nserver). This improvesupon the previousbest result, and\nexceedstheestimatedaccuracyofhumanratersaccording\nto(Russakovskyet al.,2014).\nForourensemble,weused6networks. Eachwasbased\nonBN-x30,modi\ufb01edviasomeofthefollowing: increased"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6707-6db4-11ee-9c4f-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-36", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "Forourensemble,weused6networks. Eachwasbased\nonBN-x30,modi\ufb01edviasomeofthefollowing: increased\ninitial weights in the convolutionallayers; using Dropout\n(with the Dropout probability of 5% or 10%, vs. 40%\nfor the original Inception); and using non-convolutional,\nper-activation Batch Normalization with last hidden layers of the model. Each network achieved its maximum\naccuracyafter about 6\u00b7106training steps. The ensemble\nprediction was based on the arithmetic average of class\nprobabilities predicted by the constituent networks. The\ndetailsofensembleandmulticropinferencearesimilarto\n(Szegedyet al., 2014).\nWe demonstrate in Fig. 4 that batch normalization allowsusto set new state-of-the-artby a healthymarginon\ntheImageNetclassi\ufb01cationchallengebenchmarks.\n5 Conclusion\nWe have presented a novel mechanism for dramatically\naccelerating the training of deep networks. It is based on\nthe premise that covariate shift, which is known to complicate the trainingof machine learning systems, also ap7\nModel Resolution Crops Models Top-1error Top-5error\nGoogLeNetensemble 224 144 7 - 6.67%"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6708-6db4-11ee-aedb-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-37", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "Model Resolution Crops Models Top-1error Top-5error\nGoogLeNetensemble 224 144 7 - 6.67%\nDeepImagelow-res 256 - 1 - 7.96%\nDeepImagehigh-res 512 - 1 24.88 7.42%\nDeepImageensemble variable - - - 5.98%\nBN-Inceptionsinglecrop 224 1 1 25.2% 7.82%\nBN-Inceptionmulticrop 224 144 1 21.99% 5.82%\nBN-Inceptionensemble 224 144 6 20.1% 4.9%*\nFigure 4: Batch-Normalized Inception comparison with previous stat e of the art on the provided validation set comprising50000images. *BN-Inceptionensemblehasreached4 .82%top-5erroronthe100000imagesofthetestsetof\ntheImageNetasreportedbythe test server.\nplies to sub-networks and layers, and removing it from\ninternal activations of the network may aid in training.\nOur proposed method draws its power from normalizing\nactivations, and from incorporating this normalization in\nthe network architecture itself. This ensures that the normalization is appropriately handled by any optimization\nmethod that is being used to train the network. To enable stochastic optimization methods commonly used in\ndeep network training, we perform the normalization for"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6709-6db4-11ee-9f38-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-38", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "method that is being used to train the network. To enable stochastic optimization methods commonly used in\ndeep network training, we perform the normalization for\neachmini-batch,andbackpropagatethegradientsthrough\nthe normalization parameters. Batch Normalization adds\nonly two extra parameters per activation, and in doing so\npreserves the representation ability of the network. We\npresentedanalgorithmforconstructing,training,andper forming inference with batch-normalized networks. The\nresulting networks can be trained with saturating nonlinearities, are more tolerant to increased training rates, an d\noftendonotrequireDropoutforregularization.\nMerely adding Batch Normalization to a state-of-theartimageclassi\ufb01cationmodelyieldsasubstantialspeedup\nin training. By further increasing the learning rates, removing Dropout, and applying other modi\ufb01cations afforded by Batch Normalization, we reach the previous\nstate of the art with onlya small fractionof trainingsteps\n\u2013andthenbeatthestateoftheartinsingle-networkimage\nclassi\ufb01cation. Furthermore, by combining multiple models trained with Batch Normalization, we perform better\nthanthebestknownsystemonImageNet,byasigni\ufb01cant\nmargin."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a670a-6db4-11ee-bc29-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-39", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "thanthebestknownsystemonImageNet,byasigni\ufb01cant\nmargin.\nInterestingly, our method bears similarity to the standardization layer of (G\u00a8 ulc \u00b8ehre& Bengio, 2013), though\nthe two methodsstem from very differentgoals, and perform different tasks. The goal of Batch Normalization\nis to achieve a stable distribution of activation values\nthroughout training, and in our experiments we apply it\nbefore the nonlinearity since that is where matching the\n\ufb01rst and second moments is more likely to result in a\nstable distribution. On the contrary,(G\u00a8 ulc \u00b8ehre&Bengio ,\n2013) apply the standardizationlayer to the outputof the\nnonlinearity, which results in sparser activations. In our\nlarge-scaleimage classi\ufb01cation experiments,we havenot\nobservedthenonlinearity inputstobesparse,neitherwith\nnor without Batch Normalization. Other notable differ-entiating characteristics of Batch Normalization include\nthe learned scale and shift that allow the BN transform\nto representidentity (the standardizationlayer did not re quirethissinceitwasfollowedbythelearnedlineartransform that, conceptually, absorbs the necessary scale and"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a670b-6db4-11ee-9c09-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-40", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "to representidentity (the standardizationlayer did not re quirethissinceitwasfollowedbythelearnedlineartransform that, conceptually, absorbs the necessary scale and\nshift), handling of convolutional layers, deterministic i nferencethatdoesnotdependonthemini-batch,andbatchnormalizingeachconvolutionallayerin thenetwork.\nIn this work, we have not explored the full range of\npossibilitiesthatBatchNormalizationpotentiallyenabl es.\nOur future work includes applications of our method to\nRecurrent Neural Networks (Pascanuet al., 2013), where\ntheinternalcovariateshiftandthevanishingorexploding\ngradients may be especially severe, and which would allowustomorethoroughlytestthehypothesisthatnormalizationimprovesgradientpropagation(Sec.3.3). Weplan\ntoinvestigatewhetherBatchNormalizationcanhelpwith\ndomain adaptation, in its traditional sense \u2013 i.e. whether\nthe normalization performed by the network would allow it to more easily generalize to new data distributions,perhapswithjustarecomputationofthepopulation\nmeansandvariances(Alg.2). Finally,webelievethatfurthertheoreticalanalysisofthealgorithmwouldallowstil l\nmoreimprovementsandapplications.\nReferences"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a670c-6db4-11ee-bc2e-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-41", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "moreimprovementsandapplications.\nReferences\nBengio, Yoshua and Glorot, Xavier. Understanding the\ndif\ufb01cultyoftrainingdeepfeedforwardneuralnetworks.\nInProceedings of AISTATS 2010 , volume 9, pp. 249\u2013\n256,May2010.\nDean,Jeffrey,Corrado,GregS.,Monga,Rajat,Chen,Kai,\nDevin,Matthieu,Le,QuocV., Mao,MarkZ.,Ranzato,\nMarc\u2019Aurelio,Senior,Andrew,Tucker,Paul,Yang,Ke,\nand Ng, Andrew Y. Large scale distributed deep networks. In NIPS,2012.\nDesjardins, Guillaume and Kavukcuoglu,Koray. Natural\nneuralnetworks. (unpublished).\nDuchi, John, Hazan, Elad, and Singer, Yoram. Adaptive\nsubgradientmethodsfor onlinelearning and stochastic\n8\noptimization. J.Mach.Learn.Res. ,12:2121\u20132159,July\n2011. ISSN1532-4435.\nG\u00a8 ulc \u00b8ehre, C \u00b8aglar and Bengio, Yoshua. Knowledge matters: Importanceof prior informationfor optimization."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a670d-6db4-11ee-b943-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-42", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "G\u00a8 ulc \u00b8ehre, C \u00b8aglar and Bengio, Yoshua. Knowledge matters: Importanceof prior informationfor optimization.\nCoRR,abs/1301.4083,2013.\nHe, K., Zhang, X., Ren, S., and Sun, J. Delving Deep\ninto Recti\ufb01ers: Surpassing Human-Level Performance\non ImageNet Classi\ufb01cation. ArXiv e-prints , February\n2015.\nHyv\u00a8 arinen, A. and Oja, E. Independentcomponent analysis: Algorithms and applications. Neural Netw. , 13\n(4-5):411\u2013430,May2000.\nJiang, Jing. A literature survey on domain adaptation of\nstatistical classi\ufb01ers, 2008.\nLeCun, Y., Bottou, L., Bengio, Y., and Haffner, P.\nGradient-based learning applied to document recognition.Proceedings of the IEEE , 86(11):2278\u20132324,\nNovember1998a.\nLeCun, Y., Bottou, L., Orr, G., and Muller, K. Ef\ufb01cient\nbackprop. InOrr,G.andK.,Muller(eds.), NeuralNetworks: Tricks ofthetrade .Springer,1998b."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a670e-6db4-11ee-8b13-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-43", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "backprop. InOrr,G.andK.,Muller(eds.), NeuralNetworks: Tricks ofthetrade .Springer,1998b.\nLyu, S and Simoncelli, E P. Nonlinear image representation using divisive normalization. In Proc. Computer\nVision and Pattern Recognition , pp. 1\u20138. IEEE Computer Society, Jun 23-28 2008. doi: 10.1109/CVPR.\n2008.4587821.\nNair,VinodandHinton,GeoffreyE. Recti\ufb01edlinearunits\nimprove restricted boltzmann machines. In ICML, pp.\n807\u2013814.Omnipress,2010.\nPascanu, Razvan, Mikolov, Tomas, and Bengio, Yoshua.\nOn the dif\ufb01culty of training recurrent neural networks.\nInProceedingsofthe30thInternationalConferenceon\nMachineLearning,ICML 2013,Atlanta,GA,USA,1621June2013 ,pp.1310\u20131318,2013.\nPovey, Daniel, Zhang, Xiaohui, and Khudanpur, Sanjeev. Parallel training of deep neural networks with\nnatural gradient and parameter averaging. CoRR,\nabs/1410.7455,2014."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a670f-6db4-11ee-9c71-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-44", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "natural gradient and parameter averaging. CoRR,\nabs/1410.7455,2014.\nRaiko, Tapani, Valpola, Harri, and LeCun, Yann. Deep\nlearning made easier by linear transformations in perceptrons. In International Conference on Arti\ufb01cial IntelligenceandStatistics(AISTATS) ,pp.924\u2013932,2012.\nRussakovsky,Olga,Deng,Jia,Su,Hao,Krause,Jonathan,\nSatheesh, Sanjeev, Ma, Sean, Huang, Zhiheng, Karpathy, Andrej, Khosla, Aditya, Bernstein, Michael, Berg,\nAlexander C., and Fei-Fei, Li. ImageNet Large Scale\nVisual RecognitionChallenge,2014.Saxe, Andrew M., McClelland, James L., and Ganguli,\nSurya. Exact solutions to the nonlinear dynamics\nof learning in deep linear neural networks. CoRR,\nabs/1312.6120,2013.\nShimodaira, Hidetoshi. Improving predictive inference\nunder covariate shift by weighting the log-likelihood\nfunction. JournalofStatisticalPlanningandInference ,\n90(2):227\u2013244,October2000."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6710-6db4-11ee-af02-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-45", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "under covariate shift by weighting the log-likelihood\nfunction. JournalofStatisticalPlanningandInference ,\n90(2):227\u2013244,October2000.\nSrivastava, Nitish, Hinton, Geoffrey, Krizhevsky, Alex,\nSutskever, Ilya, and Salakhutdinov, Ruslan. Dropout:\nA simple way to preventneural networksfrom over\ufb01tting.J. Mach. Learn. Res. , 15(1):1929\u20131958, January\n2014.\nSutskever, Ilya, Martens, James, Dahl, George E., and\nHinton, Geoffrey E. On the importance of initialization and momentum in deep learning. In ICML\n(3), volume 28 of JMLR Proceedings , pp. 1139\u20131147.\nJMLR.org,2013.\nSzegedy, Christian, Liu, Wei, Jia, Yangqing, Sermanet,\nPierre, Reed, Scott, Anguelov, Dragomir, Erhan, Dumitru, Vanhoucke, Vincent, and Rabinovich, Andrew. Going deeper with convolutions. CoRR,\nabs/1409.4842,2014."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6711-6db4-11ee-a174-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-46", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "abs/1409.4842,2014.\nWiesler, Simon and Ney, Hermann. A convergenceanalysis of log-lineartraining. In Shawe-Taylor,J., Zemel,\nR.S.,Bartlett,P.,Pereira,F.C.N.,andWeinberger,K.Q.\n(eds.),AdvancesinNeuralInformationProcessingSystems24,pp.657\u2013665,Granada,Spain,December2011.\nWiesler, Simon, Richard, Alexander, Schl\u00a8 uter, Ralf, and\nNey, Hermann. Mean-normalized stochastic gradient\nfor large-scale deep learning. In IEEE International\nConference on Acoustics, Speech, and Signal Processing,pp.180\u2013184,Florence,Italy,May2014.\nWu, Ren, Yan, Shengen, Shan, Yi, Dang, Qingqing, and\nSun,Gang. Deepimage: Scalingupimagerecognition,\n2015.\nAppendix\nVariantofthe Inception Model Used\nFigure 5 documents the changes that were performed\ncompared to the architecture with respect to the\nGoogleNet archictecture. For the interpretation of this\ntable, please consult (Szegedyetal., 2014). The notable\narchitecture changes compared to the GoogLeNet model\ninclude:"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6712-6db4-11ee-a617-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-47", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "table, please consult (Szegedyetal., 2014). The notable\narchitecture changes compared to the GoogLeNet model\ninclude:\n\u2022The 5\u00d75 convolutional layers are replaced by two\nconsecutive 3\u00d73 convolutional layers. This increases the maximum depth of the network by 9\n9\nweight layers. Also it increases the number of parameters by 25% and the computational cost is increasedbyabout30%.\n\u2022The number 28\u00d728 inception modules is increased\nfrom2to 3.\n\u2022Inside the modules, sometimes average, sometimes\nmaximum-poolingis employed. This is indicated in\ntheentriescorrespondingtothepoolinglayersofthe\ntable.\n\u2022There are no across the board pooling layers between any two Inception modules, but stride-2 convolution/pooling layers are employed before the \ufb01lterconcatenationin themodules3c,4e.\nOur model employed separable convolution with depth\nmultiplier 8on the \ufb01rst convolutionallayer. This reduces\nthecomputationalcost while increasingthememoryconsumptionat trainingtime.\n10\ntypepatch size/\nstrideoutput\nsizedepth #1\u00d71#3\u00d73\nreduce#3\u00d73double#3\u00d73\nreducedouble\n#3\u00d73Pool+proj"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6713-6db4-11ee-9988-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-48", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "strideoutput\nsizedepth #1\u00d71#3\u00d73\nreduce#3\u00d73double#3\u00d73\nreducedouble\n#3\u00d73Pool+proj\nconvolution* 7\u00d77/2112\u00d7112\u00d7641\nmaxpool 3\u00d73/256\u00d756\u00d764 0\nconvolution 3\u00d73/156\u00d756\u00d7192 1 64 192\nmaxpool 3\u00d73/228\u00d728\u00d7192 0\ninception (3a) 28\u00d728\u00d7256 364 64 64 64 96 avg+ 32\ninception (3b) 28\u00d728\u00d7320 364 64 96 64 96 avg+ 64\ninception (3c) stride 2 28\u00d728\u00d7576 3 0128 160 64 96max +pass through\ninception (4a) 14\u00d714\u00d7576 3224 64 96 96 128 avg+ 128\ninception (4b) 14\u00d714\u00d7576 3192 96 128 96 128 avg+ 128\ninception (4c) 14\u00d714\u00d7576 3160 128 160 128 160 avg+ 128\ninception (4d) 14\u00d714\u00d7576 396 128 192 160 192 avg+ 128\ninception (4e) stride 2 14\u00d714\u00d71024 3 0128 192 192 256max +pass through\ninception (5a) 7\u00d77\u00d71024 3352 192 320 160 224 avg+ 128"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6714-6db4-11ee-9be4-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1502.03167-49", "title": "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", "source": "http://arxiv.org/pdf/1502.03167", "start_index": 0, "chroma:document": "inception (5a) 7\u00d77\u00d71024 3352 192 320 160 224 avg+ 128\ninception (5b) 7\u00d77\u00d71024 3352 192 320 192 224 max+ 128\navgpool 7\u00d77/11\u00d71\u00d71024 0\nFigure5: Inceptionarchitecture\n11"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6715-6db4-11ee-a822-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-0", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "1\nMultiscale Combinatorial Grouping\nfor Image Segmentation and\nObject Proposal Generation\nJordi Pont-Tuset*, Pablo Arbel \u00b4aez*, Jonathan T. Barron, Member, IEEE,\nFerran Marques, Senior Member, IEEE, Jitendra Malik, Fellow, IEEE\nAbstract \u2014We propose a uni\ufb01ed approach for bottom-up hierarchical image segmentation and object proposal generation for\nrecognition, called Multiscale Combinatorial Grouping (MCG). For this purpose, we \ufb01rst develop a fast normalized cuts algorithm.\nWe then propose a high-performance hierarchical segmenter that makes effective use of multiscale information. Finally, we propose a\ngrouping strategy that combines our multiscale regions into highly-accurate object proposals by exploring ef\ufb01ciently their combinatorial\nspace. We also present Single-scale Combinatorial Grouping (SCG), a faster version of MCG that produces competitive proposals in\nunder \ufb01ve second per image. We conduct an extensive and comprehensive empirical validation on the BSDS500, SegVOC12, SBD,\nand COCO datasets, showing that MCG produces state-of-the-art contours, hierarchical regions, and object proposals.\nIndex Terms \u2014Image segmentation, object proposals, normalized cuts.\nF\n1 I NTRODUCTION"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6716-6db4-11ee-b8a1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-1", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "Index Terms \u2014Image segmentation, object proposals, normalized cuts.\nF\n1 I NTRODUCTION\nTWOparadigms have shaped the \ufb01eld of object recognition in the last decade. The \ufb01rst one, popularized\nby the Viola-Jones face detection algorithm [1], formulates object localization as window classi\ufb01cation. The basic scanning-window architecture, relying on histograms\nof gradients and linear support vector machines, was\nintroduced by Dalal and Triggs [2] in the context of\npedestrian detection and is still at the core of seminal\nobject detectors on the PASCAL challenge such as Deformable Part Models [3].\nThe second paradigm relies on perceptual grouping to\nprovide a limited number of high-quality and categoryindependent object proposals, which can then be described with richer representations and used as input to\nmore sophisticated learning methods. Examples in this\nfamily are [4], [5]. Recently, this approach has dominated\nthe PASCAL segmentation challenge [6], [7], [8], [9], improved object detection [10], \ufb01ne-grained categorization\n[11] and proven competitive in large-scale classi\ufb01cation\n[12].\nSince the power of this second paradigm is critically\ndependent on the accuracy and the number of object"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6717-6db4-11ee-ac79-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-2", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "[11] and proven competitive in large-scale classi\ufb01cation\n[12].\nSince the power of this second paradigm is critically\ndependent on the accuracy and the number of object\nproposals, an increasing body of research has delved\n\u000fJ. Pont-Tuset and F. Marques are with the Department of Signal Theory\nand Communications, Universitat Polit` ecnica de Catalunya, BarcelonaTech\n(UPC), Spain. E-mail: fjordi.pont,ferran.marques g@upc.edu\n\u000fP . Arbel\u00b4 aez is with the Department of Biomedical Engineering, Universidad de los Andes, Colombia. E-mail: pa.arbelaez@uniandes.edu.co\n\u000fJ. T. Barron, and J. Malik are with the Department of Electrical\nEngineering and Computer Science, University of California at Berkeley,\nBerkeley, CA 94720. E-mail: fbarron,malikg@eecs.berkeley.edu\n* The \ufb01rst two authors contributed equally\nFig. 1. Top: original image, instance-level ground truth\nfrom COCO and our multiscale hierarchical segmentation. Bottom: our best object proposals among 150.\ninto the problem of their generation [13], [14], [15], [12],"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6718-6db4-11ee-8dfa-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-3", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "into the problem of their generation [13], [14], [15], [12],\n[16], [17], [18], [19]. However, those approaches typically\nfocus on learning generic properties of objects from\na set of examples, while reasoning on a \ufb01xed set of\nregions and contours produced by external bottom-up\nsegmenters such as [20], [21].\nIn this paper, we propose a uni\ufb01ed approach to\nmultiscale hierarchical segmentation and object proposal\ngeneration called Multiscale Combinatorial Grouping\n(MCG). Fig. 1 shows an example of our results and Fig. 2\nan overview of our pipeline. Our main contributions are:\n\u000fAn ef\ufb01cient normalized cuts algorithm, which in\npractice provides a 20\u0002speed-up to the eigenvector\ncomputation required for contour globalization [20],\n[22] (Sect. 3.1).\n\u000fA state-of-the-art hierarchical segmenter that leverages multiscale information (Sect. 3.3).\n\u000fA grouping algorithm that produces accurate object\nproposals by ef\ufb01ciently exploring the combinatorial\nspace of our multiscale regions (Sect. 5).arXiv:1503.00848v4  [cs.CV]  1 Mar 2016\n2"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6719-6db4-11ee-aefa-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-4", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "space of our multiscale regions (Sect. 5).arXiv:1503.00848v4  [cs.CV]  1 Mar 2016\n2\nF ixed-Scale\nSegmentation R escaling &\nA lignment C ombination  R es oluti on \nC ombinatorial\nGrouping\n \nI mage P yramid  Segmentati on P yramid  Al i gned  H iera rchies  Object Proposals  \n Multiscale Hierarchy\nFig. 2. Multiscale Combinatorial Grouping . Starting from a multiresolution image pyramid, we perform hierarchical\nsegmentation at each scale independently. We align these multiple hierarchies and combine them into a single\nmultiscale segmentation hierarchy. Our grouping component then produces a ranked list of object proposals by\nef\ufb01ciently exploring the combinatorial space of these regions.\nWe conduct a comprehensive and large-scale empirical\nvalidation. On the BSDS500 (Sect. 4) we report signi\ufb01cant\nprogress in contour detection and hierarchical segmentation. On the VOC2012, SBD, and COCO segmentation\ndatasets (Sect. 6), our proposals obtain overall state-ofthe-art accuracy both as segmented proposals and as"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a671a-6db4-11ee-9591-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-5", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "datasets (Sect. 6), our proposals obtain overall state-ofthe-art accuracy both as segmented proposals and as\nbounding boxes. MCG is ef\ufb01cient, its good generalization power makes it parameter free in practice, and it\nprovides a ranked set of proposals that are competitive\nin all regimes of number of proposals.\n2 R ELATED WORK\nFor space reasons, we focus our review on recent normalized cut algorithms and object proposals for recognition.\nFast normalized cuts : The ef\ufb01cient computation of\nnormalized-cuts eigenvectors has been the subject of\nrecent work, as it is often the computational bottleneck\nin grouping algorithms. Taylor [23] presented a technique for using a simple watershed oversegmentation\nto reduce the size of the eigenvector problem, sacri\ufb01cing accuracy for speed. We take a similar approach\nof solving the eigenvector problem in a reduced space,\nthough we use simple image-pyramid operations on\nthe af\ufb01nity matrix (instead of a separate segmentation\nalgorithm) and we see no loss in performance despite a\n20\u0002speed improvement. Maire and Yu [24] presented\na novel multigrid solver for producing eigenvectors at\nmultiple scales, which speeds up \ufb01ne-scale eigenvector\ncomputation by leveraging coarse-scale solutions. Our"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a671b-6db4-11ee-9ac0-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-6", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "multiple scales, which speeds up \ufb01ne-scale eigenvector\ncomputation by leveraging coarse-scale solutions. Our\ntechnique also uses the scale-space structure of an image,\nbut instead of solving the problem at multiple scales,\nwe simply reduce the scale of the problem, solve it at\na reduced scale, and then upsample the solution while\npreserving the structure of the image. As such, our\ntechnique is faster and much simpler, requiring only a\nfew lines of code wrapped around a standard sparse\neigensolver.Object Proposals : Class-independent methods that\ngenerate object hypotheses can be divided into those\nwhose output is an image window and those that generate segmented proposals.\nAmong the former, Alexe et al. [16] propose an objectness measure to score randomly-sampled image windows based on low-level features computed on the\nsuperpixels of [21]. Manen et al. [25] propose to use the\nRandomized Prim\u2019s algorithm, Zitnick et al. [26] group\ncontours directly to produce object windows, and Cheng\net al. [27] generate box proposals at 300 images per\nsecond. In contrast to these approaches, we focus on\nthe \ufb01ner-grained task of pixel-accurate object extraction,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a671c-6db4-11ee-b8e9-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-7", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "second. In contrast to these approaches, we focus on\nthe \ufb01ner-grained task of pixel-accurate object extraction,\nrather than on window selection. However, by just taking the bounding box around our segmented proposals,\nour results are also state of the art as window proposals.\nAmong the methods that produce segmented proposals, Carreira and Sminchisescu [18] hypothesize a set of\nplacements of fore- and background seeds and, for each\ncon\ufb01guration, solve a constrained parametric min-cut\n(CPMC) problem to generate a pool of object hypotheses.\nEndres and Hoiem [19] base their category-independent\nobject proposals on an iterative generation of a hierarchy\nof regions, based on the contour detector of [20] and\nocclusion boundaries of [28]. Kim and Grauman [17]\npropose to match parts of the shape of exemplar objects,\nregardless of their class, to detected contours by [20].\nThey infer the presence and shape of a proposal object\nby adapting the matched object to the computed superpixels.\nUijlings et al. [12] present a selective search algorithm\nbased on segmentation. Starting with the superpixels\nof [21] for a variety of color spaces, they produce a set of\nsegmentation hierarchies by region merging, which are"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a671d-6db4-11ee-a1ff-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-8", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "based on segmentation. Starting with the superpixels\nof [21] for a variety of color spaces, they produce a set of\nsegmentation hierarchies by region merging, which are\nused to produce a set of object proposals. While we also\ntake advantage of different hierarchies to gain diversity,\nwe leverage multiscale information rather than different\n3\ncolor spaces.\nRecently, two works proposed to train a cascade of\nclassi\ufb01ers to learn which sets of regions should be\nmerged to form objects. Ren and Shankhnarovich [29]\nproduce full region hierarchies by iteratively merging\npairs of regions and adapting the classi\ufb01ers to different\nscales. Weiss and Taskar [30] specialize the classi\ufb01ers also\nto size and class of the annotated instances to produce\nobject proposals.\nMalisiewicz and Efros [4] took one of the \ufb01rst steps\ntowards combinatorial grouping, by running multiple\nsegmenters with different parameters and merging up\nto three adjacent regions. In [8], another step was taken\nby considering hierarchical segmentations at three different scales and combining pairs and triplets of adjacent\nregions from the two coarser scales to produce object\nproposals.\nThe most recent wave of object proposal algorithms"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a671e-6db4-11ee-aad9-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-9", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "regions from the two coarser scales to produce object\nproposals.\nThe most recent wave of object proposal algorithms\nis represented by [13], [14], and [15], which all keep the\nquality of the seminal proposal works while improving\nthe speed considerably. Kr \u00a8ahenb \u00a8uhl and Koltun [13]\n\ufb01nd object proposal by identifying critical level sets\nin geodesic distance transforms, based on seeds placed\nin learnt places in the image. Rantalankila et al. [14]\nperform a global and local search in the space of sets\nof superpixels. Humayun et al. [15] reuse a graph to\nperform many parametric min-cuts over different seeds\nin order to speed the process up.\nA substantial difference between our approach and\nprevious work is that, instead of relying on precomputed hierarchies or superpixels, we propose a uni\ufb01ed approach that produces and groups high-quality\nmultiscale regions. With respect to the combinatorial approaches of [4], [8], our main contribution is to develop\nef\ufb01cient algorithms to explore a much larger combinatorial space by taking into account a set of object examples,\nincreasing thus the likelihood of having complete objects\nin the pool of proposals. Our approach has therefore"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a671f-6db4-11ee-b696-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-10", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "increasing thus the likelihood of having complete objects\nin the pool of proposals. Our approach has therefore\nthe \ufb02exibility to adapt to speci\ufb01c applications and types\nof objects, and can produce proposals at any trade-off\nbetween their number and their accuracy.\n3 T HESEGMENTATION ALGORITHM\nConsider a segmentation of the image into regions that\npartition its domain S=fSigi. A segmentation hierarchy\nis a family of partitions fS\u0003;S1;:::;SLgsuch that: (1)\nS\u0003is the \ufb01nest set of superpixels , (2)SLis the complete\ndomain, and (3) regions from coarse levels are unions of\nregions from \ufb01ne levels. A hierarchy where each level Si\nis assigned a real-valued index \u0015ican be represented by a\ndendrogram, a region tree where the height of each node\nis its index. Furthermore, it can also be represented as an\nultrametric contour map (UCM), an image obtained by\nweighting the boundary of each pair of adjacent regions\nin the hierarchy by the index at which they are merged\n[31], [32]. This representation uni\ufb01es the problems of\ncontour detection and hierarchical image segmentation:\n*\n##\n#"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6720-6db4-11ee-ac32-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-11", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "[31], [32]. This representation uni\ufb01es the problems of\ncontour detection and hierarchical image segmentation:\n*\n##\n#\nFig. 3. Duality between a UCM and a region tree :\nSchematic view of the dual representation of a segmentation hierarchy as a region dendrogram and as an\nultrametric contour map.\na threshold at level \u0015iin the UCM produces the segmentationSi.\nFigure 3 schematizes these concepts. First, the lower\nleft corner shows the probability of boundary of a UCM.\nOne of the main properties of a UCM is that when we\nthreshold the contour strength at a certain value, we\nobtain a closed boundary map, and thus a partition.\nThresholding at different \u0015i, therefore, we obtain the\nso-called merging-sequence partitions (left column in\nFigure 3); named after the fact that a step in this sequence\ncorresponds to merging the set of regions sharing the\nboundary of strength exactly \u0015i.\nFor instance, the boundary between the wheels and\nthe \ufb02oor has strength \u00151, thus thresholding the contour\nabove\u00151makes the wheels merge with the \ufb02oor. If we\nrepresent the regions in a partition as nodes of a graph,\nwe can then represent the result of merging them as their"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6721-6db4-11ee-9bb4-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-12", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "represent the regions in a partition as nodes of a graph,\nwe can then represent the result of merging them as their\nparent in a tree. The result of sweeping all \u0015ivalues\ncan therefore be represented as a region tree, whose root\nis the region representing the whole image (right part\nof Figure 3). Given that each merging is associated with\na contour strength, the region tree is in fact a region\ndendogram.\nAs an example, in the gPb-ucm algorithm of [20],\nbrightness, color and texture gradients at three \ufb01xed disk\nsizes are \ufb01rst computed. These local contour cues are\nglobalized using spectral graph-partitioning, resulting in\nthe gPb contour detector. Hierarchical segmentation is\nthen performed by iteratively merging adjacent regions\nbased on the average gPb strength on their common\nboundary. This algorithm produces therefore a tree of\nregions at multiple levels of homogeneity in brightness,\ncolor and texture, and the boundary strength of its UCM\ncan be interpreted as a measure of contrast.\nCoarse-to-\ufb01ne is a powerful processing strategy in\ncomputer vision. We exploit it in two different ways\nto develop an ef\ufb01cient, scalable and high-performance\nsegmentation algorithm: (1) To speed-up spectral graph"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6722-6db4-11ee-8df3-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-13", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "to develop an ef\ufb01cient, scalable and high-performance\nsegmentation algorithm: (1) To speed-up spectral graph\npartitioning and (2) To create aligned segmentation hierarchies.\n4\n3.1 Fast Downsampled Eigenvector Computation\nThe normalized cuts criterion is a key globalization\nmechanism of recent high-performance contour detectors such as [20], [22]. Although powerful, such spectral\ngraph partitioning has a signi\ufb01cant computational cost\nand memory footprint that limit its scalability. In this\nsection, we present an ef\ufb01cient normalized cuts approximation which in practice preserves full performance for\ncontour detection, while having low memory requirements and providing a 20\u0002speed-up.\nGiven a symmetric af\ufb01nity matrix A, we would like\nto compute the ksmallest eigenvectors of the Laplacian\nofA. Directly computing such eigenvectors can be very\ncostly even with sophisticated solvers, due to the large\nsize ofA. We therefore present a technique for ef\ufb01ciently\napproximating the eigenvector computation by taking\nadvantage of the multiscale nature of our problem: A\nmodels af\ufb01nities between pixels in an image, and images\nnaturally lend themselves to multiscale or pyramid-like"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6723-6db4-11ee-8719-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-14", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "models af\ufb01nities between pixels in an image, and images\nnaturally lend themselves to multiscale or pyramid-like\nrepresentations and algorithms.\nOur algorithm is inspired by two observations: 1) if\nAis bistochastic (the rows and columns of Asum to 1)\nthen the eigenvectors of the Laplacian Aare equal to\nthe eigenvectors of the Laplacian of A2, and 2) because\nof the scale-similar nature of images, the eigenvectors\nof a \u201cdownsampled\u201d version of Ain which every other\npixel has been removed should be similar to the eigenvectors of A. Let us de\ufb01ne pixel decimate (A), which\ntakes an af\ufb01nity matrix Aand returns the set of indices\nof rows/columns in Acorresponding to a decimated\nversion of the image from which Awas constructed. That\nis, ifi=pixel decimate (A), thenA[i;i]is a decimated\nmatrix in which alternating rows and columns of the imagehave been removed. Computing the eigenvectors of\nA[i;i]works poorly, as decimation disconnects pixels in\nthe af\ufb01nity matrix, but the eigenvectors of the decimated"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6724-6db4-11ee-a4e7-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-15", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "A[i;i]works poorly, as decimation disconnects pixels in\nthe af\ufb01nity matrix, but the eigenvectors of the decimated\nsquared af\ufb01nity matrix A2[i;i]are similar to those of A,\nbecause by squaring the matrix before decimation we\nintuitively allow each pixel to propagate information to\nall of its neighbors in the graph, maintaining connections even after decimation. Our algorithm works by\nef\ufb01ciently computing A2[i;i]asA[:;i]TA[:;i]1(the naive\napproach of \ufb01rst squaring Aand then decimating it is\nprohibitively expensive), computing the eigenvectors of\nA2[i;i], and then \u201cupsampling\u201d those eigenvectors back\nto the space of the original image by pre-multiplying\nbyA[:;i]. This squaring-and-decimation procedure can\nbe applied recursively several times, each application\nimproving ef\ufb01ciency while slightly sacri\ufb01cing accuracy.\nPseudocode for our algorithm, which we call\n\u201cDNCuts\u201d (Downsampled Normalized Cuts) is given in"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6725-6db4-11ee-aafd-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-16", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "Pseudocode for our algorithm, which we call\n\u201cDNCuts\u201d (Downsampled Normalized Cuts) is given in\nAlgorithm 1, where Ais our af\ufb01nity matrix and dis\nthe number of times that our squaring-and-decimation\noperation is applied. Our algorithm repeatedly applies\nour joint squaring-and-decimation procedure, computes\n1. The Matlab-like notation A[:; i]indicates keeping the columns of\nmatrix Awhose indices are in the set i.Algorithm 1 dncuts (A;d;k )\n1:A0 A\n2:fors= [1;2;:::;d ]do\n3:is pixel decimate (As\u00001)\n4:Bs As\u00001[ :; is]\n5:Cs diag (Bs~1)\u00001Bs\n6:As CT\nsBs\n7:end for\n8:Xd ncuts (Ad;k)\n9:fors= [d;d\u00001;:::; 1]do\n10:Xs\u00001 CsXs\n11:end for\n12:return whiten (X0)\nFig. 4. Example of segmentation projection . In order to\n\u201csnap\u201d the boundaries of a segmentation R(left) to those"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6726-6db4-11ee-9240-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-17", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "Fig. 4. Example of segmentation projection . In order to\n\u201csnap\u201d the boundaries of a segmentation R(left) to those\nof a segmentation S(middle), since they do not align, we\ncompute\u0019(R;S)(right) by assigning to each segment in\nSits mode among the labels of R.\nthe smallest keigenvectors of the \ufb01nal \u201cdownsampled\u201d matrix Adby using a standard sparse eigensolver\nncuts (Ad;k), and repeatedly \u201cupsamples\u201d those eigenvectors. Because our Ais not bistochastic and decimation\nis not an orthonormal operation, we must do some\nnormalization throughout the algorithm (line 5) and\nwhiten the resulting eigenvectors (line 10). We found that\nvalues ofd= 2 ord= 3 worked well in practice. Larger\nvalues ofdyielded little speed improvement (as much of\nthe cost is spent downsampling A0) and start negatively\naffecting accuracy. Our technique is similar to Nystrom\u2019s\nmethod for computing the eigenvectors of a subset of A,\nbut our squaring-and-decimation procedure means that\nwe do not depend on long-range connections between\npixels.\n3.2 Aligning Segmentation Hierarchies"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6727-6db4-11ee-8c3f-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-18", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "but our squaring-and-decimation procedure means that\nwe do not depend on long-range connections between\npixels.\n3.2 Aligning Segmentation Hierarchies\nIn order to leverage multi-scale information, our approach combines segmentation hierarchies computed\nindependently at multiple image resolutions. However, since subsampling an image removes details and\nsmooths away boundaries, the resulting UCMs are misaligned, as illustrated in the second panel of Fig. 2. In this\nsection, we propose an algorithm to align an arbitrary\nsegmentation hierarchy to a target segmentation and,\nin Sect. 5, we show its effectiveness for multi-scale\nsegmentation.\nThe basic operation is to \u201csnap\u201d the boundaries of a\nsegmentationR=fRigito a segmentation S=fSjgj, as\nillustrated in Fig. 4. For this purpose, we de\ufb01ne L(Sj),\n5\nthe new label of a region Sj2S, as the majority label of\nits pixels inR:\nL(Sj) = arg max\nijSj\\Rij\njSjj(1)\nWe call the segmentation de\ufb01ned by this new labeling of\nall the regions of Stheprojection ofRontoSand denote\nit by\u0019(R;S)."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6728-6db4-11ee-99b1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-19", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "We call the segmentation de\ufb01ned by this new labeling of\nall the regions of Stheprojection ofRontoSand denote\nit by\u0019(R;S).\nIn order to project an UCM onto a target segmentation\nS, which we denote \u0019(UCM;S), we project in turn each\nof the levels of the hierarchy onto S. Note that, since\nall the levels are projected onto the same segmentation,\nthe family of projections is by construction a hierarchy of segmentations. This procedure is summarized in\npseudo-code in Algorithm 2.\nAlgorithm 2 UCM Rescaling and Alignment\nRequire: An UCM with a set of levels [t1;:::;tK]\nRequire: A target segmentation S\u0003\n1:UCM\u0019 0\n2:fort= [t1;:::;tK]do\n3:S sampleHierarchy (UCM;t)\n4:S rescaleSegmentation (S;S\u0003)\n5:S \u0019(S;S\u0003)\n6:contours extractBoundary (S)\n7: UCM\u0019 max(UCM\u0019;t\u0003contours )\n8:end for\n9:return UCM\u0019\nObserve that the routines sampleHierarchy and\nextractBoundary can be computed ef\ufb01ciently because"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6729-6db4-11ee-bd82-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-20", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "8:end for\n9:return UCM\u0019\nObserve that the routines sampleHierarchy and\nextractBoundary can be computed ef\ufb01ciently because\nthey involve only thresholding operations and connected\ncomponents labeling. The complexity is thus dominated\nby rescaleSegmentation in Step 4, a nearest neighbor\ninterpolation, and the projection in Step 5, which are\ncomputedKtimes.\n3.3 Multiscale Hierarchical Segmentation\nSingle-scale segmentation : We consider as input\nthe following local contour cues: (1) brightness, color and\ntexture differences in half-disks of three sizes [33], (2)\nsparse coding on patches [22], and (3) structured forest\ncontours [34]. We globalize the contour cues independently using our fast eigenvector gradients of Sect. 3.1,\ncombine global and local cues linearly, and construct\nan UCM based on the mean contour strength. We tried\nlearning weights using gradient ascent on the F-measure\non the training set [20], but evaluating the \ufb01nal hierarchies rather than open contours. We observed that this\nobjective favors the quality of contours at the expense of\nregions and obtained better overall results by optimizing\nthe Segmentation Covering metric [20]."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a672a-6db4-11ee-aeba-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-21", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "objective favors the quality of contours at the expense of\nregions and obtained better overall results by optimizing\nthe Segmentation Covering metric [20].\nHierarchy Alignment : We construct a multiresolution pyramid with Nscales by subsampling / supersampling the original image and applying our singlescale segmenter. In order to preserve thin structures\nand details, we declare as set of possible boundarylocations the \ufb01nest superpixels in the highest-resolution.\nThen, applying recursively Algorithm 2, we project each\ncoarser UCM onto the next \ufb01ner scale until aligning it\nto the highest resolution superpixels.\nMultiscale Hierarchy : After alignment, we have a\n\ufb01xed set of boundary locations, and Nstrengths for\neach of them, coming from the different scales. We\nformulate this problem as binary boundary classi\ufb01cation\nand train a classi\ufb01er that combines these Nfeatures into\na single probability of boundary estimation. We experimented with several learning strategies for combining\nUCM strengths: (a) Uniform weights transformed into\nprobabilities with Platt\u2019s method. (b) SVMs and logistic\nregression, with both linear and additive kernels. (c)"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a672b-6db4-11ee-9484-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-22", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "probabilities with Platt\u2019s method. (b) SVMs and logistic\nregression, with both linear and additive kernels. (c)\nRandom Forests. (d) The same algorithm as for singlescale. We found the results with all learning methods\nsurprisingly similar, in agreement with the observation\nreported by [33]. This particular learning problem, with\nonly a handful of dimensions and millions of data points,\nis relatively easy and performance is mainly driven by\nour already high-performing and well calibrated features. We therefore use the simplest option (a).\n4 E XPERIMENTS ON THE BSDS500\nWe conduct extensive experiments on the BSDS500 [35],\nusing the standard evaluation metrics and following the\nbest practice rules of that dataset. We also report results\nwith a recent evaluation metric Fop[36], [37], PrecisionRecall for objects and parts, using the publicly-available\ncode.\nSingle-scale Segmentation : Table 1-top shows the\nperformance of our single-scale segmenter for different\ntypes of input contours on the validation set of the\nBSDS500. We obtain high-quality hierarchies for all the\ncues considered, showing the generality of our approach.\nFurthermore, when using them jointly (row \u2019Comb.\u2019\nin top panel), our segmenter outperforms the versions\nwith individual cues, suggesting its ability to leverage"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a672c-6db4-11ee-b29a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-23", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "Furthermore, when using them jointly (row \u2019Comb.\u2019\nin top panel), our segmenter outperforms the versions\nwith individual cues, suggesting its ability to leverage\ndiversi\ufb01ed inputs. In terms of ef\ufb01ciency, our fast normalized cuts algorithm provides an average 20 \u0002speedup over [20], starting from the same local cues, with\nno signi\ufb01cant loss in accuracy and with a low memory\nfootprint.\nMultiscale Segmentation : Table 1-bottom evaluates\nour full approach in the same experimental conditions as\nthe upper panel. We observe a consistent improvement\nin performance in all the metrics for all the inputs, which\nvalidates our architecture for multiscale segmentation.\nWe experimented with the range of scales and found\nN=f0:5;1;2gadequate for our purposes. A \ufb01ner\nsampling or a wider range of scales did not provide\nnoticeable improvements. We tested also two degraded\nversions of our system (not shown in the table). For\nthe \ufb01rst one, we resized contours to the original image\nresolution, created UCMs and combined them with the\nsame method as our \ufb01nal system. For the second one, we\ntransformed per-scale UCMs to the original resolution,\n6\nBoundary Region"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a672d-6db4-11ee-8b02-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-24", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "same method as our \ufb01nal system. For the second one, we\ntransformed per-scale UCMs to the original resolution,\n6\nBoundary Region\nFb Fop SC PRI VI\nInput ODS OIS ODS OIS ODS OIS ODS OIS ODS OISSingle-ScalePb [33] 0.702 0.733 0.334 0.370 0.577 0.636 0.801 0.847 1.692 1.490\nSC [22] 0.697 0.725 0.264 0.306 0.540 0.607 0.777 0.835 1.824 1.659\nSF [34] 0.719 0.737 0.338 0.399 0.582 0.651 0.803 0.851 1.608 1.432\nComb. 0.719 0.750 0.358 0.403 0.602 0.655 0.809 0.855 1.619 1.405MultiscalePb [33] 0.713 0.745 0.350 0.389 0.598 0.656 0.807 0.856 1.601 1.418\nSC [22] 0.705 0.734 0.331 0.384 0.579 0.647 0.799 0.851 1.637 1.460"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a672e-6db4-11ee-9821-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-25", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "SC [22] 0.705 0.734 0.331 0.384 0.579 0.647 0.799 0.851 1.637 1.460\nSF [34] 0.725 0.744 0.370 0.420 0.600 0.660 0.810 0.854 1.557 1.390\nComb. 0.725 0.757 0.371 0.408 0.611 0.670 0.813 0.862 1.548 1.367\nTABLE 1\nBSDS500 val set. Control experiments for single-scale\n(top) and multiscale (bottom) hierarchical segmentation\nwith different input contour detectors\nbut omitted the strength transfer to the \ufb01nest superpixels\nbefore combining them. The \ufb01rst ablated version produces interpolation artifacts and smooths away details,\nwhile the second one suffers from misalignment. Both\nfail to improve performance over the single-scale result,\nwhich provides additional empirical support for our\nmultiscale approach. We also observed a small degradation in performance when forcing the input contour\ndetector to use only the original image resolution, which\nindicates the advantages of considering multiscale information at all stages of processing.\nSince there are no drastic changes in our results when\ntaking as input the different individual cues or their combination, in the sequel we use the version with structured"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a672f-6db4-11ee-ad7c-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-26", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "Since there are no drastic changes in our results when\ntaking as input the different individual cues or their combination, in the sequel we use the version with structured\nforests for ef\ufb01ciency reasons, which we denote MCGUCM-Our .\nComparison with state-of-the-art. : Figure 5 compares our multiscale hierarchical segmenter MCG ( )\nand our single-scale hierarchical segmenter SCG ( )\non the BSDS500 test set against all the methods for\nwhich there is publicly available code. We also compare\nto the recent ISCRA [29] hierarchies ( ), provided\nprecomputed by the authors. We obtain consistently the\nbest results to date on BSDS500 for all operating regimes,\nboth in terms of boundary and region quality.\nNote that the range of object scales in the BSDS500\nis limited, which translates into modest absolute gains\nfrom MCG ( ) with respect to SCG ( ) in terms\nof boundary evaluation (left-hand plot), but more signi\ufb01cant improvements in terms of objects and parts\n(right-hand plot). We will also observe more substantial\nimprovements with respect to gPb-UCM ( ) when we\nmove to PASCAL, SBD, and COCO in Section 6 (e.g. see\nFig. 9)."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6730-6db4-11ee-971a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-27", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "move to PASCAL, SBD, and COCO in Section 6 (e.g. see\nFig. 9).\nGround-Truth Hierarchy : In order to gain further\ninsights, we transfer the strength of each ground-truth\nsegmentation to our highest-resolution superpixels SN\u0003\nand construct a combined hierarchy. This approximation to the semantic hierarchy, Ground-Truth Hierarchy\n(GTH) in Fig. 5, is an upper-bound for our approach as\nboth share the same boundary locations and the only\ndifference is their strength. Since the strength of GTHis proportional to the number of subjects marking it, it\nprovides naturally the correct semantic ordering, where\nouter object boundaries are stronger than internal parts.\nRecently, Maire et al. [38] developed an annotation\ntool where the user encodes explicitly the \u201cperceptual\nstrength\u201d of each contour. Our approach provides an\nalternative where the semantic hierarchy is reconstructed\nby sampling \ufb02at annotations from multiple subjects.\n5 O BJECT PROPOSAL GENERATION\nThe image segmentation algorithm presented in the\nprevious sections builds on low-level features, so its\nregions are unlikely to represent accurately complete\nobjects with heterogeneous parts. In this context, object\nproposal techniques create a set of hypotheses, possibly\noverlapping, which are more likely to represent full"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6731-6db4-11ee-8253-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-28", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "objects with heterogeneous parts. In this context, object\nproposal techniques create a set of hypotheses, possibly\noverlapping, which are more likely to represent full\nobject instances.\nOur approach to object proposal generation is to\ncombinatorially look for sets of regions from our segmentation hierarchies that merged together are likely\nto represent complete objects. In this section, we \ufb01rst\ndescribe the ef\ufb01cient computation of certain region descriptors on a segmentation tree. Then, we describe how\nwe use these techniques to ef\ufb01ciently explore the sets of\nmerged regions from the hierarchy. Finally, we explain\nhow we train the parameters of our algorithm for object\nproposals and how we rank the candidates by their\nprobability of representing an object.\nFast computation of descriptors : Let us assume, for\ninstance, we want to compute the area of all regions\nin the hierarchy. Intuitively, working strictly on the\nmerging-sequence partitions, we would need to scan all\npixels in all partitions. On the other hand, working on\nthe region tree allows us to scan the image only once to\ncompute the area of the leaves, and then propagate the\narea to all parents as the addition of the areas of their\nchildren.\nAs a drawback, the algorithms become intricate in\nterms of coding and necessary data structures. Take, for"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6732-6db4-11ee-8bcb-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-29", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "area to all parents as the addition of the areas of their\nchildren.\nAs a drawback, the algorithms become intricate in\nterms of coding and necessary data structures. Take, for\ninstance, the computation of the neighbors of a certain\nregion, which is trivial via scanning the partition on the\nmerging sequence (look for region labels in the adjacent\nboundary pixels), but need tailored data structures and\nalgorithms in the region tree.\nFormally, let us assume the image has ppixels, and we\nbuild a hierarchy based on ssuperpixels (leaves of the\ntree), andmmergings (different UCM strength values).\nThe cost of computing the area on all regions using the\nmerging-sequence partitions will be the cost of scanning\nall pixels in these partitions, thus p\u0001(m+1). In contrast,\nthe cost using the region tree will involve scanning the\nimage once, and then propagating the area values, so\np+m, which is notably faster.\nWe built tailored algorithms and data structures to\ncompute the bounding box, perimeter, and neighbors of\na region using the region tree representation.\n7\nBoundaries\n0 0:10:20:30:40:50:60:70:80:9 100:10:20:30:40:50:60:70:80:91\nRecallPrecision"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6733-6db4-11ee-9d9d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-30", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "RecallPrecision\nHuman [0.81-0.21] ISCRA [0.72]\nGTH [0.96] NCuts [0.63]\nMCG-Our [0.74] EGB [0.61]\nSCG-Our [0.74] MShift [0.60]\ngPb-UCM [0.73] Quadtree [0.41]Objects and Parts\n0 0:10:20:30:40:50:60:70:80:9 100:10:20:30:40:50:60:70:80:91\nRecallPrecisionHuman [0.56-0.05]\nGTH [0.64]\nMCG-Our [0.38]\nSCG-Our [0.35]\nISCRA [0.35]\ngPb-UCM [0.35]\nMShift [0.23]\nNCuts [0.21]\nEGB [0.16]\nQuadtree [0.06]\nFig. 5. BSDS500 test set. Precision-Recall curves for boundaries [35] (left) and for objects and parts [36] (right). The\nmarker on each curve is placed on the Optimal Dataset Scale (ODS), and its F measure is presented in brackets in the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6734-6db4-11ee-be5e-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-31", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "marker on each curve is placed on the Optimal Dataset Scale (ODS), and its F measure is presented in brackets in the\nlegend. The isolated red asterisks refer to the human performance assessed on the same image (one human partition\nagainst the rest of human annotations on the same image) and on a different image (one human partition against the\nhuman partitions of a different, randomly selected, image).\nCombinatorial Grouping of Proposals : We can\ncast object segmentation as selecting some regions in\nthe hierarchy, or in other words, as a combinatorial\noptimization problem on the hierarchy. To illustrate this\nprinciple, Figure 6(a) shows the simpli\ufb01ed representation\nof the hierarchy in Figure 3. Figure 6(b) and (c) show two\nobject proposals, and their representation in the region\nhierarchy.\nR4\nR5\nR1R2R3R6\nR9\nR7\nR1R2R3R4R8\nR5R6\n(a)\nR9\nR7\nR1R2R3R4R8\nR5R6\n(b)\nR9\nR7\nR1R2R3R4R8\nR5R6\n(c)"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6735-6db4-11ee-bd8a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-32", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "R5R6\n(b)\nR9\nR7\nR1R2R3R4R8\nR5R6\n(c)\nFig. 6. Object segmentation as combinatorial optimization : Examples of objects (b), (c), formed by selecting regions from a hierarchy (a).\nSince hierarchies are built taking only low-level features into account, and do not use semantic information,\nobjects will usually not be optimally represented using a\nsingle region in the hierarchy. As an example, Figure 6(c)\nshows the optimum representation of the car, consisting\nof three regions.\nA sensible approach to create object proposals is therefore to explore the set of n-tuples of regions. The main\nidea behind MCG is to explore this set ef\ufb01ciently, taking\nadvantage of the region tree representation, via the fastcomputation of region neighbors.\nThe whole set of tuples, however, is huge, and so it\nis not feasible to explore it exhaustively. Our approach\nranks the proposals using the height of their regions in\nthe tree (UCM strength) and explores the tree from the\ntop, but only down to a certain threshold. To speed the\nprocess up, we design a top-down algorithm to compute\nthe region neighbors and thus only compute them down\nto a certain depth in the tree."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6736-6db4-11ee-b42b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-33", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "process up, we design a top-down algorithm to compute\nthe region neighbors and thus only compute them down\nto a certain depth in the tree.\nTo further improve the results, we not only consider\nthen-tuples from the resulting MCG-UCM-Our hierarchy, but also the rest of hierarchies computed at different\nscales. As we will show in the experiments, diversity\nsigni\ufb01cantly improves the proposal results.\nParameter Learning via Pareto Front Optimization :\nMCG takes a set of diverse hierarchies and computes the\nn-tuples up to a certain UCM strength. We can interpret\nthen-tuples from each hierarchy as a ranked list of Ni\nproposals that are put together to create the \ufb01nal set of\nNpproposals.\nAt training time, we would like to \ufb01nd, for different values of Np, the number of proposals from each\nranked list Nisuch that the joint pool of Npproposals\nhas the best achievable quality. We frame this learning\nproblem as a Pareto front optimization [39], [40] with\ntwo con\ufb02icting objective functions: number of proposals\nand achievable quality. At test time, we select a working\npoint on the Pareto front, represented by the\b\nNi"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6737-6db4-11ee-8ec0-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-34", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "and achievable quality. At test time, we select a working\npoint on the Pareto front, represented by the\b\nNi\t\nvalues, based either on the number of proposals Npwe\ncan handle or on the minimum achievable quality our\n8\napplication needs, and we combine the Nitop proposals\nfrom each hierarchy list.\nFormally, assuming Rranked lists Li, an exhaustive\nlearning algorithm would consider all possible values\nof theR-tuplefN1;:::;NRg, whereNi2f0;:::;jLijg;\nadding up toQR\n1jLijparameterizations to try, which is\nintractable in our setting.\nFigure 7 illustrates the learning process. To reduce the\ndimensionality of the search space, we start by selecting\ntwo ranked lists L1,L2(green curves) and we sample\nthe list atSlevels of number of proposals (green dots).\nWe then scan the full S2different parameterizations to\ncombine the proposals from both (blue dots). In other\nwords, we analyze the sets of proposals created by\ncombining the top N1fromL1(green dots) and the top\nN2fromL2.\n1011021031040.30.60.9\nNumber of proposalsQuality\n1011021031040.30.60.9\nNumber of proposalsQuality"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6738-6db4-11ee-9aa4-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-35", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "N2fromL2.\n1011021031040.30.60.9\nNumber of proposalsQuality\n1011021031040.30.60.9\nNumber of proposalsQuality\n1011021031040.30.60.9\nNumber of proposalsQualityRanked lists of object proposals\nL 1 L RPareto front\nreduction of parameters\n{N 1 \u00b7 \u00b7 \u00b7  N R}\nFig. 7. Pareto front learning : Training the combinatorial\ngeneration of proposals using the Pareto front\nThe key step of the optimization consists in discarding\nthose parameterizations whose quality point is not in\nthe Pareto front (red curve). (i.e., those parameterizations\nthat can be substituted by another with better quality\nwith the same number of proposals, or by one with the\nsame quality with less proposals.) We sample the Pareto\nfront toSpoints and we iterate the process until all the\nranked lists are combined.\nEach point in the \ufb01nal Pareto front corresponds to a\nparticular parameterization fN1;:::;NRg. At train time,\nwe choose a point on this curve, either at a given\nnumber of proposals Ncor at the achievable quality\nwe are interested in (black triangle) and store the parameters\b\nN1;:::;NR\t\n. At test time, we combine the\b"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6739-6db4-11ee-84a3-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-36", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "we are interested in (black triangle) and store the parameters\b\nN1;:::;NR\t\n. At test time, we combine the\b\nN1;:::;NR\t\ntop proposals from each ranked list. The\nnumber of sampled con\ufb01gurations using the proposed\nalgorithm is (R\u00001)S2, that is, we have reduced an\nexponential problem ( SR) to a quadratic one.\nRegressed Ranking of Proposals : To further reduce\nthe number of proposals, we train a regressor from\nlow-level features, as in [18]. Since the proposals are\nall formed by a set of regions from a reduced set of\nhierarchies, we focus on features that can be computed\nef\ufb01ciently in a bottom-up fashion, as explained previously.\nWe compute the following features:\n\u000fSize and location : Area and perimeter of the candidate; area, position, and aspect ratio of the bounding\nbox; and the area balance between the regions in thecandidate.\n\u000fShape : Perimeter (and sum of contour strength)\ndivided by the squared root of the area; and area of the\nregion divided by that of the bounding box.\n\u000fContours : Sum of contour strength at the boundaries, mean contour strength at the boundaries; minimum and maximum UCM threshold of appearance and"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a673a-6db4-11ee-9ac2-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-37", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "\u000fContours : Sum of contour strength at the boundaries, mean contour strength at the boundaries; minimum and maximum UCM threshold of appearance and\ndisappearance of the regions forming the candidate.\nWe train a Random Forest using these features to regress\nthe object overlap with the ground truth, and diversify\nthe ranking based on Maximum Marginal Relevance\nmeasures [18]. We tune the random forest learning on\nhalf training set and validating on the other half. For\nthe \ufb01nal results, we train on the training set and evaluate\nour proposals on the validation set of PASCAL 2012.\n6 E XPERIMENTS ON PASCAL VOC, SBD,\nAND COCO\nThis section presents our large-scale empirical validation\nof the object proposal algorithm described in the previous section. We perform experiments in three annotated\ndatabases, with a variety of measures that demonstrate\nthe state-of-the-art performance of our algorithm.\nDatasets and Evaluation Measures : We conduct experiments in the following three annotated datasets: the\nsegmentation challenge of PASCAL 2012 Visual Object\nClasses (SegVOC12) [41], the Berkeley Semantic Boundaries Dataset (SBD) [42], and the Microsoft Common\nObjects in Context (COCO) [43]. They all consist of\nimages with annotated objects of different categories."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a673b-6db4-11ee-88de-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-38", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "Objects in Context (COCO) [43]. They all consist of\nimages with annotated objects of different categories.\nTable 2 summarizes the number of images and object\ninstances in each database.\nNumber of Number of Number of\nClasses Images Objects\nSegVOC12 20 2 913 9 847\nSBD 20 12 031 32 172\nCOCO 80 123 287 910 983\nTABLE 2\nSizes of the databases\nRegarding the performance metrics, we measure the\nachievable quality with respect to the number of proposals, that is, the quality we would have if an oracle\nselected the best proposal among the pool. This aligns\nwith the fact that object proposals are a preprocessing\nstep for other algorithms that will represent and classify\nthem. We want, therefore, the achievable quality within\nthe proposals to be as high as possible, while reducing\nthe number of proposals to make the \ufb01nal system as fast\nas possible.\nAs a measure of quality of a speci\ufb01c proposal with\nrespect to an annotated object, we consider the Jaccard\nindexJ, also known as overlap or intersection over\nunion; which is de\ufb01ned as the size of the intersection\nof the two pixel sets over the size of their union.\n9\nTo compute the overall quality for the whole database,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a673c-6db4-11ee-a93f-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-39", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "of the two pixel sets over the size of their union.\n9\nTo compute the overall quality for the whole database,\nwe \ufb01rst select the best proposal for each annotated\ninstance with respect to J. The Jaccard index at instance\nlevel (Ji) is then de\ufb01ned as the mean best overlap for all\nthe ground-truth instances in the database, also known\nas Best Spatial Support score (BSS) [4] or Average Best\nOverlap (ABO) [12].\nComputing the mean of the best overlap on all objects,\nas done by Ji, hides the distribution of quality among\ndifferent objects. As an example, Ji= 0:5can mean\nthat the algorithm covers half the objects perfectly and\ncompletely misses the other half, or can also mean that\nall the objects are covered exactly at J= 0:5. This\ninformation might be useful to decide which algorithm\nto use. Computing a histogram of the best overlap would\nprovide very rich information, but then the resulting\nplot would be 3D (number of proposals, bins, and bin\ncounts). Alternatively, we propose to plot different percentiles of the histogram.\nInterestingly, a certain percentile of the histogram\nof best overlaps consists in computing the number of\nobjects whose best overlap is above a certain Jaccard"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a673d-6db4-11ee-85a3-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-40", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "Interestingly, a certain percentile of the histogram\nof best overlaps consists in computing the number of\nobjects whose best overlap is above a certain Jaccard\nthreshold, which can be interpreted as the best achievable recall of the technique over a certain threshold. We\ncompute the recall at three different thresholds: J= 0:5,\nJ=0:7, andJ=0:85.\nLearning Strategy Evaluation : We \ufb01rst estimate the\nloss in performance due to not sweeping all the possible\nvalues offN1;:::;NRgin the combination of proposal\nlists via the proposed greedy strategy. To do so, we will\ncompare this strategy with the full combination on a\nreduced problem to make the latter feasible. Speci\ufb01cally,\nwe combine the 4 ranked lists coming from the singletons at all scales, instead of the full 16 lists coming from\nsingletons, pairs, triplets, and 4-tuples. We also limit the\nsearch to 20 000 proposals, further speeding the process\nup.\nIn this situation, the mean loss in achievable quality\nalong the full curve of parameterization is Ji= 0:0002 ,\nwith a maximum loss of Ji=0:004(0:74%). In exchange,\nour proposed learning strategy on the full 16 ranked lists\ntakes about 20 seconds to compute on the training set of"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a673e-6db4-11ee-9e47-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-41", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "our proposed learning strategy on the full 16 ranked lists\ntakes about 20 seconds to compute on the training set of\nSegVOC12, while the singleton-limited full combination\ntakes 4 days (the full combination would take months).\nCombinatorial Grouping : We now evaluate the\nPareto front optimization strategy in the training set of\nSegVOC12. As before, we extract the lists of proposals\nfrom the three scales and the multiscale hierarchy, for\nsingletons, pairs, triplets, and 4-tuples of regions, leading\nto 16 lists, ranked by the minimum UCM strength of the\nregions forming each proposal.\nFigure 8 shows the Pareto front evolution of Jiwith\nrespect to the number of proposals for up to 1, 2, 3, and\n4 regions per proposal (4, 8, 12, and 16 lists, respectively)\nat training and testing time on SegVOC12. As baselines,\nwe plot the raw singletons from MCG-UCM-Our, gPbUCM, and Quadtree; as well as the uniform combination\nof scales.103104105020406080100\nNumber of proposalsRegion distribution percentageScale 2.0\nScale 1.0\nScale 0.5\nMulti-Scale\nFig. 10. Region distribution learnt by the Pareto front\noptimization on SegVOC12."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a673f-6db4-11ee-9117-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-42", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "Scale 1.0\nScale 0.5\nMulti-Scale\nFig. 10. Region distribution learnt by the Pareto front\noptimization on SegVOC12.\nThe improvement of considering the combination of\nall 1-region proposals ( ) from the 3 scales and the\nMCG-UCM-Our with respect to the raw MCG-UCM-Our\n( ) is signi\ufb01cant, which corroborates the gain in diversity obtained from hierarchies at different scales. In turn,\nthe addition of 2- and 3-region proposals ( and )\nnoticeably improves the achievable quality. This shows\nthat hierarchies do not get full objects in single regions,\nwhich makes sense given that they are built using lowlevel features only. The improvement when adding 4tuples ( ) is marginal at the number of proposals we\nare considering. When analyzing the equal distribution\nof proposals from the four scales ( ), we see that the\nless proposals we consider, the more relevant the Pareto\noptimization becomes. At the selected working point, the\ngain of the Pareto optimization is 2 points.\nFigure 10 shows the distribution of proposals from\neach of the scales combined in the Pareto front. We\nsee that the coarse scale (0.5) is the most picked at\nlow number of proposals, and the rest come into play\nwhen increasing their number, since one can afford more"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6740-6db4-11ee-bb97-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-43", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "see that the coarse scale (0.5) is the most picked at\nlow number of proposals, and the rest come into play\nwhen increasing their number, since one can afford more\ndetailed proposals. The multi-scale hierarchy is the one\nwith less weight, since it is created from the other three.\nPareto selection and ranking : Back to Figure 8,\nthe red asterisk ( ) marks the selected con\ufb01guration\b\nN1;:::;NR\t\nin the Pareto front (black triangle in Figure 7), which is selected at a practical level of proposals.\nThe red plus sign ( ) represents the set of proposals after\nremoving those duplicate proposals whose overlap leads\nto a Jaccard higher than 0.95. The proposals at this point\nare the ones that are ranked by the learnt regressor ( ).\nAt test time (right-hand plot), we directly combine the\nlearnt\b\nN1;:::;NR\t\nproposals from each ranked list.\nNote that the Pareto optimization does not over\ufb01t, given\nthe similar result in the training and validation datasets.\nWe then remove duplicates and rank the results. In this\ncase, note the difference between the regressed result in"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6741-6db4-11ee-9c85-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-44", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "the similar result in the training and validation datasets.\nWe then remove duplicates and rank the results. In this\ncase, note the difference between the regressed result in\nthe training and validation sets, which re\ufb02ects over\ufb01tting, but despite this we found it bene\ufb01cial with respect\nto the non-regressed result.\n10\nTraining\n1021031041051060:40:50:60:70:80:9\nNumber of proposalsJaccard index at instance level ( Ji)Validation\n1021031041051060:40:50:60:70:80:9\nNumber of proposalsJaccard index at instance level ( Ji)Pareto up to 4-tuples\nPareto up to triplets\nPareto up to pairs\nPareto only singletons\nRaw Ours-multi singl.\nRaw gPb-UCM singl.\nRaw Quadtree singl.\nEqual distribution\nSelected con\ufb01guration\nFiltered candidates\nRegressed ranking\nFig. 8. Pareto front evaluation. Achievable quality of our proposals for singletons, pairs, triplets, and 4-tuples; and\nthe raw proposals from the hierarchies on PASCAL SegVOC12 training (left) and validation (right) sets.\nPASCAL SegVOC12"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6742-6db4-11ee-8582-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-45", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "the raw proposals from the hierarchies on PASCAL SegVOC12 training (left) and validation (right) sets.\nPASCAL SegVOC12\n1021031040:30:40:50:60:70:80:9\nNumber of proposalsJaccard index at instance level ( Ji)SBD\n1021031040:30:40:50:60:70:80:9\nNumber of proposalsJaccard index at instance level ( Ji)COCO\n1021031040:30:40:50:60:70:80:9\nNumber of proposalsJaccard index at instance level ( Ji)MCG-Our\nSCG-Our\nCPMC [18]\nCI [19]\nGOP [13]\nGLS [14]\nRIGOR [15]\nShSh [17]\nSeSe [12]\nMCG-UCM-Our\ngPb-UCM\nQuadtree\nFig. 9. Object Proposals: Jaccard index at instance level. Results on SegVOC12, SBD, and COCO.\nIn the validation set of SegVOC12, the full set of\nproposals (i.e., combining the full 16 lists) would contain\nmillions of proposals per image. The multiscale combinatorial grouping allows us to reduce the number of"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6743-6db4-11ee-bde1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-46", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "proposals (i.e., combining the full 16 lists) would contain\nmillions of proposals per image. The multiscale combinatorial grouping allows us to reduce the number of\nproposals to 5 086 with a very high achievable Jiof0:81\n(). The regressed ranking ( ) allows us to further\nreduce the number of proposals below this point.\nSegmented Proposals: Comparison with State of\nthe Art : We \ufb01rst compare our results against those\nmethods that produce segmented object proposals [13],\n[14], [15], [12], [16], [17], [18], [19], using the implementations from the respective authors. We train MCG on\nthe training set of SegVOC12, and we use the learnt\nparameters on the validation sets of SegVOC12, SBD,\nand COCO.\nFigure 9 shows the achievable quality at instance level\n(Ji) of all methods on the validation set of SegVOC12,\nSBD, and COCO. We plot the raw regions of MCGUCM-Our, gPb-UCM, and QuadTree as baselines where\navailable. We also evaluate a faster single-scale version of\nMCG ( Single-scale Combinatorial Grouping - SCG ), which\ntakes the hierarchy at the native scale only and combines"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6744-6db4-11ee-8ea9-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-47", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "MCG ( Single-scale Combinatorial Grouping - SCG ), which\ntakes the hierarchy at the native scale only and combines\nup to 4 regions per proposal. This approach decreases\nthe computational load one order of magnitude whilekeeping competitive results.\nMCG proposals ( ) signi\ufb01cantly outperform the\nstate-of-the-art at all regimes. The bigger the database is,\nthe better MCG results are with respect to the rest, which\nshows that our techniques better generalize to unseen\nimages (recall that MCG is trained only in SegVOC12).\nAs commented on the measures description, Jishows\nmean aggregate results, so they can mask the distribution of quality among objects in the database. Figure 11\nshows the recall at three different Jaccard levels. First,\nthese plots further highlight how challenging COCO\nis, since we observe a signi\ufb01cant drop in performance,\nmore pronounced than when measured by JiandJc.\nAnother interesting result comes from observing the\nevolution of the plots for the three different Jaccard\nvalues. Take for instance the performance of GOP ( )\nagainst MCG-Our ( ) in SBD. While for J=0:5GOP\nslightly outperforms MCG, the higher the threshold, the\nbetter MCG. Overall, MCG has specially good results"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6745-6db4-11ee-aeb5-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-48", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "slightly outperforms MCG, the higher the threshold, the\nbetter MCG. Overall, MCG has specially good results\nat higherJvalues. In other words, if one looks for\nproposals of very high accuracy, MCG is the method\nwith highest recall, at all regimes and in all databases.\nIn all measures and databases, SCG ( ) obtains very\ncompetitive results, especially if we take into account\n11\nPASCAL SegVOC12\n10210310400:10:20:30:40:50:60:70:80:9\nJ=0:5\nJ=0:7\nJ=0:85\nNumber of proposalsRecallSBD\n10210310400:10:20:30:40:50:60:70:80:9\nJ= 0:5\nJ= 0:7\nJ= 0:85\nNumber of proposalsRecallCOCO\n10210310400:10:20:30:40:50:60:70:80:9\nJ= 0:5\nJ= 0:7\nJ= 0:85\nNumber of proposalsRecallMCG-Our\nSCG-Our\nCPMC [18]\nCI [19]\nGOP [13]\nGLS [14]\nRIGOR [15]\nShSh [17]\nSeSe [12]"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6746-6db4-11ee-9533-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-49", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "CI [19]\nGOP [13]\nGLS [14]\nRIGOR [15]\nShSh [17]\nSeSe [12]\nQuadtree\nFig. 11. Segmented Object Proposals: Recall at different Jaccard levels. Percentage of annotated objects for\nwhich there is a proposal whose overlap with the segmented ground-truth shapes (not boxes) is above J= 0:5,\nJ= 0:7, andJ= 0:85, for different number of proposals per image. Results on SegVOC12, SBD, and COCO.\nPASCAL SegVOC12\n10210310400:10:20:30:40:50:60:70:80:9\nJ= 0:5\nJ= 0:7\nJ= 0:85\nNumber of proposalsRecallSBD\n10210310400:10:20:30:40:50:60:70:80:9\nJ= 0:5\nJ= 0:7\nJ= 0:85\nNumber of proposalsRecallCOCO\n10210310400:10:20:30:40:50:60:70:80:9\nJ= 0:5\nJ= 0:7\nJ= 0:85\nNumber of proposalsRecallMCG-Our\nSCG-Our\nCPMC [18]"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a6747-6db4-11ee-b88b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1503.00848-50", "title": "Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation", "source": "http://arxiv.org/pdf/1503.00848", "start_index": 0, "chroma:document": "J= 0:5\nJ= 0:7\nJ= 0:85\nNumber of proposalsRecallMCG-Our\nSCG-Our\nCPMC [18]\nCI [19]\nGOP [13]\nGLS [14]\nRIGOR [15]\nShSh [17]\nSeSe [12]\nRP [25]\nEB [26]\nBING [27]\nObj [16]\nQuadtree\nFig. 12. Bounding-Box Proposals: Recall at different Jaccard levels. Percentage of annotated objects for which\nthere is a bounding box proposal whose overlap with the ground-truth boxes is above J= 0:5,J= 0:7, andJ= 0:85,\nfor different number of proposals per image. Results on SegVOC12, SBD, and COCO.\nthat it is 7\u0002faster than MCG, as we will see in next\nsections.\nThe complementarity of MCG with respect to other\nproposal techniques, and their combination using the\nPareto front optimization is studied in [44].\nBoxes Proposals: Comparison with State of the\nArt: Although focused in providing segmented object\nproposals, MCG may also be used to provide boxes\nproposals, by taking the bounding box of the segmented"}


