# embeddings_queue

created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18d4-6db4-11ee-af83-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-11", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "layers are fully connected, with an N-way softmax as\nthe output, where N is the number of categories.\nThe deep network described above needs a \ufb01xed\nimage size. However, we notice that the requirement\nof \ufb01xed sizes is only due to the fully-connected layers\nthat demand \ufb01xed-length vectors as inputs. On the\nother hand, the convolutional layers accept inputs of\narbitrary sizes. The convolutional layers use sliding\n\ufb01lters, and their outputs have roughly the same aspect\nratio as the inputs. These outputs are known as feature\nmaps [1] - they involve not only the strength of the\nresponses, but also their spatial positions.\nIn Figure 2, we visualize some feature maps. They\nare generated by some \ufb01lters of the conv 5layer. Figure 2(c) shows the strongest activated images of these\n\ufb01lters in the ImageNet dataset. We see a \ufb01lter can be\nactivated by some semantic content. For example, the\n55-th \ufb01lter (Figure 2, bottom left) is most activated by\na circle shape; the 66-th \ufb01lter (Figure 2, top right) is"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18d5-6db4-11ee-ad9d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-12", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "a circle shape; the 66-th \ufb01lter (Figure 2, top right) is\nmost activated by a ^-shape; and the 118-th \ufb01lter (Figure 2, bottom right) is most activated by a _-shape.\nThese shapes in the input images (Figure 2(a)) activate\nthe feature maps at the corresponding positions (the\narrows in Figure 2).\nIt is worth noticing that we generate the feature\nmaps in Figure 2 without \ufb01xing the input size. These\nfeature maps generated by deep convolutional layers are analogous to the feature maps in traditional\nmethods [27], [28]. In those methods, SIFT vectors\n[29] or image patches [28] are densely extracted and\nthen encoded, e.g., by vector quantization [16], [15],\n[30], sparse coding [17], [18], or Fisher kernels [19].\nThese encoded features consist of the feature maps,\nand are then pooled by Bag-of-Words (BoW) [16] or\nspatial pyramids [14], [15]. Analogously, the deep\nconvolutional features can be pooled in a similar way.\n2.2 The Spatial Pyramid Pooling Layer\nThe convolutional layers accept arbitrary input sizes,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18d6-6db4-11ee-9d1a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-13", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "convolutional features can be pooled in a similar way.\n2.2 The Spatial Pyramid Pooling Layer\nThe convolutional layers accept arbitrary input sizes,\nbut they produce outputs of variable sizes. The classi\ufb01ers (SVM/softmax) or fully-connected layers require\nconvolutional layersfeature maps of conv 5\n(arbitrary size)\nfixed-length representation\ninput image16\u00d7256-d 4\u00d7256-d 256-d\u2026...\n\u2026...\nspatial pyramid pooling layerfully-connected layers (fc 6, fc 7)Figure 3: A network structure with a spatial pyramid\npooling layer . Here 256 is the \ufb01lter number of the\nconv 5layer, and conv 5is the last convolutional layer.\n\ufb01xed-length vectors. Such vectors can be generated\nby the Bag-of-Words (BoW) approach [16] that pools\nthe features together. Spatial pyramid pooling [14],\n[15] improves BoW in that it can maintain spatial\ninformation by pooling in local spatial bins. These\nspatial bins have sizes proportional to the image size,\nso the number of bins is \ufb01xed regardless of the image\nsize. This is in contrast to the sliding window pooling\nof the previous deep networks [3], where the number"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18d7-6db4-11ee-8b4e-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-14", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "so the number of bins is \ufb01xed regardless of the image\nsize. This is in contrast to the sliding window pooling\nof the previous deep networks [3], where the number\nof sliding windows depends on the input size.\nTo adopt the deep network for images of arbitrary sizes, we replace the last pooling layer ( e.g.,\npool 5, after the last convolutional layer) with a spatial\npyramid pooling layer . Figure 3 illustrates our method.\nIn each spatial bin, we pool the responses of each\n\ufb01lter (throughout this paper we use max pooling).\nThe outputs of the spatial pyramid pooling are kMdimensional vectors with the number of bins denoted\nasM(kis the number of \ufb01lters in the last convolutional layer). The \ufb01xed-dimensional vectors are the\ninput to the fully-connected layer.\nWith spatial pyramid pooling, the input image can\n4\nbe of any sizes. This not only allows arbitrary aspect\nratios, but also allows arbitrary scales. We can resize\nthe input image to any scale ( e.g.,min(w;h)=180, 224,\n...) and apply the same deep network. When the\ninput image is at different scales, the network (with\nthe same \ufb01lter sizes) will extract features at different"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18d8-6db4-11ee-a4f1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-15", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "...) and apply the same deep network. When the\ninput image is at different scales, the network (with\nthe same \ufb01lter sizes) will extract features at different\nscales. The scales play important roles in traditional\nmethods, e.g., the SIFT vectors are often extracted at\nmultiple scales [29], [27] (determined by the sizes of\nthe patches and Gaussian \ufb01lters). We will show that\nthe scales are also important for the accuracy of deep\nnetworks.\nInterestingly, the coarsest pyramid level has a single\nbin that covers the entire image. This is in fact a\n\u201cglobal pooling\u201d operation, which is also investigated\nin several concurrent works. In [31], [32] a global\naverage pooling is used to reduce the model size\nand also reduce over\ufb01tting; in [33], a global average\npooling is used on the testing stage after all fc layers\nto improve accuracy; in [34], a global max pooling is\nused for weakly supervised object recognition. The\nglobal pooling operation corresponds to the traditional Bag-of-Words method.\n2.3 Training the Network\nTheoretically, the above network structure can be\ntrained with standard back-propagation [1], regardless of the input image size. But in practice the GPU"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18d9-6db4-11ee-a4d1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-16", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "Theoretically, the above network structure can be\ntrained with standard back-propagation [1], regardless of the input image size. But in practice the GPU\nimplementations (such as cuda-convnet [3] and Caffe\n[35]) are preferably run on \ufb01xed input images. Next\nwe describe our training solution that takes advantage\nof these GPU implementations while still preserving\nthe spatial pyramid pooling behaviors.\nSingle-size training\nAs in previous works, we \ufb01rst consider a network taking a \ufb01xed-size input (224 \u0002224) cropped from images.\nThe cropping is for the purpose of data augmentation.\nFor an image with a given size, we can pre-compute\nthe bin sizes needed for spatial pyramid pooling.\nConsider the feature maps after conv 5that have a size\nofa\u0002a(e.g., 13\u000213). With a pyramid level of n\u0002n\nbins, we implement this pooling level as a sliding\nwindow pooling, where the window size win=da=ne\nand stride str=ba=ncwithd\u0001eandb\u0001cdenoting\nceiling and \ufb02oor operations. With an l-level pyramid,\nwe implement lsuch layers. The next fully-connected\nlayer (fc 6) will concatenate the loutputs. Figure 4"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18da-6db4-11ee-a9e1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-17", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "we implement lsuch layers. The next fully-connected\nlayer (fc 6) will concatenate the loutputs. Figure 4\nshows an example con\ufb01guration of 3-level pyramid\npooling (3\u00023, 2\u00022, 1\u00021) in the cuda-convnet style [3].\nThe main purpose of our single-size training is to\nenable the multi-level pooling behavior. Experiments\nshow that this is one reason for the gain of accuracy.\nMulti-size training\nOur network with SPP is expected to be applied on\nimages of any sizes. To address the issue of varying\n[fc6]\ntype=fcoutputs=4096inputs=pool3x3,pool2x2,pool1x1[pool1x1]\ntype=poolpool=maxinputs=conv5sizeX=13stride=13[pool3x3]\ntype=poolpool=maxinputs=conv5sizeX=5stride=4[pool2x2]\ntype=poolpool=maxinputs=conv5sizeX=7stride=6Figure 4: An example 3-level pyramid pooling in the\ncuda-convnet style [3]. Here sizeX is the size of the\npooling window. This con\ufb01guration is for a network"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18db-6db4-11ee-beef-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-18", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "cuda-convnet style [3]. Here sizeX is the size of the\npooling window. This con\ufb01guration is for a network\nwhose feature map size of conv 5is 13\u000213, so the\npool 3\u00023, pool 2\u00022, and pool 1\u00021layers will have 3 \u00023,\n2\u00022, and 1\u00021 bins respectively.\nimage sizes in training, we consider a set of prede\ufb01ned sizes. We consider two sizes: 180 \u0002180 in addition to 224\u0002224. Rather than crop a smaller 180 \u0002180\nregion, we resize the aforementioned 224 \u0002224 region\nto 180\u0002180. So the regions at both scales differ only\nin resolution but not in content/layout. For the network to accept 180 \u0002180 inputs, we implement another\n\ufb01xed-size-input (180 \u0002180) network. The feature map\nsize after conv 5isa\u0002a= 10\u000210 in this case. Then we\nstill usewin=da=neandstr=ba=ncto implement\neach pyramid pooling level. The output of the spatial\npyramid pooling layer of this 180-network has the\nsame \ufb01xed length as the 224-network. As such, this\n180-network has exactly the same parameters as the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18dc-6db4-11ee-8403-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-19", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "same \ufb01xed length as the 224-network. As such, this\n180-network has exactly the same parameters as the\n224-network in each layer. In other words, during\ntraining we implement the varying-input-size SPP-net\nby two \ufb01xed-size networks that share parameters.\nTo reduce the overhead to switch from one network\n(e.g., 224) to the other ( e.g., 180), we train each full\nepoch on one network, and then switch to the other\none (keeping all weights) for the next full epoch. This\nis iterated. In experiments, we \ufb01nd the convergence\nrate of this multi-size training to be similar to the\nabove single-size training.\nThe main purpose of our multi-size training is to\nsimulate the varying input sizes while still leveraging\nthe existing well-optimized \ufb01xed-size implementations. Besides the above two-scale implementation, we\nhave also tested a variant using s\u0002sas input where\nsis randomly and uniformly sampled from [180;224]\nat each epoch. We report the results of both variants\nin the experiment section.\nNote that the above single/multi-size solutions are\nfor training only. At the testing stage, it is straightforward to apply SPP-net on images of any sizes.\n5"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18dd-6db4-11ee-ae0e-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-20", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "Note that the above single/multi-size solutions are\nfor training only. At the testing stage, it is straightforward to apply SPP-net on images of any sizes.\n5\nmodel conv 1 conv 2 conv 3 conv 4 conv 5 conv 6 conv 7\nZF-5 96\u000272, str 2 256\u000252, str 2 384\u000232384\u000232256\u000232\nLRN, pool 32, str 2 LRN, pool 32, str 2 - map size 55\u000255 27\u000227 13\u000213 13\u000213 13\u000213\nConvnet*-5 96\u0002112, str 4 256\u000252384\u000232384\u000232256\u000232\nLRN, LRN, pool 32, str 2 pool 32, 2 - map size 55\u000255 27\u000227 13\u000213 13\u000213 13\u000213\nOverfeat-5/7 96\u000272, str 2 256\u000252512\u000232512\u000232512\u000232512\u000232512\u000232\npool 32, str 3, LRN pool 22, str 2\nmap size 36\u000236 18\u000218 18\u000218 18\u000218 18\u000218 18\u000218 18\u000218\nTable 1: Network architectures: \ufb01lter number \u0002\ufb01lter size ( e.g.,96\u000272), \ufb01lter stride ( e.g., str 2), pooling window"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18de-6db4-11ee-becf-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-21", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "size ( e.g., pool 32), and the output feature map size ( e.g., map size 55\u000255). LRN represents Local Response\nNormalization. The padding is adjusted to produce the expected output feature map size.\n3 SPP- NET FOR IMAGE CLASSIFICATION\n3.1 Experiments on ImageNet 2012 Classi\ufb01cation\nWe train the networks on the 1000-category training\nset of ImageNet 2012. Our training algorithm follows\nthe practices of previous work [3], [4], [36]. The images are resized so that the smaller dimension is 256,\nand a 224\u0002224 crop is picked from the center or the\nfour corners from the entire image1. The data are augmented by horizontal \ufb02ipping and color altering [3].\nDropout [3] is used on the two fully-connected layers.\nThe learning rate starts from 0.01, and is divided by 10\n(twice) when the error plateaus. Our implementation\nis based on the publicly available code of cuda-convnet\n[3] and Caffe [35]. All networks in this paper can be\ntrained on a single GeForce GTX Titan GPU (6 GB\nmemory) within two to four weeks.\n3.1.1 Baseline Network Architectures\nThe advantages of SPP are independent of the convolutional network architectures used. We investigate"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18df-6db4-11ee-98a9-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-22", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "memory) within two to four weeks.\n3.1.1 Baseline Network Architectures\nThe advantages of SPP are independent of the convolutional network architectures used. We investigate\nfour different network architectures in existing publications [3], [4], [5] (or their modi\ufb01cations), and we\nshow SPP improves the accuracy of all these architectures. These baseline architectures are in Table 1 and\nbrie\ufb02y introduced below:\n\u000fZF-5 : this architecture is based on Zeiler and Fergus\u2019s (ZF) \u201cfast\u201d (smaller) model [4]. The number\nindicates \ufb01ve convolutional layers.\n\u000fConvnet*-5 : this is a modi\ufb01cation on Krizhevsky\net al.\u2019s network [3]. We put the two pooling layers\nafter conv 2and conv 3(instead of after conv 1and\nconv 2). As a result, the feature maps after each\nlayer have the same size as ZF-5.\n\u000fOverfeat-5/7 : this architecture is based on the\nOverfeat paper [5], with some modi\ufb01cations as in\n[6]. In contrast to ZF-5/Convnet*-5, this architecture produces a larger feature map ( 18\u000218instead"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18e0-6db4-11ee-bfa0-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-23", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "[6]. In contrast to ZF-5/Convnet*-5, this architecture produces a larger feature map ( 18\u000218instead\nof13\u000213) before the last pooling layer. A larger\n\ufb01lter number (512) is used in conv 3and the following convolutional layers. We also investigate\n1. In [3], the four corners are picked from the corners of the\ncentral 256\u0002256 crop.a deeper architecture with 7 convolutional layers,\nwhere conv 3to conv 7have the same structures.\nIn the baseline models, the pooling layer after the last\nconvolutional layer generates 6\u00026feature maps, with\ntwo 4096-d fc layers and a 1000-way softmax layer\nfollowing. Our replications of these baseline networks\nare in Table 2 (a). We train 70 epochs for ZF-5 and\n90 epochs for the others. Our replication of ZF-5 is\nbetter than the one reported in [4]. This gain is because\nthe corner crops are from the entire image, as is also\nreported in [36].\n3.1.2 Multi-level Pooling Improves Accuracy\nIn Table 2 (b) we show the results using singlesize training. The training and testing sizes are both\n224\u0002224. In these networks, the convolutional layers"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18e1-6db4-11ee-88fa-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-24", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "In Table 2 (b) we show the results using singlesize training. The training and testing sizes are both\n224\u0002224. In these networks, the convolutional layers\nhave the same structures as the corresponding baseline models, whereas the pooling layer after the \ufb01nal\nconvolutional layer is replaced with the SPP layer. For\nthe results in Table 2, we use a 4-level pyramid. The\npyramid isf6\u00026, 3\u00023, 2\u00022, 1\u00021g(totally 50 bins).\nFor fair comparison, we still use the standard 10view prediction with each view a 224 \u0002224 crop. Our\nresults in Table 2 (b) show considerable improvement\nover the no-SPP baselines in Table 2 (a). Interestingly,\nthe largest gain of top-1 error (1.65%) is given by the\nmost accurate architecture. Since we are still using the\nsame 10 cropped views as in (a), these gains are solely\nbecause of multi-level pooling.\nIt is worth noticing that the gain of multi-level\npooling is notsimply due to more parameters; rather,\nit is because the multi-level pooling is robust to the\nvariance in object deformations and spatial layout\n[15]. To show this, we train another ZF-5 network with"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18e2-6db4-11ee-9829-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-25", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "it is because the multi-level pooling is robust to the\nvariance in object deformations and spatial layout\n[15]. To show this, we train another ZF-5 network with\na different 4-level pyramid: f4\u00024, 3\u00023, 2\u00022, 1\u00021g\n(totally 30 bins). This network has fewer parameters\nthan its no-SPP counterpart, because its fc 6layer has\n30\u0002256-d inputs instead of 36 \u0002256-d. The top-1/top5 errors of this network are 35.06/14.04. This result\nis similar to the 50-bin pyramid above (34.98/14.14),\nbut considerably better than the no-SPP counterpart\n(35.99/14.76).\n6\ntop-1 error (%)\nZF-5 Convnet*-5 Overfeat-5 Overfeat-7\n(a) no SPP 35.99 34.93 34.13 32.01\n(b) SPP single-size trained 34.98 (1.01) 34.38 (0.55) 32.87 (1.26) 30.36 (1.65)\n(c) SPP multi-size trained 34.60 (1.39) 33.94 (0.99) 32.26 (1.87) 29.68 (2.33)"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18e3-6db4-11ee-ba3f-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-26", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "top-5 error (%)\nZF-5 Convnet*-5 Overfeat-5 Overfeat-7\n(a) no SPP 14.76 13.92 13.52 11.97\n(b) SPP single-size trained 14.14 (0.62) 13.54 (0.38) 12.80 (0.72) 11.12 (0.85)\n(c) SPP multi-size trained 13.64 (1.12) 13.33 (0.59) 12.33 (1.19) 10.95 (1.02)\nTable 2: Error rates in the validation set of ImageNet 2012. All the results are obtained using standard 10-view\ntesting. In the brackets are the gains over the \u201cno SPP\u201d baselines.\nSPP on test view top-1 val\nZF-5, single-size trained 1 crop 38.01\nZF-5, single-size trained 1 full 37.55\nZF-5, multi-size trained 1 crop 37.57\nZF-5, multi-size trained 1 full 37.07\nOverfeat-7, single-size trained 1 crop 33.18\nOverfeat-7, single-size trained 1 full 32.72\nOverfeat-7, multi-size trained 1 crop 32.57"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18e4-6db4-11ee-ad70-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-27", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "Overfeat-7, single-size trained 1 full 32.72\nOverfeat-7, multi-size trained 1 crop 32.57\nOverfeat-7, multi-size trained 1 full 31.25\nTable 3: Error rates in the validation set of ImageNet\n2012 using a single view. The images are resized so\nmin(w;h) = 256 . The crop view is the central 224 \u0002224\nof the image.\n3.1.3 Multi-size Training Improves Accuracy\nTable 2 (c) shows our results using multi-size training.\nThe training sizes are 224 and 180, while the testing\nsize is still 224. We still use the standard 10-view\nprediction. The top-1/top-5 errors of all architectures\nfurther drop. The top-1 error of SPP-net (Overfeat-7)\ndrops to 29.68%, which is 2.33% better than its noSPP counterpart and 0.68% better than its single-size\ntrained counterpart.\nBesides using the two discrete sizes of 180 and\n224, we have also evaluated using a random size\nuniformly sampled from [180;224]. The top-1/5 error\nof SPP-net (Overfeat-7) is 30.06%/10.96%. The top1 error is slightly worse than the two-size version,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18e5-6db4-11ee-9b01-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-28", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "of SPP-net (Overfeat-7) is 30.06%/10.96%. The top1 error is slightly worse than the two-size version,\npossibly because the size of 224 (which is used for\ntesting) is visited less. But the results are still better\nthe single-size version.\nThere are previous CNN solutions [5], [36] that deal\nwith various scales/sizes, but they are mostly based\non testing. In Overfeat [5] and Howard\u2019s method [36],\nthe single network is applied at multiple scales in the\ntesting stage, and the scores are averaged. Howard\nfurther trains two different networks on low/highresolution image regions and averages the scores. To\nour knowledge, our method is the \ufb01rst one that trains\na single network with input images of multiple sizes.3.1.4 Full-image Representations Improve Accuracy\nNext we investigate the accuracy of the full-image\nviews. We resize the image so that min(w;h)=256\nwhile maintaining its aspect ratio. The SPP-net is\napplied on this full image to compute the scores of\nthe full view. For fair comparison, we also evaluate\nthe accuracy of the single view in the center 224 \u0002224\ncrop (which is used in the above evaluations). The\ncomparisons of single-view testing accuracy are in"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18e6-6db4-11ee-9b3d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-29", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "the accuracy of the single view in the center 224 \u0002224\ncrop (which is used in the above evaluations). The\ncomparisons of single-view testing accuracy are in\nTable 3. Here we evaluate ZF-5/Overfeat-7. The top-1\nerror rates are all reduced by the full-view representation. This shows the importance of maintaining the\ncomplete content. Even though our network is trained\nusing square images only, it generalizes well to other\naspect ratios.\nComparing Table 2 and Table 3, we \ufb01nd that the\ncombination of multiple views is substantially better\nthan the single full-image view. However, the fullimage representations are still of good merits. First,\nwe empirically \ufb01nd that (discussed in the next subsection) even for the combination of dozens of views,\nthe additional two full-image views (with \ufb02ipping)\ncan still boost the accuracy by about 0.2%. Second,\nthe full-image view is methodologically consistent\nwith the traditional methods [15], [17], [19] where the\nencoded SIFT vectors of the entire image are pooled\ntogether. Third, in other applications such as image\nretrieval [37], an image representation, rather than a\nclassi\ufb01cation score, is required for similarity ranking.\nA full-image representation can be preferred."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18e7-6db4-11ee-99b8-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-30", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "retrieval [37], an image representation, rather than a\nclassi\ufb01cation score, is required for similarity ranking.\nA full-image representation can be preferred.\n3.1.5 Multi-view Testing on Feature Maps\nInspired by our detection algorithm (described in\nthe next section), we further propose a multi-view\ntesting method on the feature maps. Thanks to the\n\ufb02exibility of SPP , we can easily extract the features\nfrom windows (views) of arbitrary sizes from the\nconvolutional feature maps.\nOn the testing stage, we resize an image so\nmin(w;h) =swheresrepresents a prede\ufb01ned scale\n(like 256). Then we compute the convolutional feature maps from the entire image. For the usage of\n7\nmethod test scales test views top-1 val top-5 val top-5 test\nKrizhevsky et al. [3] 1 10 40.7 18.2\nOverfeat (fast) [5] 1 - 39.01 16.97\nOverfeat (fast) [5] 6 - 38.12 16.27\nOverfeat (big) [5] 4 - 35.74 14.18\nHoward (base) [36] 3 162 37.0 15.8\nHoward (high-res) [36] 3 162 36.8 16.2"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18e8-6db4-11ee-8d54-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-31", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "Howard (base) [36] 3 162 37.0 15.8\nHoward (high-res) [36] 3 162 36.8 16.2\nZeiler & Fergus (ZF) (fast) [4] 1 10 38.4 16.5\nZeiler & Fergus (ZF) (big) [4] 1 10 37.5 16.0\nChat\ufb01eld et al. [6] 1 10 - 13.1\nours (SPP O-7) 1 10 29.68 10.95\nours (SPP O-7) 6 96+2full 27.86 9.14 9.08\nTable 4: Error rates in ImageNet 2012. All the results are based on a single network . The number of views in\nOverfeat depends on the scales and strides, for which there are several hundreds at the \ufb01nest scale.\n\ufb02ipped views, we also compute the feature maps of\nthe \ufb02ipped image. Given any view (window) in the\nimage, we map this window to the feature maps (the\nway of mapping is in Appendix), and then use SPP\nto pool the features from this window (see Figure 5).\nThe pooled features are then fed into the fc layers\nto compute the softmax score of this window. These\nscores are averaged for the \ufb01nal prediction. For the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18e9-6db4-11ee-a95e-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-32", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "The pooled features are then fed into the fc layers\nto compute the softmax score of this window. These\nscores are averaged for the \ufb01nal prediction. For the\nstandard 10-view, we use s= 256 and the views\nare 224\u0002224 windows on the corners or center. Experiments show that the top-5 error of the 10-view\nprediction on feature maps is within 0.1% around the\noriginal 10-view prediction on image crops.\nWe further apply this method to extract multiple\nviews from multiple scales. We resize the image to six\nscaless2 f224;256;300;360;448;560gand compute\nthe feature maps on the entire image for each scale.\nWe use 224\u0002224 as the view size for any scale,\nso these views have different relative sizes on the\noriginal image for different scales. We use 18 views\nfor each scale: one at the center, four at the corners,\nand four on the middle of each side, with/without\n\ufb02ipping (when s= 224 there are 6 different views).\nThe combination of these 96 views reduces the top-5\nerror from 10.95% to 9.36%. Combining the two fullimage views (with \ufb02ipping) further reduces the top-5\nerror to 9.14%."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18ea-6db4-11ee-b913-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-33", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "error from 10.95% to 9.36%. Combining the two fullimage views (with \ufb02ipping) further reduces the top-5\nerror to 9.14%.\nIn the Overfeat paper [5], the views are also extracted from the convolutional feature maps instead\nof image crops. However, their views cannot have arbitrary sizes; rather, the windows are those where the\npooled features match the desired dimensionality. We\nempirically \ufb01nd that these restricted windows are less\nbene\ufb01cial than our \ufb02exibly located/sized windows.\n3.1.6 Summary and Results for ILSVRC 2014\nIn Table 4 we compare with previous state-of-theart methods. Krizhevsky et al .\u2019s [3] is the winning\nmethod in ILSVRC 2012; Overfeat [5], Howard\u2019s [36],\nand Zeiler and Fergus\u2019s [4] are the leading methodsrank team top-5 test\n1 GoogLeNet [32] 6.66\n2 VGG [33] 7.32\n3 ours 8.06\n4 Howard 8.11\n5 DeeperVision 9.50\n6 NUS-BST 9.79\n7 TTIC ECP 10.22"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18eb-6db4-11ee-8436-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-34", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "3 ours 8.06\n4 Howard 8.11\n5 DeeperVision 9.50\n6 NUS-BST 9.79\n7 TTIC ECP 10.22\nTable 5: The competition results of ILSVRC 2014 classi\ufb01cation [26]. The best entry of each team is listed.\nin ILSVRC 2013. We only consider single-network\nperformance for manageable comparisons.\nOur best single network achieves 9.14% top-5 error\non the validation set. This is exactly the single-model\nentry we submitted to ILSVRC 2014 [26]. The top-5\nerror is 9.08% on the testing set (ILSVRC 2014 has\nthe same training/validation/testing data as ILSVRC\n2012). After combining eleven models, our team\u2019s result ( 8.06%) is ranked #3 among all 38 teams attending\nILSVRC 2014 (Table 5). Since the advantages of SPPnet should be in general independent of architectures,\nwe expect that it will further improve the deeper and\nlarger convolutional architectures [33], [32].\n3.2 Experiments on VOC 2007 Classi\ufb01cation\nOur method can generate a full-view image representation. With the above networks pre-trained on\nImageNet, we extract these representations from the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18ec-6db4-11ee-b07b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-35", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "Our method can generate a full-view image representation. With the above networks pre-trained on\nImageNet, we extract these representations from the\nimages in the target datasets and re-train SVM classi\ufb01ers [38]. In the SVM training, we intentionally do\nnot use any data augmentation (\ufb02ip/multi-view). We\nl2-normalize the features for SVM training.\nThe classi\ufb01cation task in Pascal VOC 2007 [22]\ninvolves 9,963 images in 20 categories. 5,011 images\nare for training, and the rest are for testing. The\nperformance is evaluated by mean Average Precision\n(mAP). Table 6 summarizes the results.\n8\n(a) (b) (c) (d) (e)\nmodel no SPP (ZF-5) SPP (ZF-5) SPP (ZF-5) SPP (ZF-5) SPP (Overfeat-7)\ncrop crop full full full\nsize 224\u0002224 224\u0002224 224\u0002- 392\u0002- 364\u0002conv 4 59.96 57.28 - - conv 5 66.34 65.43 - - pool 5=7(6\u00026) 69.14 68.76 70.82 71.67 76.09"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18ed-6db4-11ee-8437-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-36", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "fc6=8 74.86 75.55 77.32 78.78 81.58\nfc7=9 75.90 76.45 78.39 80.10 82.44\nTable 6: Classi\ufb01cation mAP in Pascal VOC 2007. For SPP-net, the pool 5=7layer uses the 6\u00026 pyramid level.\n(a) (b) (c) (d)\nmodel no SPP (ZF-5) SPP (ZF-5) SPP (ZF-5) SPP (Overfeat-7)\ncrop crop full full\nsize 224\u0002224 224\u0002224 224\u0002- 224\u0002conv 4 80.12 81.03 - conv 5 84.40 83.76 - pool 5=7(6\u00026) 87.98 87.60 89.46 91.46\nSPP pool 5=7 - 89.47 91.44 93.42\nfc6=8 87.86 88.54 89.50 91.83\nfc7=9 85.30 86.10 87.08 90.00\nTable 7: Classi\ufb01cation accuracy in Caltech101. For SPP-net, the pool 5=7layer uses the 6\u00026 pyramid level.\nWe start from a baseline in Table 6 (a). The model is"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18ee-6db4-11ee-8126-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-37", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "We start from a baseline in Table 6 (a). The model is\nZF-5 without SPP . To apply this model, we resize the\nimage so that its smaller dimension is 224, and crop\nthe center 224\u0002224 region. The SVM is trained via\nthe features of a layer. On this dataset, the deeper the\nlayer is, the better the result is. In Table 6 (b), we replace the no-SPP net with our SPP-net. As a \ufb01rst-step\ncomparison, we still apply the SPP-net on the center\n224\u0002224 crop. The results of the fc layers improve.\nThis gain is mainly due to multi-level pooling.\nTable 6 (c) shows our results on full images, where\nthe images are resized so that the shorter side is 224.\nWe \ufb01nd that the results are considerably improved\n(78.39% vs. 76.45%). This is due to the full-image\nrepresentation that maintains the complete content.\nBecause the usage of our network does not depend\non scale, we resize the images so that the smaller\ndimension is sand use the same network to extract\nfeatures. We \ufb01nd that s= 392 gives the best results\n(Table 6 (d)) based on the validation set. This is mainly\nbecause the objects occupy smaller regions in VOC"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18ef-6db4-11ee-881d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-38", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "(Table 6 (d)) based on the validation set. This is mainly\nbecause the objects occupy smaller regions in VOC\n2007 but larger regions in ImageNet, so the relative\nobject scales are different between the two sets. These\nresults indicate scale matters in the classi\ufb01cation tasks,\nand SPP-net can partially address this \u201cscale mismatch\u201d issue.\nIn Table 6 (e) the network architecture is replaced\nwith our best model (Overfeat-7, multi-size trained),\nand the mAP increases to 82.44% . Table 8 summarizes\nour results and the comparisons with the state-of-theart methods. Among these methods, VQ [15], LCC\n[18], and FK [19] are all based on spatial pyramids\nmatching, and [13], [4], [34], [6] are based on deepnetworks. In these results, Oquab et al.\u2019s (77.7%) and\nChat\ufb01eld et al .\u2019s (82.42%) are obtained by network\n\ufb01ne-tuning and multi-view testing. Our result is comparable with the state of the art, using only a single\nfull-image representation and without \ufb01ne-tuning.\n3.3 Experiments on Caltech101"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18f0-6db4-11ee-87f1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-39", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "full-image representation and without \ufb01ne-tuning.\n3.3 Experiments on Caltech101\nThe Caltech101 dataset [21] contains 9,144 images in\n102 categories (one background). We randomly sample 30 images per category for training and up to 50\nimages per category for testing. We repeat 10 random\nsplits and average the accuracy. Table 7 summarizes\nour results.\nThere are some common observations in the Pascal\nVOC 2007 and Caltech101 results: SPP-net is better\nthan the no-SPP net (Table 7 (b) vs. (a)), and the fullview representation is better than the crop ((c) vs. (b)).\nBut the results in Caltech101 have some differences\nwith Pascal VOC. The fully-connected layers are less\naccurate, and the SPP layers are better. This is possibly\nbecause the object categories in Caltech101 are less\nrelated to those in ImageNet, and the deeper layers\nare more category-specialized. Further, we \ufb01nd that\nthe scale 224 has the best performance among the\nscales we tested on this dataset. This is mainly because\nthe objects in Caltech101 also occupy large regions of\nthe images, as is the case of ImageNet.\nBesides cropping, we also evaluate warping the\nimage to \ufb01t the 224 \u0002224 size. This solution maintains"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18f1-6db4-11ee-94be-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-40", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "the images, as is the case of ImageNet.\nBesides cropping, we also evaluate warping the\nimage to \ufb01t the 224 \u0002224 size. This solution maintains\nthe complete content, but introduces distortion. On\nthe SPP (ZF-5) model, the accuracy is 89.91% using\nthe SPP layer as features - lower than 91.44% which\nuses the same model on the undistorted full image.\n9\nmethod VOC 2007 Caltech101\nVQ [15]y56.07 74.41 \u00061.0\nLLC [18]y57.66 76.95 \u00060.4\nFK [19]y61.69 77.78 \u00060.6\nDeCAF [13] - 86.91 \u00060.7\nZeiler & Fergus [4] 75.90z86.5\u00060.5\nOquab et al. [34] 77.7 Chat\ufb01eld et al. [6] 82.42 88.54\u00060.3\nours 82.44 93.42 \u00060.5\nTable 8: Classi\ufb01cation results for Pascal VOC 2007\n(mAP) and Caltech101 (accuracy).ynumbers reported\nby [27].zour implementation as in Table 6 (a).\nTable 8 summarizes our results compared with the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18f2-6db4-11ee-b75c-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-41", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "(mAP) and Caltech101 (accuracy).ynumbers reported\nby [27].zour implementation as in Table 6 (a).\nTable 8 summarizes our results compared with the\nstate-of-the-art methods on Caltech101. Our result\n(93.42% ) exceeds the previous record (88.54%) by a\nsubstantial margin (4.88%).\n4 SPP- NET FOR OBJECT DETECTION\nDeep networks have been used for object detection.\nWe brie\ufb02y review the recent state-of-the-art R-CNN\nmethod [7]. R-CNN \ufb01rst extracts about 2,000 candidate windows from each image via selective search\n[20]. Then the image region in each window is warped\nto a \ufb01xed size (227 \u0002227). A pre-trained deep network\nis used to extract the feature of each window. A\nbinary SVM classi\ufb01er is then trained on these features\nfor detection. R-CNN generates results of compelling\nquality and substantially outperforms previous methods. However, because R-CNN repeatedly applies the\ndeep convolutional network to about 2,000 windows\nper image, it is time-consuming. Feature extraction is\nthe major timing bottleneck in testing.\nOur SPP-net can also be used for object detection.\nWe extract the feature maps from the entire image"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18f3-6db4-11ee-a97b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-42", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "the major timing bottleneck in testing.\nOur SPP-net can also be used for object detection.\nWe extract the feature maps from the entire image\nonly once (possibly at multiple scales). Then we apply the spatial pyramid pooling on each candidate\nwindow of the feature maps to pool a \ufb01xed-length\nrepresentation of this window (see Figure 5). Because\nthe time-consuming convolutions are only applied\nonce, our method can run orders of magnitude faster.\nOur method extracts window-wise features from\nregions of the feature maps, while R-CNN extracts\ndirectly from image regions. In previous works, the\nDeformable Part Model (DPM) [23] extracts features\nfrom windows in HOG [24] feature maps, and the\nSelective Search (SS) method [20] extracts from windows in encoded SIFT feature maps. The Overfeat\ndetection method [5] also extracts from windows of\ndeep convolutional feature maps, but needs to prede\ufb01ne the window size. On the contrary, our method\nenables feature extraction in arbitrary windows from\nthe deep convolutional feature maps.\nspatial pyramid \npooling layer\nfeature maps of conv 5\nconvolutional layersfixed-length representation\ninput imagewindow\u2026...fully-connected layers (fc 6, fc 7)Figure 5: Pooling features from arbitrary windows"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18f4-6db4-11ee-a721-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-43", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "convolutional layersfixed-length representation\ninput imagewindow\u2026...fully-connected layers (fc 6, fc 7)Figure 5: Pooling features from arbitrary windows\non feature maps. The feature maps are computed\nfrom the entire image. The pooling is performed in\ncandidate windows.\n4.1 Detection Algorithm\nWe use the \u201cfast\u201d mode of selective search [20] to\ngenerate about 2,000 candidate windows per image.\nThen we resize the image such that min(w;h) =s,\nand extract the feature maps from the entire image.\nWe use the SPP-net model of ZF-5 (single-size trained)\nfor the time being. In each candidate window, we use\na 4-level spatial pyramid (1 \u00021, 2\u00022, 3\u00023, 6\u00026, totally\n50 bins) to pool the features. This generates a 12,800d (256\u000250) representation for each window. These\nrepresentations are provided to the fully-connected\nlayers of the network. Then we train a binary linear\nSVM classi\ufb01er for each category on these features.\nOur implementation of the SVM training follows\n[20], [7]. We use the ground-truth windows to generate the positive samples. The negative samples are\nthose overlapping a positive window by at most 30%"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18f5-6db4-11ee-a262-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-44", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "[20], [7]. We use the ground-truth windows to generate the positive samples. The negative samples are\nthose overlapping a positive window by at most 30%\n(measured by the intersection-over-union (IoU) ratio).\nAny negative sample is removed if it overlaps another\nnegative sample by more than 70%. We apply the standard hard negative mining [23] to train the SVM. This\nstep is iterated once. It takes less than 1 hour to train\nSVMs for all 20 categories. In testing, the classi\ufb01er\nis used to score the candidate windows. Then we use\nnon-maximum suppression [23] (threshold of 30%) on\nthe scored windows.\nOur method can be improved by multi-scale feature\nextraction. We resize the image such that min(w;h) =\ns2S=f480;576;688;864;1200g, and compute the\nfeature maps of conv 5for each scale. One strategy of\ncombining the features from these scales is to pool\nthem channel-by-channel. But we empirically \ufb01nd\nthat another strategy provides better results. For each\ncandidate window, we choose a single scale s2S\nsuch that the scaled candidate window has a number\nof pixels closest to 224 \u0002224. Then we only use the\nfeature maps extracted from this scale to compute"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18f6-6db4-11ee-b331-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-45", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "such that the scaled candidate window has a number\nof pixels closest to 224 \u0002224. Then we only use the\nfeature maps extracted from this scale to compute\n10\nthe feature of this window. If the pre-de\ufb01ned scales\nare dense enough and the window is approximately\nsquare, our method is roughly equivalent to resizing\nthe window to 224 \u0002224 and then extracting features\nfrom it. Nevertheless, our method only requires computing the feature maps once (at each scale) from the\nentire image, regardless of the number of candidate\nwindows.\nWe also \ufb01ne-tune our pre-trained network, following [7]. Since our features are pooled from the conv 5\nfeature maps from windows of any sizes, for simplicity we only \ufb01ne-tune the fully-connected layers.\nIn this case, the data layer accepts the \ufb01xed-length\npooled features after conv 5, and the fc 6;7layers and\na new 21-way (one extra negative category) fc 8layer\nfollow. The fc 8weights are initialized with a Gaussian\ndistribution of \u001b=0.01. We \ufb01x all the learning rates to\n1e-4 and then adjust to 1e-5 for all three layers. During"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18f7-6db4-11ee-a4ec-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-46", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "distribution of \u001b=0.01. We \ufb01x all the learning rates to\n1e-4 and then adjust to 1e-5 for all three layers. During\n\ufb01ne-tuning, the positive samples are those overlapping with a ground-truth window by [0:5;1], and\nthe negative samples by [0:1;0:5). In each mini-batch,\n25% of the samples are positive. We train 250k minibatches using the learning rate 1e-4, and then 50k\nmini-batches using 1e-5. Because we only \ufb01ne-tune\nthe fc layers, the training is very fast and takes about\n2 hours on the GPU (excluding pre-caching feature\nmaps which takes about 1 hour). Also following [7],\nwe use bounding box regression to post-process the\nprediction windows. The features used for regression\nare the pooled features from conv 5(as a counterpart\nof the pool 5features used in [7]). The windows used\nfor the regression training are those overlapping with\na ground-truth window by at least 50%.\n4.2 Detection Results\nWe evaluate our method on the detection task of the\nPascal VOC 2007 dataset. Table 9 shows our results\non various layers, by using 1-scale ( s=688) or 5-scale."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18f8-6db4-11ee-a6b1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-47", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "Pascal VOC 2007 dataset. Table 9 shows our results\non various layers, by using 1-scale ( s=688) or 5-scale.\nHere the R-CNN results are as reported in [7] using\nthe AlexNet [3] with 5 conv layers. Using the pool 5\nlayers (in our case the pooled features), our result\n(44.9%) is comparable with R-CNN\u2019s result (44.2%).\nBut using the non-\ufb01ne-tuned fc 6layers, our results are\ninferior. An explanation is that our fc layers are pretrained using image regions, while in the detection\ncase they are used on the feature map regions. The\nfeature map regions can have strong activations near\nthe window boundaries, while the image regions may\nnot. This difference of usages can be addressed by\n\ufb01ne-tuning. Using the \ufb01ne-tuned fc layers (ftfc 6;7), our\nresults are comparable with or slightly better than\nthe \ufb01ne-tuned results of R-CNN. After bounding box\nregression, our 5-scale result ( 59.2%) is 0.7% better\nthan R-CNN (58.5%), and our 1-scale result (58.0%)\nis 0.5% worse."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18f9-6db4-11ee-8b41-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-48", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "than R-CNN (58.5%), and our 1-scale result (58.0%)\nis 0.5% worse.\nIn Table 10 we further compare with R-CNN using the same pre-trained model of SPPnet (ZF-5). InSPP (1-sc) SPP (5-sc) R-CNN\n(ZF-5) (ZF-5) (Alex-5)\npool 5 43.0 44.9 44.2\nfc6 42.5 44.8 46.2\nftfc6 52.3 53.7 53.1\nftfc7 54.5 55.2 54.2\nftfc7bb 58.0 59.2 58.5\nconv time (GPU) 0.053s 0.293s 8.96s\nfc time (GPU) 0.089s 0.089s 0.07s\ntotal time (GPU) 0.142s 0.382s 9.03s\nspeedup ( vs. RCNN) 64\u0002 24\u0002 Table 9: Detection results (mAP) on Pascal VOC 2007.\n\u201cft\u201d and \u201cbb\u201d denote \ufb01ne-tuning and bounding box\nregression.\nSPP (1-sc) SPP (5-sc) R-CNN"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18fa-6db4-11ee-9ff1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-49", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "regression.\nSPP (1-sc) SPP (5-sc) R-CNN\n(ZF-5) (ZF-5) (ZF-5)\nftfc7 54.5 55.2 55.1\nftfc7bb 58.0 59.2 59.2\nconv time (GPU) 0.053s 0.293s 14.37s\nfc time (GPU) 0.089s 0.089s 0.089s\ntotal time (GPU) 0.142s 0.382s 14.46s\nspeedup ( vs. RCNN) 102\u0002 38\u0002 Table 10: Detection results (mAP) on Pascal VOC 2007,\nusing the same pre-trained model of SPP (ZF-5).\nthis case, our method and R-CNN have comparable\naveraged scores. The R-CNN result is boosted by\nthis pre-trained model. This is because of the better\narchitecture of ZF-5 than AlexNet, and also because\nof the multi-level pooling of SPPnet (if using the noSPP ZF-5, the R-CNN result drops). Table 11 shows\nthe results for each category.\nTable 11 also includes additional methods. Selective\nSearch (SS) [20] applies spatial pyramid matching on"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18fb-6db4-11ee-a67e-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-50", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "the results for each category.\nTable 11 also includes additional methods. Selective\nSearch (SS) [20] applies spatial pyramid matching on\nSIFT feature maps. DPM [23] and Regionlet [39] are\nbased on HOG features [24]. The Regionlet method\nimproves to 46.1% [8] by combining various features including conv 5. DetectorNet [40] trains a deep\nnetwork that outputs pixel-wise object masks. This\nmethod only needs to apply the deep network once\nto the entire image, as is the case for our method. But\nthis method has lower mAP (30.5%).\n4.3 Complexity and Running Time\nDespite having comparable accuracy, our method is\nmuch faster than R-CNN. The complexity of the convolutional feature computation in R-CNN is O(n\u0001\n2272)with the window number n(\u00182000). This complexity of our method is O(r\u0001s2)at a scales, where\nris the aspect ratio. Assume ris about 4/3. In the\nsingle-scale version when s= 688 , this complexity is\n11\nmethod mAP areo bike bird boat bottle bus car cat chair cow table dog horse mbike person plant sheep sofa train tv"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18fc-6db4-11ee-83b0-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-51", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "11\nmethod mAP areo bike bird boat bottle bus car cat chair cow table dog horse mbike person plant sheep sofa train tv\nDPM [23] 33.7 33.2 60.3 10.2 16.1 27.3 54.3 58.2 23.0 20.0 24.1 26.7 12.7 58.1 48.2 43.2 12.0 21.1 36.1 46.0 43.5\nSS [20] 33.8 43.5 46.5 10.4 12.0 9.3 49.4 53.7 39.4 12.5 36.9 42.2 26.4 47.0 52.4 23.5 12.1 29.9 36.3 42.2 48.8\nRegionlet [39] 41.7 54.2 52.0 20.3 24.0 20.1 55.5 68.7 42.6 19.2 44.2 49.1 26.6 57.0 54.5 43.4 16.4 36.6 37.7 59.4 52.3"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18fd-6db4-11ee-b03f-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-52", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "DetNet [40] 30.5 29.2 35.2 19.4 16.7 3.7 53.2 50.2 27.2 10.2 34.8 30.2 28.2 46.6 41.7 26.2 10.3 32.8 26.8 39.8 47.0\nRCNN ftfc 7(A5) 54.2 64.2 69.7 50.0 41.9 32.0 62.6 71.0 60.7 32.7 58.5 46.5 56.1 60.6 66.8 54.2 31.5 52.8 48.9 57.9 64.7\nRCNN ftfc 7(ZF5) 55.1 64.8 68.4 47.0 39.5 30.9 59.8 70.5 65.3 33.5 62.5 50.3 59.5 61.6 67.9 54.1 33.4 57.3 52.9 60.2 62.9\nSPP ftfc 7(ZF5) 55.2 65.5 65.9 51.7 38.4 32.7 62.6 68.6 69.7 33.1 66.6 53.1 58.2 63.6 68.8 50.4 27.4 53.7 48.2 61.7 64.7"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18fe-6db4-11ee-880f-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-53", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "RCNN bb (A5) 58.5 68.1 72.8 56.8 43.0 36.8 66.3 74.2 67.6 34.4 63.5 54.5 61.2 69.1 68.6 58.7 33.4 62.9 51.1 62.5 64.8\nRCNN bb (ZF5) 59.2 68.4 74.0 54.0 40.9 35.2 64.1 74.4 69.8 35.5 66.9 53.8 64.2 69.9 69.6 58.9 36.8 63.4 56.0 62.8 64.9\nSPP bb (ZF5) 59.2 68.6 69.7 57.1 41.2 40.5 66.3 71.3 72.5 34.4 67.3 61.7 63.1 71.0 69.8 57.6 29.7 59.0 50.2 65.2 68.0\nTable 11: Comparisons of detection results on Pascal VOC 2007.\nmethod mAP areo bike bird boat bottle bus car cat chair cow table dog horse mbike person plant sheep sofa train tv"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a18ff-6db4-11ee-8f4e-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-54", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "Table 11: Comparisons of detection results on Pascal VOC 2007.\nmethod mAP areo bike bird boat bottle bus car cat chair cow table dog horse mbike person plant sheep sofa train tv\nSPP-net (1) 59.2 68.6 69.7 57.1 41.2 40.5 66.3 71.3 72.5 34.4 67.3 61.7 63.1 71.0 69.8 57.6 29.7 59.0 50.2 65.2 68.0\nSPP-net (2) 59.1 65.7 71.4 57.4 42.4 39.9 67.0 71.4 70.6 32.4 66.7 61.7 64.8 71.7 70.4 56.5 30.8 59.9 53.2 63.9 64.6\ncombination 60.9 68.5 71.7 58.7 41.9 42.5 67.7 72.1 73.8 34.7 67.0 63.4 66.0 72.5 71.3 58.9 32.8 60.9 56.1 67.9 68.8\nTable 12: Detection results on VOC 2007 using model combination. The results of both models use \u201cftfc 7bb\u201d.\nabout 1/160 of R-CNN\u2019s; in the 5-scale version, this"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1900-6db4-11ee-aad2-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-55", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "about 1/160 of R-CNN\u2019s; in the 5-scale version, this\ncomplexity is about 1/24 of R-CNN\u2019s.\nIn Table 10, we provide a fair comparison on the\nrunning time of the feature computation using the\nsame SPP (ZF-5) model . The implementation of RCNN is from the code published by the authors implemented in Caffe [35]. We also implement our feature\ncomputation in Caffe . In Table 10 we evaluate the\naverage time of 100 random VOC images using GPU.\nR-CNN takes 14.37s per image for convolutions, while\nour 1-scale version takes only 0.053s per image. So\nours is 270\u0002faster than R-CNN. Our 5-scale version\ntakes 0.293s per image for convolutions, so is 49 \u0002\nfaster than R-CNN. Our convolutional feature computation is so fast that the computational time of fc layers\ntakes a considerable portion. Table 10 shows that the\nGPU time of computing the 4,096-d fc 7features is\n0.089s per image. Considering both convolutional and\nfully-connected features, our 1-scale version is 102\u0002\nfaster than R-CNN and is 1.2% inferior; our 5-scale"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1901-6db4-11ee-b1b5-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-56", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "fully-connected features, our 1-scale version is 102\u0002\nfaster than R-CNN and is 1.2% inferior; our 5-scale\nversion is 38\u0002faster and has comparable results.\nWe also compares the running time in Table 9 where\nR-CNN uses AlexNet [3] as is in the original paper\n[7]. Our method is 24 \u0002to 64\u0002faster. Note that the\nAlexNet [3] has the same number of \ufb01lters as our ZF5 on each conv layer. The AlexNet is faster because\nit uses splitting on some layers, which was designed\nfor two GPUs in [3].\nWe further achieve an ef\ufb01cient full system with the\nhelp of the recent window proposal method [25]. The\nSelective Search (SS) proposal [20] takes about 1-2 seconds per image on a CPU. The method of EdgeBoxes\n[25] only takes\u00180.2s. Note that it is suf\ufb01cient to use\na fast proposal method during testing only. Using the\nsame model trained as above (using SS), we test proposals generated by EdgeBoxes only. The mAP is 52.8\nwithout bounding box regression. This is reasonableconsidering that EdgeBoxes are not used for training.\nThen we use both SS and EdgeBox as proposals in"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1902-6db4-11ee-b0cc-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-57", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "without bounding box regression. This is reasonableconsidering that EdgeBoxes are not used for training.\nThen we use both SS and EdgeBox as proposals in\nthe training stage, and adopt only EdgeBoxes in the\ntesting stage. The mAP is 56.3 without bounding box\nregression, which is better than 55.2 (Table 10) due to\nadditional training samples. In this case, the overall\ntesting time is\u00180.5s per image including all steps\n(proposal and recognition). This makes our method\npractical for real-world applications.\n4.4 Model Combination for Detection\nModel combination is an important strategy for boosting CNN-based classi\ufb01cation accuracy [3]. We propose a simple combination method for detection.\nWe pre-train another network in ImageNet, using\nthe same structure but different random initializations. Then we repeat the above detection algorithm.\nTable 12 (SPP-net (2)) shows the results of this network. Its mAP is comparable with the \ufb01rst network\n(59.1% vs. 59.2%), and outperforms the \ufb01rst network\nin 11 categories.\nGiven the two models, we \ufb01rst use either model\nto score all candidate windows on the test image.\nThen we perform non-maximum suppression on the\nunion of the two sets of candidate windows (with"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1903-6db4-11ee-8efc-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-58", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "to score all candidate windows on the test image.\nThen we perform non-maximum suppression on the\nunion of the two sets of candidate windows (with\ntheir scores). A more con\ufb01dent window given by\none method can suppress those less con\ufb01dent given\nby the other method. After combination, the mAP\nis boosted to 60.9% (Table 12). In 17 out of all 20\ncategories the combination performs better than either\nindividual model. This indicates that the two models\nare complementary.\nWe further \ufb01nd that the complementarity is mainly\nbecause of the convolutional layers. We have tried to\ncombine two randomly initialized \ufb01ne-tuned results\nof the same convolutional model, and found no gain.\n12\n4.5 ILSVRC 2014 Detection\nThe ILSVRC 2014 detection [26] task involves 200\ncategories. There are \u0018450k/20k/40k images in the\ntraining/validation/testing sets. We focus on the task\nof the provided-data-only track (the 1000-category\nCLS training data is not allowed to use).\nThere are three major differences between the detection (DET) and classi\ufb01cation (CLS) training datasets,\nwhich greatly impacts the pre-training quality. First,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1904-6db4-11ee-8bf5-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-59", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "There are three major differences between the detection (DET) and classi\ufb01cation (CLS) training datasets,\nwhich greatly impacts the pre-training quality. First,\nthe DET training data is merely 1/3 of the CLS\ntraining data. This seems to be a fundamental challenge of the provided-data-only DET task. Second, the\ncategory number of DET is 1/5 of CLS. To overcome\nthis problem, we harness the provided subcategory\nlabels2for pre-training. There are totally 499 nonoverlapping subcategories ( i.e., the leaf nodes in the\nprovided category hierarchy). So we pre-train a 499category network on the DET training set. Third, the\ndistributions of object scales are different between\nDET/CLS training sets. The dominant object scale in\nCLS is about 0.8 of the image length, but in DET is\nabout 0.5. To address the scale difference, we resize\neach training image to min(w;h) = 400 (instead of\n256), and randomly crop 224\u0002224views for training.\nA crop is only used when it overlaps with a ground\ntruth object by at least 50%.\nWe verify the effect of pre-training on Pascal VOC\n2007. For a CLS-pre-training baseline, we consider"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1905-6db4-11ee-8af0-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-60", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "truth object by at least 50%.\nWe verify the effect of pre-training on Pascal VOC\n2007. For a CLS-pre-training baseline, we consider\nthe pool 5features (mAP 43.0% in Table 9). Replaced\nwith a 200-category network pre-trained on DET, the\nmAP signi\ufb01cantly drops to 32.7%. A 499-category\npre-trained network improves the result to 35.9%.\nInterestingly, even if the amount of training data\ndo not increase, training a network of more categories boosts the feature quality. Finally, training with\nmin(w;h) = 400 instead of 256 further improves the\nmAP to 37.8%. Even so, we see that there is still a\nconsiderable gap to the CLS-pre-training result. This\nindicates the importance of big data to deep learning.\nFor ILSVRC 2014, we train a 499-category Overfeat7 SPP-net. The remaining steps are similar to the\nVOC 2007 case. Following [7], we use the validation\nset to generate the positive/negative samples, with\nwindows proposed by the selective search fast mode.\nThe training set only contributes positive samples\nusing the ground truth windows. We \ufb01ne-tune the fc\nlayers and then train the SVMs using the samples in"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1906-6db4-11ee-967a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-61", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "The training set only contributes positive samples\nusing the ground truth windows. We \ufb01ne-tune the fc\nlayers and then train the SVMs using the samples in\nboth validation and training sets. The bounding box\nregression is trained on the validation set.\nOur single model leads to 31.84% mAP in the\nILSVRC 2014 testing set [26]. We combine six similar\nmodels using the strategy introduced in this paper.\nThe mAP is 35.11% in the testing set [26]. This result\nranks #2 in the provided-data-only track of ILSVRC\n2014 (Table 13) [26]. The winning result is 37.21% from\n2. Using the provided subcategory labels is allowed, as is explicitly stated in the competition introduction.rank team mAP\n1 NUS 37.21\n2 ours 35.11\n3 UvA 32.02\n- (our single-model) (31.84)\n4 Southeast-CASIA 30.47\n5 1-HKUST 28.86\n6 CASIA CRIPAC 2 28.61\nTable 13: The competition results of ILSVRC 2014\ndetection (provided-data-only track) [26]. The best\nentry of each team is listed.\nNUS, which uses contextual information.\nOur system still shows great advantages on speed"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1907-6db4-11ee-ba59-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-62", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "entry of each team is listed.\nNUS, which uses contextual information.\nOur system still shows great advantages on speed\nfor this dataset. It takes our single model 0.6 seconds\n(0.5 for conv, 0.1 for fc, excluding proposals) per testing image on a GPU extracting convolutional features\nfrom all 5 scales. Using the same model, it takes 32\nseconds per image in the way of RCNN. For the 40k\ntesting images, our method requires 8 GPU \u0001hours to\ncompute convolutional features, while RCNN would\nrequire 15 GPU\u0001days.\n5 C ONCLUSION\nSPP is a \ufb02exible solution for handling different scales,\nsizes, and aspect ratios. These issues are important in\nvisual recognition, but received little consideration in\nthe context of deep networks. We have suggested a solution to train a deep network with a spatial pyramid\npooling layer. The resulting SPP-net shows outstanding accuracy in classi\ufb01cation/detection tasks and\ngreatly accelerates DNN-based detection. Our studies\nalso show that many time-proven techniques/insights\nin computer vision can still play important roles in\ndeep-networks-based recognition.\nAPPENDIX A\nIn the appendix, we describe some implementation\ndetails:\nMean Subtraction.\nThe 224\u0002224 cropped training/testing images are"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1908-6db4-11ee-bc97-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-63", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "APPENDIX A\nIn the appendix, we describe some implementation\ndetails:\nMean Subtraction.\nThe 224\u0002224 cropped training/testing images are\noften pre-processed by subtracting the per-pixel mean\n[3]. When input images are in any sizes, the \ufb01xedsize mean image is not directly applicable. In the\nImageNet dataset, we warp the 224 \u0002224 mean image\nto the desired size and then subtract it. In Pascal VOC\n2007 and Caltech101, we use the constant mean (128)\nin all the experiments.\nImplementation of Pooling Bins.\nWe use the following implementation to handle all\nbins when applying the network. Denote the width\nand height of the conv 5feature maps (can be the\nfull image or a window) as wandh. For a pyramid level with n\u0002nbins, the (i;j)-th bin is in the\nrange of [bi\u00001\nnwc;di\nnwe]\u0002[bj\u00001\nnhc;dj\nnhe]. Intuitively,\n13\nbottle: 0. 24person:1.20\nsheep:1.52\nchair:0.21\ndiningtable:0.78person:1.16pers on:1.05"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1909-6db4-11ee-a152-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-64", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "sheep:1.52\nchair:0.21\ndiningtable:0.78person:1.16pers on:1.05\npottedplant:0.21chair:4.79pottedplant:0.73\nc hair: 0. 33diningtable: 0.34\nchair:0.89\nbus:0.56\ncar:3.24\nca r:3.4 5pers on:1.52\ntrain:0.31\ntrain:1.62\npottedplant:0.33\npottedplant:0.78\nsofa:0.55tvmonitor:1.77\naeroplane: 0.45\naeroplane:1.40aeroplane:1.01\naeroplane:0.94aeroplane:0.93\naeroplane:0.91aeroplane:0.79\naeroplane:0.57 aeroplane:0.54\nperson:0.93\nperson:0.68\nhorse:1.73pers on:1.91\nboat:0.60pers on:3.20\ncar:2.52\nbus:2.01\ncar:0.93per son:4.16person:0.79\npers on:0.32\nhors e:1.68\nhors e:0.61\nhorse:1.29"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a190a-6db4-11ee-9bd9-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-65", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "pers on:0.32\nhors e:1.68\nhors e:0.61\nhorse:1.29\nper son:1.23\nc hair:0.87pers on:1.79\nper son:0.91\nsofa:0.22person:0.85\nsofa:0.58\nc ow:2.36\ncow:1.88co w:1 .86\ncow:1.82cow:1.39 cow:1.31\nca t:0.5 2person:1.02\nperson:0.40\nbicy cle:2.85bicy cle:2.71\nbicycle:2.04bicy cle:0.67person:3.35\nperson:2.39\npers on:2.11\npers on:0.27\nperson:0.22bus:1.42\nperson:3.29person:1.18\nbottle:1.15pottedplant:0.81\ns heep:1.81s heep:1.17\nsheep:0.81\nbird:0.24pott edplant:0.35 pot tedplant: 0. 20\ncar:1.31person:1.60\nperson:0.62\ndog:0.37person:0.38\ndog:0.99person:1.48"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a190b-6db4-11ee-a811-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-66", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "car:1.31person:1.60\nperson:0.62\ndog:0.37person:0.38\ndog:0.99person:1.48\npers on:0.22\ncow:0.80person:3.29\nperson:2.69pers on:2.42\nperson:1.05\nperson:0.92pers on:0.76\nbird:1.39\nbird:0.84\nbottle:1.20diningtable:0.96pers on:1.53pers on:1.52\npers on:0.73\ncar:0.12\ncar:0.11\ncar:0.04car:0.03\ncar:3.98car:1.95car:1.39\ncar:0.50\nbird:1.47\nsofa:0.41 person:2.15pers on:0 .86tvmonitor:2.24\nmotorbike:1.11motorbike:0.74person:1.36\npers on:1.10\nFigure 6: Example detection results of \u201cSPP-net ftfc 7bb\u201d on the Pascal VOC 2007 testing set (59.2% mAP).\nAll windows with scores >0 are shown. The predicted category/score are marked. The window color is"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a190c-6db4-11ee-a998-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-67", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "All windows with scores >0 are shown. The predicted category/score are marked. The window color is\nassociated with the predicted category. These images are manually selected because we \ufb01nd them impressive.\nVisit our project website to see all 4,952 detection results in the testing set.\nif rounding is needed, we take the \ufb02oor operation on\nthe left/top boundary and ceiling on the right/bottom\nboundary.\nMapping a Window to Feature Maps.\nIn the detection algorithm (and multi-view testing\non feature maps), a window is given in the image\ndomain, and we use it to crop the convolutional feature maps ( e.g., conv 5) which have been sub-sampled\nseveral times. So we need to align the window on the\nfeature maps.\nIn our implementation, we project the corner point\nof a window onto a pixel in the feature maps, such\nthat this corner point in the image domain is closest\nto the center of the receptive \ufb01eld of that feature mappixel. The mapping is complicated by the padding\nof all convolutional and pooling layers. To simplify\nthe implementation, during deployment we pad bp=2c\npixels for a layer with a \ufb01lter size of p. As such, for\na response centered at (x0;y0), its effective receptive"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a190d-6db4-11ee-87bd-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-68", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "pixels for a layer with a \ufb01lter size of p. As such, for\na response centered at (x0;y0), its effective receptive\n\ufb01eld in the image domain is centered at (x;y) =\n(Sx0;Sy0)whereSis the product of all previous\nstrides. In our models, S= 16 for ZF-5 on conv 5,\nandS= 12 for Overfeat-5/7 on conv 5=7. Given a\nwindow in the image domain, we project the left (top)\nboundary by: x0=bx=Sc+ 1 and the right (bottom)\nboundaryx0=dx=Se\u00001. If the padding is not bp=2c,\nwe need to add a proper offset to x.\n14\nREFERENCES\n[1] Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard,\nW. Hubbard, and L. D. Jackel, \u201cBackpropagation applied to\nhandwritten zip code recognition,\u201d Neural computation , 1989."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a190e-6db4-11ee-a2a3-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-69", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "W. Hubbard, and L. D. Jackel, \u201cBackpropagation applied to\nhandwritten zip code recognition,\u201d Neural computation , 1989.\n[2] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. FeiFei, \u201cImagenet: A large-scale hierarchical image database,\u201d in\nCVPR , 2009.\n[3] A. Krizhevsky, I. Sutskever, and G. Hinton, \u201cImagenet classi\ufb01cation with deep convolutional neural networks,\u201d in NIPS ,\n2012.\n[4] M. D. Zeiler and R. Fergus, \u201cVisualizing and understanding\nconvolutional neural networks,\u201d arXiv:1311.2901 , 2013.\n[5] P . Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus,\nand Y. LeCun, \u201cOverfeat: Integrated recognition, localization\nand detection using convolutional networks,\u201d arXiv:1312.6229 ,\n2013.\n[6] A. V . K. Chat\ufb01eld, K. Simonyan and A. Zisserman, \u201cReturn of"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a190f-6db4-11ee-bfb1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-70", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "2013.\n[6] A. V . K. Chat\ufb01eld, K. Simonyan and A. Zisserman, \u201cReturn of\nthe devil in the details: Delving deep into convolutional nets,\u201d\ninArXiv:1405.3531 , 2014.\n[7] R. Girshick, J. Donahue, T. Darrell, and J. Malik, \u201cRich feature\nhierarchies for accurate object detection and semantic segmentation,\u201d in CVPR , 2014.\n[8] W. Y. Zou, X. Wang, M. Sun, and Y. Lin, \u201cGeneric object detection with dense neural patterns and regionlets,\u201d in\nArXiv:1404.4316 , 2014.\n[9] A. S. Razavian, H. Azizpour, J. Sullivan, and S. Carlsson, \u201cCnn\nfeatures off-the-shelf: An astounding baseline for recogniton,\u201d\ninCVPR 2014, DeepVision Workshop , 2014.\n[10] Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, \u201cDeepface:\nClosing the gap to human-level performance in face veri\ufb01cation,\u201d in CVPR , 2014."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1910-6db4-11ee-9dd2-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-71", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "Closing the gap to human-level performance in face veri\ufb01cation,\u201d in CVPR , 2014.\n[11] N. Zhang, M. Paluri, M. Ranzato, T. Darrell, and L. Bourdevr,\n\u201cPanda: Pose aligned networks for deep attribute modeling,\u201d\ninCVPR , 2014.\n[12] Y. Gong, L. Wang, R. Guo, and S. Lazebnik, \u201cMulti-scale\norderless pooling of deep convolutional activation features,\u201d\ninArXiv:1403.1840 , 2014.\n[13] J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng,\nand T. Darrell, \u201cDecaf: A deep convolutional activation feature\nfor generic visual recognition,\u201d arXiv:1310.1531 , 2013.\n[14] K. Grauman and T. Darrell, \u201cThe pyramid match kernel:\nDiscriminative classi\ufb01cation with sets of image features,\u201d in\nICCV , 2005.\n[15] S. Lazebnik, C. Schmid, and J. Ponce, \u201cBeyond bags of features: Spatial pyramid matching for recognizing natural scene"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1911-6db4-11ee-ae66-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-72", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "[15] S. Lazebnik, C. Schmid, and J. Ponce, \u201cBeyond bags of features: Spatial pyramid matching for recognizing natural scene\ncategories,\u201d in CVPR , 2006.\n[16] J. Sivic and A. Zisserman, \u201cVideo google: a text retrieval\napproach to object matching in videos,\u201d in ICCV , 2003.\n[17] J. Yang, K. Yu, Y. Gong, and T. Huang, \u201cLinear spatial pyramid\nmatching using sparse coding for image classi\ufb01cation,\u201d in\nCVPR , 2009.\n[18] J. Wang, J. Yang, K. Yu, F. Lv, T. Huang, and Y. Gong, \u201cLocalityconstrained linear coding for image classi\ufb01cation,\u201d in CVPR ,\n2010.\n[19] F. Perronnin, J. S \u00b4anchez, and T. Mensink, \u201cImproving the \ufb01sher\nkernel for large-scale image classi\ufb01cation,\u201d in ECCV , 2010.\n[20] K. E. van de Sande, J. R. Uijlings, T. Gevers, and A. W. Smeulders, \u201cSegmentation as selective search for object recognition,\u201d"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1912-6db4-11ee-bfb3-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-73", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "inICCV , 2011.\n[21] L. Fei-Fei, R. Fergus, and P . Perona, \u201cLearning generative\nvisual models from few training examples: An incremental\nbayesian approach tested on 101 object categories,\u201d CVIU ,\n2007.\n[22] M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and\nA. Zisserman, \u201cThe PASCAL Visual Object Classes Challenge\n2007 (VOC2007) Results,\u201d 2007.\n[23] P . F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan, \u201cObject detection with discriminatively trained partbased models,\u201d P AMI , 2010.\n[24] N. Dalal and B. Triggs, \u201cHistograms of oriented gradients for\nhuman detection,\u201d in CVPR , 2005.\n[25] C. L. Zitnick and P . Doll \u00b4ar, \u201cEdge boxes: Locating object\nproposals from edges,\u201d in ECCV , 2014.\n[26] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1913-6db4-11ee-8b94-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-74", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "[26] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh,\nS. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein\net al. , \u201cImagenet large scale visual recognition challenge,\u201d\narXiv:1409.0575 , 2014.\n[27] K. Chat\ufb01eld, V . Lempitsky, A. Vedaldi, and A. Zisserman, \u201cThe\ndevil is in the details: an evaluation of recent feature encoding\nmethods,\u201d in BMVC , 2011.\n[28] A. Coates and A. Ng, \u201cThe importance of encoding versus\ntraining with sparse coding and vector quantization,\u201d in ICML ,\n2011.\n[29] D. G. Lowe, \u201cDistinctive image features from scale-invariant\nkeypoints,\u201d IJCV , 2004.\n[30] J. C. van Gemert, J.-M. Geusebroek, C. J. Veenman, and A. W.\nSmeulders, \u201cKernel codebooks for scene categorization,\u201d in"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1914-6db4-11ee-a2b5-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-75", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "Smeulders, \u201cKernel codebooks for scene categorization,\u201d in\nECCV , 2008.[31] M. Lin, Q. Chen, and S. Yan, \u201cNetwork in network,\u201d\narXiv:1312.4400 , 2013.\n[32] C. Szegedy, W. Liu, Y. Jia, P . Sermanet, S. Reed, D. Anguelov,\nD. Erhan, V . Vanhoucke, and A. Rabinovich, \u201cGoing deeper\nwith convolutions,\u201d arXiv:1409.4842 , 2014.\n[33] K. Simonyan and A. Zisserman, \u201cVery deep convolutional\nnetworks for large-scale image recognition,\u201d arXiv:1409.1556 ,\n2014.\n[34] M. Oquab, L. Bottou, I. Laptev, J. Sivic et al. , \u201cLearning and\ntransferring mid-level image representations using convolutional neural networks,\u201d in CVPR , 2014.\n[35] Y. Jia, \u201cCaffe: An open source convolutional architecture\nfor fast feature embedding,\u201d http://caffe.berkeleyvision.org/,\n2013."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1915-6db4-11ee-bf4a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-76", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "for fast feature embedding,\u201d http://caffe.berkeleyvision.org/,\n2013.\n[36] A. G. Howard, \u201cSome improvements on deep convolutional\nneural network based image classi\ufb01cation,\u201d ArXiv:1312.5402 ,\n2013.\n[37] H. Jegou, F. Perronnin, M. Douze, J. Sanchez, P . Perez, and\nC. Schmid, \u201cAggregating local image descriptors into compact\ncodes,\u201d TP AMI , vol. 34, no. 9, pp. 1704\u20131716, 2012.\n[38] C.-C. Chang and C.-J. Lin, \u201cLibsvm: a library for support\nvector machines,\u201d ACM Transactions on Intelligent Systems and\nTechnology (TIST) , 2011.\n[39] X. Wang, M. Yang, S. Zhu, and Y. Lin, \u201cRegionlets for generic\nobject detection,\u201d in ICCV , 2013.\n[40] C. Szegedy, A. Toshev, and D. Erhan, \u201cDeep neural networks\nfor object detection,\u201d in NIPS , 2013.\nCHANGELOG\narXiv v1 . Initial technical report for ECCV 2014 paper."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1916-6db4-11ee-b03d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1406.4729-77", "title": "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", "source": "http://arxiv.org/pdf/1406.4729", "start_index": 0, "chroma:document": "for object detection,\u201d in NIPS , 2013.\nCHANGELOG\narXiv v1 . Initial technical report for ECCV 2014 paper.\narXiv v2 . Submitted version for TPAMI. Includes extra\nexperiments of SPP on various architectures. Includes\ndetails for ILSVRC 2014.\narXiv v3 . Accepted version for TPAMI. Includes comparisons with R-CNN using the same architecture.\nIncludes detection experiments using EdgeBoxes.\narXiv v4 . Revised \u201cMapping a Window to Feature\nMaps\u201d in Appendix for easier implementation."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1917-6db4-11ee-87a8-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-0", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "Accepted as a workshop contribution at ICLR 2015\nLEARNING DEEPSTRUCTURED MODELS\nLiang-Chieh Chen\u0003\nDepartment of Computer Science, UCLA, lcchen@cs.ucla.edu\nAlexander G. Schwing\u0003\nDepartment of Computer Science, University of Toronto, aschwing@cs.toronto.edu\nAlan L. Yuille\nDepartment of Statistics, UCLA, yuille@stat.ucla.edu\nRaquel Urtasun\nDepartment of Computer Science, University of Toronto, urtasun@cs.toronto.edu\nABSTRACT\nMany problems in real-world applications involve predicting several random variables which are statistically related. Markov random \ufb01elds (MRFs) are a great\nmathematical tool to encode such relationships. The goal of this paper is to combine MRFs with deep learning algorithms to estimate complex representations\nwhile taking into account the dependencies between the output random variables.\nTowards this goal, we propose a training algorithm that is able to learn structured\nmodels jointly with deep features that form the MRF potentials. Our approach is\nef\ufb01cient as it blends learning and inference and makes use of GPU acceleration.\nWe demonstrate the effectiveness of our algorithm in the tasks of predicting words\nfrom noisy images, as well as multi-class classi\ufb01cation of Flickr photographs. We"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1918-6db4-11ee-9144-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-1", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "We demonstrate the effectiveness of our algorithm in the tasks of predicting words\nfrom noisy images, as well as multi-class classi\ufb01cation of Flickr photographs. We\nshow that joint learning of the deep features and the MRF parameters results in\nsigni\ufb01cant performance gains.\n1 I NTRODUCTION\nDeep learning algorithms attempt to model high-level abstractions of the data using architectures composed of multiple non-linear transformations. A multiplicity of variants have been proposed (Hinton et al., 1984; LeCun et al., 1998; Hinton & Salakhutdinov, 2006; Bengio et al., 2007;\nSalakhutdinov & Hinton, 2012; Zeiler & Fergus, 2014) and shown to be extremely successful in a\nwide variety of applications including object detection, speech recognition as well as natural language processing (Lee et al., 2009; Socher et al., 2012; Jia, 2013; Krizhevsky et al., 2013; Eigen\net al., 2014). Recently, state-of-the-art results have been achieved in many computer vision tasks,\noutperforming competitive methods by a large margin (Krizhevsky et al., 2013; Girshick et al.,\n2014).\nDeep neural networks can, however, be even more powerful when combined with graphical models"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1919-6db4-11ee-ac45-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-2", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "2014).\nDeep neural networks can, however, be even more powerful when combined with graphical models\nin order to capture the statistical dependencies between the variables of interest. For example, Deng\net al. (2014) exploit mutual exclusion, overlapping and subsumption properties of class labels in order to better predict in large scale classi\ufb01cation tasks. In pose estimation, more accurate predictions\ncan be obtained when encoding the spatial relationships between joint locations (Tompson et al.,\n2014).\nIt is, however, an open problem how to develop scalable deep learning algorithms that can learn\nhigher-order knowledge taking into account the output variable\u2019s dependencies. Existing approaches\noften rely on a two-step process (Nowozin et al., 2011; Xu et al., 2014) where a non-linear classi\ufb01er\nthat employs deep features is trained \ufb01rst, and its output is used to generate potentials for the structured predictor. This piece-wise training is, however, suboptimal as the deep features are learned\n\u0003The \ufb01rst two authors contributed equally to this work.\n1arXiv:1407.2538v3  [cs.LG]  27 Apr 2015\nAccepted as a workshop contribution at ICLR 2015"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a191a-6db4-11ee-96ef-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-3", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "1arXiv:1407.2538v3  [cs.LG]  27 Apr 2015\nAccepted as a workshop contribution at ICLR 2015\nwhile ignoring the dependencies between the variables of interest, e.g., independently learned segmentation and detection features (Hariharan et al., 2014) might be focusing on predicting the same\nexamples correctly. But when learned jointly they can improve their predictive power by exploiting\ncomplementary information to \ufb01x additional mistakes.\nIn this paper we extend deep learning algorithms to learn complex representations taking into account the dependencies between the output random variables. Towards this goal, we propose a\nlearning algorithm that is able to learn structured models with arbitrary graphs jointly with deep\nfeatures that form Markov random \ufb01eld (MRF) potentials. Our approach is ef\ufb01cient as it blends\nlearning and inference resulting in a single loop algorithm which makes use of GPU acceleration.\nWe demonstrate the effectiveness of our method in the tasks of predicting words from noisy images,\nand multi-class classi\ufb01cation of Flickr photographs. We show that joint learning of deep features\nand MRF parameters results in big performance gains.\n2 L EARNING DEEPSTRUCTURED MODELS"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a191b-6db4-11ee-ac62-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-4", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "and MRF parameters results in big performance gains.\n2 L EARNING DEEPSTRUCTURED MODELS\nIn this section we investigate how to learn \u2018deep features\u2019 that take into account the dependencies between the output variables. Let y2Y be the set of random variables y= (y1;:::;yN)\nthat we are interested in predicting. In this work we assume the space of valid con\ufb01gurations to\nbe a product space, i.e.,Y=QN\ni=1Yi, and the domain of each individual variable yito be discrete, i.e.,Yi=f1;:::;jYijg. Given input data x2X and parameters w2RAof the function\nF(x;y;w) :X\u0002Y\u0002 RA!R, inference amounts to \ufb01nding the highest scoring con\ufb01guration\ny\u0003= arg max yF(x;y;w):Note that ifFis a deep network and there are no connections between\nthe output variables to be predicted, inference corresponds to a forward pass to evaluate the function,\nfollowed by independently \ufb01nding the largest response for each variable. This can be interpreted as\ninference in a graphical model with only unary potentials. However, for arbitrary graphical models"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a191c-6db4-11ee-b830-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-5", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "inference in a graphical model with only unary potentials. However, for arbitrary graphical models\nit is NP-hard to \ufb01nd the maximizing con\ufb01guration y\u0003since the inference program generally requires\na search over a space of sizeQN\ni=1jYij. Note also that log-linear models are a special case of this\nprogram, with F(x;y;w) =w>\u001e(x;y)and\u001e(x;y)denoting a feature vector computed using the\ninput-output pair (x;y).\nIn this work, we consider the general setting where F(x;y;w)is an arbitrary scalar-valued function ofwand(x;y). In our experiments Fis a function composition of non-linear base mappings\nlike convolutions, recti\ufb01cations, pooling etc. We let the probability of an arbitrary con\ufb01guration ^y\nbe given by the annealed soft-max p(x;y)(^yjw;\u000f) =1\nZ\u000f(x;w)exp(F(x;^y;w))1=\u000f:HerebyZ\u000f(x;w)\nrefers to the partition function, normalizing the distribution p(x;y)to lie within the probability simplex\u0001viaZ(x;w) =P"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a191d-6db4-11ee-b563-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-6", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "refers to the partition function, normalizing the distribution p(x;y)to lie within the probability simplex\u0001viaZ(x;w) =P\n^y2Yexp(F(x;^y;w))1=\u000f. The annealing/temperature parameter \u000f\u00150is\nused to adjust the uniformity of the distribution. We consider general graphical models where the\ncomputation of Z\u000f(x;w)is #P-hard.\n2.1 L EARNING VIA GRADIENT DESCENT\nDuring learning, given a training set Dof input-output pairs (x;y)2D, we are interested in \ufb01nding\nthe parameters wof the model. We do so by maximizing the data likelihood, i.e., minimizing the\nnegative log-likelihood \u0000lnQ\n(x;y)2Dp(x;y)(yjw;\u000f)which yields\nmin\nwX\n(x;y)2D(\u000flnZ\u000f(x;w)\u0000F(x;y;w)): (1)\nNote that this is equivalent to maximizing the cross-entropy between a target distributionp(x;y);tg(^y) =\u000e(^y=y)placing all its mass on the groundtruth label,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a191e-6db4-11ee-be10-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-7", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "and the model distribution p(x;y)(^yjw;\u000f). Hence Eq. (1) is equivalently obtained by\nmaxwP\n(x;y);^y2Yp(x;y);tg(^y) lnp(x;y)(^yjw;\u000f). It is easily possible to incorporate more general target distributions into Eq. (1). Note also that \u000f= 0recovers the structured hinge loss objective.\nMinimizing Eq. (1) w.r.t. wrequires computation of the gradient@\n@wP\n(x;y)2D\u0000lnp(x;y)(yjw;\u000f),\nwhich is given by a transformed difference between the distributions of the model p(x;y)(^yjw;\u000f)and\n2\nAccepted as a workshop contribution at ICLR 2015\nRepeat until stopping criteria\n1. Forward pass to compute F(x;^y;w)\n2. Compute p(x;y)(^yjw;\u000f)\n3. Backward pass via chain rule to obtain gradient\n4. Update parameters w\nFigure 1: Gradient descent algorithm for learning deep structured models.\nthe targetp(x;y);tg(^y):\nX\n(x;y)2DX\n^y2Y@"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a191f-6db4-11ee-aa61-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-8", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "the targetp(x;y);tg(^y):\nX\n(x;y)2DX\n^y2Y@\n@wF(x;^y;w)\u0000\np(x;y)(^yjw;\u000f)\u0000p(x;y);tg(^y)\u0001\n: (2)\nA gradient descent algorithm for minimizing Eq. (1) will iterate between the following steps: (i) For\na givenwevaluate the function F, (ii) compute the model distribution p(x;y)(^yjw;\u000f), (iii) propagate\nthe difference between the model and target distribution using a backward pass (resembling the chain\nrule for composite functions) and (iv) update the parameters w. This is summarized in Fig. 1.\n2.2 A PPROXIMATE LEARNING\nNote that for general graphical models the exact computation of p(x;y)(^yjw;\u000f)is not possible. As a\nconsequence it is intractable to compute the exact gradient of the cost-function given in Eq. (2) and\none has to resort to approximate solutions.\nInspired by approximations used for log-linear models, we make use of the following identity (Wainwright & Jordan, 2008; Koller & Friedman, 2009):"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1920-6db4-11ee-8870-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-9", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "Inspired by approximations used for log-linear models, we make use of the following identity (Wainwright & Jordan, 2008; Koller & Friedman, 2009):\n\u000flnZ\u000f(x;w) = max\np(x;y)(^y)2\u0001E[F(x;^y;w)] +\u000fH(p(x;y)); (3)\nwhere Edenotes an expectation over p(x;y)(^y)andHrefers to the entropy.\nFor most applications, F(x;y;w)decomposes into a sum of functions, each depending on a local\nsubset of variables yr,i.e.,F(x;y;w) =P\nr2Rfr(x;yr;w):Herebyris a restriction of the variable\ntupley= (y1;:::;yN)to the subset r\u0012f1;:::;Ng,i.e.,yr= (yi)i2r. All subsets rrequired to\ncompute the model function Fare summarized in the set R.\nPlugging this decomposition into Eq. (3), we equivalently get the log-partition function\n\u000flnZ\u000f(x;w)viamaxp(x;y)(^y)2\u0001P"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1921-6db4-11ee-ba40-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-10", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "\u000flnZ\u000f(x;w)viamaxp(x;y)(^y)2\u0001P\nr;^yrp(x;y);r(^yr)fr(x;^yr;w) +\u000fH(p(x;y));where we use\nmarginalsp(x;y);r(^yr) =P\nynyrp(x;y)(y).\nDespite the assumed locality of the scoring function, the learning task remains computationally\nchallenging since the entropy H(p(x;y))can only be computed exactly for a very small set of models,\ne.g., models for which the joint distribution p(x;y)(y)is equivalently described by low tree-width\nmodels. In addition, the marginalization constraints are exponential in size.\nTo deal with both issues a common solution in log-linear models is to approximate the true marginals\np(x;y);rwith local beliefs b(x;y);rthat are not required to ful\ufb01ll marginalization constraints globally,\nbut only locally (Wainwright & Jordan, 2008). That means marginals b(x;y);rare not required to arise\nfrom a common joint distribution p(x;y). In addition, we approximate the entropy via the fractional"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1922-6db4-11ee-abfc-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-11", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "from a common joint distribution p(x;y). In addition, we approximate the entropy via the fractional\nentropy (Wiegerinck & Heskes, 2003), i.e.,H(p(x;y))\u0019P\nrcrH(b(x;y);r). Counting numbers cr\nare employed to weight the marginal entropies. Putting all this together, we obtain the following\napproximation for \u000flnZ\u000f(x;w):\nmax\nb(x;y)2C(x;y)X\nr;^yrb(x;y);r(^yr)fr(x;^yr;w) +X\nr\u000fcrH(b(x;y);r): (4)\nHereby beliefs are constrained to the local polytope\nC(x;y)=(8r b (x;y);r2\u0001\n8r;^yr;p2P(r)P\n^ypn^yrb(x;y);p(^yp) =b(x;y);r(^yr);\n3\nAccepted as a workshop contribution at ICLR 2015\nLet~cr;p=cp=(cr+P\np02P(r)cp0). Repeat until stopping criteria"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1923-6db4-11ee-b98d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-12", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "3\nAccepted as a workshop contribution at ICLR 2015\nLet~cr;p=cp=(cr+P\np02P(r)cp0). Repeat until stopping criteria\n1. Forward pass to compute fr(x;^yr;w)8(x;y);r;yr\n2. Get beliefs b(x;y);r/exp^fr(x;^yr;w;\u0015)\n\u000fcrby iterating over r:8(x;y);p2P(r);^yr\n\u0016(x;y);p!r(^yr) =\u000fcplnX\n^ypn^yrexpfp(x;^yp;w)\u0000P\np02P(p)\u0015(x;y);p!p0(^yp) +P\nr02C(p)nr\u0015(x;y);r0!p(^yr0)\n\u000fcp\n\u0015(x;y);r!p(^yr)/~cr;p0\n@fr(x;^yr;w) +X\nc2C(r)\u0015(x;y);c!r(^yc) +X\np2P(r)\u0016(x;y);p!r(^yr)1\nA\u0000\u0016(x;y);p!r(^yr)"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1924-6db4-11ee-8308-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-13", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "p2P(r)\u0016(x;y);p!r(^yr)1\nA\u0000\u0016(x;y);p!r(^yr)\n3. Backward pass via chain-rule for gradient g=P\n(x;y);r;^yrb(x;y);r(^yr)rwfr(x;^yr;w)\u0000rwF(w)\n4. Update parameters wusing stepsize \u0011viaw w\u0000\u0011g\nFigure 2: Ef\ufb01cient learning algorithm that blends learning and inference.\nwithP(r)the set of parents of region r,i.e.,P(r)\u0012fp2R :r\u001apg, which subsumes those\nregions for which we want the marginalization constraint to hold. Conversely, we de\ufb01ne the set of\nchildren asC(r) =fc2R:r2P(c)g.\nWe can thus rewrite the learning problem by plugging the approximations derived in Eq. (4) into\nEq. (1). This gives rise to the new approximated learning program\nmin\nwX\n(x;y)2D0\n@max\nb(x;y)2C(x;y)8\n<\n:X"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1925-6db4-11ee-8b93-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-14", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "min\nwX\n(x;y)2D0\n@max\nb(x;y)2C(x;y)8\n<\n:X\nr;^yrb(x;y);r(^yr)fr(x;^yr;w) +X\nr\u000fcrH(b(x;y);r)9\n=\n;\u0000F(x;y;w)1\nA:(5)\nTo iteratively update the parameters for the non-smooth approximated cost function given in Eq. (5)\nwe require a sub-gradient w.r.t. w, which in turn requires to solve the maximization w.r.t. the beliefs\nbexactly. This is a non-trivial task in itself as inference in general graphical models is NP-hard.\nIterative message passing algorithms (Pearl, 1988; Yedidia et al., 2005; Wainwright et al., 2005;\nWeiss et al., 2007; Meltzer et al., 2009) are typically employed. Importantly, note that combining\nthe procedure outlined in Fig. 1 with iterative message passing to approximate p(x;y)(^yjw;\u000f)results\nin a double-loop algorithm which would be slow for many graphical models of interest."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1926-6db4-11ee-920b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-15", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "in a double-loop algorithm which would be slow for many graphical models of interest.\n2.3 E FFICIENT APPROXIMATE LEARNING BY BLENDING LEARNING AND INFERENCE\nIn this section we propose a more ef\ufb01cient algorithm that is based on the principle of blending\nlearning ( i.e., parameter updates) and inference. Thus we are interested in only performing a single\nmessage passing iteration before updating the parameters w. Note that simply reducing the number\nof iterations is generally not an option as the obtained beliefs b(x;y);rare by no means accurate.\nHowever, assuming all counting numbers crto be positive, we can derive an algorithm that is able\nto interleave minimization w.r.t. wand maximization of the beliefs b. Such a procedure is more\nef\ufb01cient as we are able to update the parameters wmuch more frequently.\nTo interleave both programs we convert maximization of the beliefs into a minimization by employing the dual program as detailed in the following claim. This is possible since the maximization\nproblem is concave in b(x;y)if8r,\u000fcr\u00150.\nClaim 1 Assume\u000fcr\u001508rand letF(w) =P\n(x;y)2DF(x;y;w)denote the sum of empirical"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1927-6db4-11ee-98fe-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-16", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "Claim 1 Assume\u000fcr\u001508rand letF(w) =P\n(x;y)2DF(x;y;w)denote the sum of empirical\nfunction observations. Let \u0015(x;y);r!p(^yr)be the Lagrange multipliers for each marginalization\nconstraintP\n^ypn^yrb(x;y);p(^yp) =b(x;y);r(^yr)within the polytope C(x;y). Then the approximated\ngeneral structured prediction task shown in Eq. (5)is equivalent to\nmin\nw;\u0015X\n(x;y);r\u000fcrlnX\n^yrexp^fr(x;^yr;w;\u0015)\n\u000fcr\u0000F(w); (6)\n4\nAccepted as a workshop contribution at ICLR 2015\nbanal julep resty\ndrein yojan mothy\nFigure 3: Samples from the Word50 dataset. High degree of rotation, scaling and translation.\nwhere we employed the re-parameterization score ^fr(x;^yr;w;\u0015) =fr(x;^yr;w) +P\nc2C(r)\u0015(x;y);c!r(^yc)\u0000P"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1928-6db4-11ee-b229-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-17", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "c2C(r)\u0015(x;y);c!r(^yc)\u0000P\np2P(r)\u0015(x;y);r!p(^yr).\nProof: To obtain the dual of the maximization w.r.t. b(x;y)we utilize its Lagrangian L(x;y)=P\nr;^yrb(x;y);r(^yr)^fr(x;^yr;w;\u0015)+P\nr\u000fcrH(b(x;y);r):Maximization of the Lagrangian w.r.t. the primal variables bis possible by employing the relationship stated in Eq. (3) locally 8r. We then obtain\nthe dual function being the \ufb01rst term in Eq. (6). For strict convexity, i.e.,\u000fcr>0, we reconstruct the beliefs to be proportional to the exponentiated, loss-augmented re-parameterization score\nb(x;y);r/exp^fr(x;^yr;w;\u0015)\n\u000fcr:For\u000fcr= 0the beliefs correspond to a uniform distribution over the set\nof maximizers of the loss-augmented re-parameterization score ^fr(x;^yr;w;\u0015). \u0004"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1929-6db4-11ee-b305-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-18", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "of maximizers of the loss-augmented re-parameterization score ^fr(x;^yr;w;\u0015). \u0004\nIt is important to note that by applying duality we managed to convert the min-max task in Eq. (5)\ninto a single minimization as shown in Eq. (6). Performing block coordinate descent updates to\nminimize Eq. (6), we are therefore able to interleave both, updating the weights ( i.e., learning) and\nthe messages ( i.e., inference). This results in a more ef\ufb01cient algorithm, as inference does not have\nto be run until convergence. Even a single update of the messages suf\ufb01ces. We note that this is\npossible only if \u000fcr\u001508r. Strictly speaking, we require concavity only within the set of feasible\nbeliefsC(x;y). However, for simplicity we neglect this extension in the following.\nFig. 2 summarizes our ef\ufb01cient deep structured prediction algorithm which iterates between the\nfollowing steps. Given parameters wwe perform a standard forward pass to compute fr(x;^yr;w)\nfor all regions. We then iterate through all regions rand use block-coordinate descent to \ufb01nd the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a192a-6db4-11ee-8b17-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-19", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "for all regions. We then iterate through all regions rand use block-coordinate descent to \ufb01nd the\nglobally optimal value of Eq. (6) w.r.t. \u0015(x;y);r!p(^yr)8(x;y);^yr;p2P(r). This can be done in\nclosed form and therefore is computed very ef\ufb01ciently. We refer the reader to Schwing (2013) for\na derivation in the log-linear setting. We then compute the gradient using a standard backward pass\nbefore we update the parameters by performing a step of size \u0011along the negative gradient.\n2.4 I MPLEMENTATION DETAILS\nWe implemented the general algorithm presented in Fig. 2 in C++ as a library for Linux, Windows\nand OS X platforms. It supports usage of the GPU for the forward and backward pass using both,\nstandard linear algebra packages and manually tuned GPU-kernels. In addition to standard gradient descent, we allow speci\ufb01cation of both mini-batches, moments and different regularizers like\n2-norm and1-norm. Between iterations the step-size can be reduced based on either the negative log-likelihood or validation set performance. Contrasting available deep learning packages, our"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a192b-6db4-11ee-8e43-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-20", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "2-norm and1-norm. Between iterations the step-size can be reduced based on either the negative log-likelihood or validation set performance. Contrasting available deep learning packages, our\nfunctionFis speci\ufb01ed using a general computation tree. Hence we support an arbitrarily nested\nfunction structure composed of data, parameters and function prototypes (convolution, af\ufb01ne function aka fully connected, dropout, local response normalization, pooling, recti\ufb01ed linear, sigmoid\nand softmax units). The aforementioned library is accompanied by a program performing learning,\ninference and gradient checks. To accommodate for large datasets it reads data from HDF5 storage\nwhile a second thread simultaneously performs the computation. Google protocol buffers are employed to effectively specify the function Fwithout the need to modify any source code. We will\nrelease this library upon publication. We believe that it will be useful for many researchers.\n5\nAccepted as a workshop contribution at ICLR 2015\nGraph MLP Method H1= 128H1= 256H1= 512H1= 768H1= 1024\n1st order Markov One LayerUnary only 8.60 / 61.32 10.80 / 64.41 12.50 / 65.69 12.95 / 66.66 13.40 / 67.02"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a192c-6db4-11ee-a152-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-21", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "JointTrain 16.80 / 65.28 25.20 / 70.75 31.80 / 74.90 33.05 / 76.42 34.30 / 77.02\nPwTrain 12.70 / 64.35 18.00 / 68.27 22.80 / 71.29 23.25 / 72.62 26.30 / 73.96\nPreTrainJoint 20.65 /67.42 25.70 /71.65 31.70 / 75.56 34.50 /77.14 35.85 /78.05\n2nd order Markov One LayerJointTrain 25.50 / 67.13 34.60 / 73.19 45.55 / 79.60 51.55 /82.37 54.05 /83.57\nPwTrain 10.05 / 58.90 14.10 / 63.44 18.10 / 67.31 20.40 / 70.14 22.20 / 71.25\nPreTrainJoint 28.15 /69.07 36.85 /75.21 45.75 /80.09 50.10 / 82.30 52.25 / 83.39\nH1= 512H2= 32H2= 64H2= 128H2= 256H2= 512"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a192d-6db4-11ee-b062-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-22", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "H1= 512H2= 32H2= 64H2= 128H2= 256H2= 512\n1st order Markov Two LayerUnary only 15.25 / 69.04 18.15 / 70.66 19.00 / 71.43 19.20 / 72.06 20.40 / 72.51\nJointTrain 35.95 / 76.92 43.80 / 81.64 44.75 / 82.22 46.00 / 82.96 47.70 / 83.64\nPwTrain 34.85 / 79.11 38.95 / 80.93 42.75 / 82.38 45.10 / 83.67 45.75 / 83.88\nPreTrainJoint 42.25 /81.10 44.85 /82.96 46.85 /83.50 47.95 /84.21 47.05 / 84.08\n2nd order Markov Two LayerJointTrain 54.65 / 83.98 61.80 / 87.30 66.15 / 89.09 64.85 / 88.93 68.00 / 89.96\nPwTrain 39.95 / 81.14 48.25 / 84.45 52.65 / 86.24 57.10 / 87.61 62.90 / 89.49"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a192e-6db4-11ee-946c-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-23", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "PwTrain 39.95 / 81.14 48.25 / 84.45 52.65 / 86.24 57.10 / 87.61 62.90 / 89.49\nPreTrainJoint 62.60 /88.03 65.80 /89.32 68.75 /90.47 68.60 /90.42 69.35 /90.75\nTable 1: Word / Character accuracy. Performance improves as (1) joint-training is employed, (2) the\nmodel is more structured, and (3) deeper unary classi\ufb01ers are utilized. The number of hidden units\nfor the \ufb01rst and second layer are denoted as H1andH2respectively.\nabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz\nabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz\n00.511.522.533.544.55\nx 1043004005006007008009001000Neg. Log\u2212Likelihood\nIteration  \nJointTrain\nPwTrain\nPreTrainJoint\nUnary weights distance-1 edges distance-2 edges Neg. Log-Likelihood"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a192f-6db4-11ee-b6f7-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-24", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "Iteration  \nJointTrain\nPwTrain\nPreTrainJoint\nUnary weights distance-1 edges distance-2 edges Neg. Log-Likelihood\nFigure 4: (left) Subset of the learned unary weights. Pairwise weights (middle two panels), the\ndarker, the larger the weight. (right) Negative log-likelihood for different learning approaches.\n3 E XPERIMENTAL EVALUATION\nWe demonstrate the performance of our model on two tasks: word recognition and image classi\ufb01cation. We investigate four strategies to learn the model parameters. \u2018 Unary only \u2019 denotes training\nonly unary classi\ufb01ers while ignoring the structure of the graphical model, i.e., pairwise weights are\nequal to 0. \u2018JointTrain \u2019 initializes all weights at random and trains them jointly. \u2018 PwTrain \u2019 uses\npiecewise training by \ufb01rst training the unary potentials and then keeping them \ufb01xed when learning the pairwise potentials. \u2018 PreTrainJoint \u2019 pre-trains the unaries but jointly optimizes pairwise\nweights as well as unary weights in a second step.\n3.1 W ORD RECOGNITION : W ORD50\nOur \ufb01rst task consists of word recognition from noisy images. Towards this goal, we created a"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1930-6db4-11ee-b486-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-25", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "3.1 W ORD RECOGNITION : W ORD50\nOur \ufb01rst task consists of word recognition from noisy images. Towards this goal, we created a\nchallenging dataset by randomly selecting 50 words, each consisting of \ufb01ve characters. We then\ngenerated writing variations of each word as follows: we took the lower case characters from the\nChars74K dataset (de Campos et al., 2009), and inserted them in random background image patches\n(similar to Larochelle et al. (2007)) by alpha matting, i.e., characters have transparency. To increase\nthe dif\ufb01culty, we perturbed each character image of size 28\u000228by scaling, rotation and translation.\nAs shown in Fig. 3 the task is very challenging, some characters are fairly dif\ufb01cult to recognize even\nfor humans. We denote the resulting dataset \u2018 Word50 .\u2019 The training, validation and test sets have\n10;000,2;000and2;000variations of words respectively.\nWe experimented with graphical models composed of unary and pairwise regions de\ufb01ned over \ufb01ve"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1931-6db4-11ee-87e8-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-26", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "We experimented with graphical models composed of unary and pairwise regions de\ufb01ned over \ufb01ve\nrandom variables, one per character. We encode unary potentials fr(x;yi;wu)using multi-layer perceptrons (MLPs) with recti\ufb01ed linear units (ReLU). Unless otherwise stated, we de\ufb01ne all pairwise\n6\nAccepted as a workshop contribution at ICLR 2015\nH1=128 H1=256 H1=512 H1=768 H1=1024161820222426283032343638\n  \nLinear\nPairH16\nPairH32\nPairH64\nH2=32 H2=64 H2=128 H2=256 H2=512343638404244464850\n  \nLinear\nPairH16\nPairH32\nPairH64\nOne-Layer MLP Chain Two-Layer MLP Chain\nFigure 5: Learning non-linear pairwise functions: Word recognition as a function of the number\nof hidden units for the unary potential. Colors represent different number of hidden units for the\npairwise potentials. The y-axis shows the word accuracy of using Linear function, or 16 (PairH16),\n32 (PairH32), and 64 (PairH64) hidden units for the pairwise function."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1932-6db4-11ee-8343-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-27", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "32 (PairH32), and 64 (PairH64) hidden units for the pairwise function.\nfemale/indoor/portrait sky/plant life/tree water/animals/sea animals/dog/indoor indoor/\ufb02ower/plant life\nfemale/indoor/portrait sky/plant life/tree water/animals/sky animals/dog ;\nFigure 6: Flickr test set images and some assigned tags as well as our predictions (bottom row).\ninteractions via\nfr(x;yi;yj;wp) =X\nmnWmn\u0001\u000e(yi=m;yj=n); (7)\nwherer=fi;jg,wp=fWg,Wmnis the element of matrix W, and\u000erefers to the indicator\nfunction.\nFor all experiments, we share all unary weights across the nodes of the graphical model as well as all\npairwise weights for all edges. Note that due to the use of ReLU units, the negative log-likelihood\nis non-smooth, non-linear and non-convex w.r.t. w. Because of the non-smoothness of F, we utilize\nmomentum based sub-gradient descent methods to estimate the weights. In particular, we use a"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1933-6db4-11ee-88a2-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-28", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "momentum based sub-gradient descent methods to estimate the weights. In particular, we use a\nmini-batch size of 100, a step size of 0:01and a momentum of 0:95. If the unary potential is pretrained, the initial step size is reduced to 0:001. All the unary classi\ufb01ers are trained with 100;000\niterations over mini-batches. For all experiments, the validation set is only used to decrease the\nstep size, i.e., if the accuracy on the validation set decreases, we reduce the step size by 0:5. We\nuse\u000f= 1, setcr= 1 for all regions r, and perform 10 message passing iterations to compute the\nmarginal beliefs b(x;y);rat step 2 in Fig. 2 when dealing with loopy models.\nWe experiment with two graphical models, Markov models of \ufb01rst ( i.e., there are links only between\nyiandyi+1) and second order ( i.e., there are links between yiandyi+1,yi+2) as well as two types of\nunary potentials with varying degree of structure. We report two metrics, the average character and\nword accuracy, which correspond to Hamming loss and zero-one loss respectively. Table 1 depicts\nthe results for the different models, learning strategies and number of hidden units. We observe the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1934-6db4-11ee-b335-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-29", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "word accuracy, which correspond to Hamming loss and zero-one loss respectively. Table 1 depicts\nthe results for the different models, learning strategies and number of hidden units. We observe the\nfollowing trends.\nJoint training helps: Joint training with pre-trained unary classi\ufb01ers (PreTrainJoint) outperforms\nall the other approaches in almost all cases. The piecewise training method (PwTrain), unable to\nadapt the non-linearities while learning pairwise weights, often leads to performance worse than\njoint training.\nStructure helps: Adding structure to the model is key to capture complex dependencies. As shown\nin Table 1, more structured models ( i.e., second order Markov model) consistently improves performance.\n7\nAccepted as a workshop contribution at ICLR 2015\n0.00 0.68 0.04 0.06 0.02 0.24 0.03 \u22120.00 \u22120.01 0.01 0.04 \u22120.00 \u22120.05 \u22120.01 0.07 \u22120.01 \u22120.00 \u22120.12 0.04 0.01 0.01 0.02 0.04 0.02"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1935-6db4-11ee-865f-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-30", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "0.68 0.00 0.06 0.06 \u22120.00 0.36 0.03 \u22120.08 \u22120.05 \u22120.03 0.02 \u22120.06 \u22120.12 \u22120.05 0.74 \u22120.04 \u22120.03 \u22120.21 0.01 \u22120.03 \u22120.03 \u22120.03 0.05 \u22120.03\n0.04 0.06 0.00 0.05 \u22120.06 0.07 \u22120.12 \u22120.07 \u22120.35 \u22120.03 \u22120.46 \u22120.02 \u22120.34 0.11 0.02 \u22120.15 \u22120.14 \u22120.01 \u22120.07 \u22120.21 0.03 \u22120.08 0.06 \u22120.03\n0.06 0.06 0.05 0.00 0.10 0.11 0.07 0.09 0.03 0.10 0.01 0.10 0.02 0.09 0.06 0.08 0.07 0.07 0.08 0.06 0.09 0.09 0.08 0.10"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1936-6db4-11ee-8ee0-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-31", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "0.02 \u22120.00 \u22120.06 0.10 0.00 0.04 0.08 0.05 0.16 0.17 \u22120.02 0.09 \u22120.02 0.06 0.03 0.14 0.36 0.06 0.05 0.01 0.08 0.14 0.06 0.10\n0.24 0.36 0.07 0.11 0.04 0.00 0.01 0.03 \u22120.02 0.05 \u22120.02 0.04 \u22120.01 0.03 0.12 0.02 0.01 \u22120.07 0.05 0.05 0.03 0.04 0.07 0.05\n0.03 0.03 \u22120.12 0.07 0.08 0.01 0.00 0.02 0.14 0.07 0.14 0.04 0.05 0.03 0.06 0.08 0.07 \u22120.03 0.36 0.10 0.04 0.05 0.04 0.07"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1937-6db4-11ee-9cbb-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-32", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "\u22120.00 \u22120.08 \u22120.07 0.09 0.05 0.03 0.02 0.00 0.02 0.07 \u22120.03 0.07 0.34 0.04 \u22120.04 0.04 0.04 0.02 0.05 0.06 0.06 0.06 0.02 0.07\n\u22120.01 \u22120.05 \u22120.35 0.03 0.16 \u22120.02 0.14 0.02 0.00 0.12 0.22 0.04 0.24 \u22120.02 \u22120.00 0.44 0.12 \u22120.04 0.10 0.30 0.01 0.23 0.05 0.11\n0.01 \u22120.03 \u22120.03 0.10 0.17 0.05 0.07 0.07 0.12 0.00 \u22120.00 0.09 0.09 0.07 0.01 0.12 0.26 0.06 0.06 0.10 0.07 0.12 0.07 0.18"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1938-6db4-11ee-a336-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-33", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "0.04 0.02 \u22120.46 0.01 \u22120.02 \u22120.02 0.14 \u22120.03 0.22 \u22120.00 0.00 0.01 0.04 \u22120.05 0.06 0.08 \u22120.04 \u22120.06 0.14 0.09 \u22120.00 0.06 0.03 0.02\n\u22120.00 \u22120.06 \u22120.02 0.10 0.09 0.04 0.04 0.07 0.04 0.09 0.01 0.00 0.04 0.07 \u22120.01 0.06 0.09 0.26 0.06 0.05 0.07 0.09 0.05 0.09\n\u22120.05 \u22120.12 \u22120.34 0.02 \u22120.02 \u22120.01 0.05 0.34 0.24 0.09 0.04 0.04 0.00 \u22120.03 \u22120.07 0.09 0.01 0.01 0.08 0.68 0.02 0.05 \u22120.07 0.10"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1939-6db4-11ee-8ced-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-34", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "\u22120.01 \u22120.05 0.11 0.09 0.06 0.03 0.03 0.04 \u22120.02 0.07 \u22120.05 0.07 \u22120.03 0.00 \u22120.01 0.03 0.03 0.03 0.05 0.01 0.06 0.06 0.04 0.07\n0.07 0.74 0.02 0.06 0.03 0.12 0.06 \u22120.04 \u22120.00 0.01 0.06 \u22120.01 \u22120.07 \u22120.01 0.00 0.00 \u22120.01 \u22120.10 0.04 \u22120.02 0.01 0.00 0.06 0.01\n\u22120.01 \u22120.04 \u22120.15 0.08 0.14 0.02 0.08 0.04 0.44 0.12 0.08 0.06 0.09 0.03 0.00 0.00 0.09 \u22120.00 0.07 0.11 0.05 0.22 \u22120.01 0.10"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a193a-6db4-11ee-b085-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-35", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "\u22120.00 \u22120.03 \u22120.14 0.07 0.36 0.01 0.07 0.04 0.12 0.26 \u22120.04 0.09 0.01 0.03 \u22120.01 0.09 0.00 0.05 0.02 0.03 0.05 0.10 0.03 0.27\n\u22120.12 \u22120.21 \u22120.01 0.07 0.06 \u22120.07 \u22120.03 0.02 \u22120.04 0.06 \u22120.06 0.26 0.01 0.03 \u22120.10 \u22120.00 0.05 0.00 0.02 0.00 0.22 0.03 \u22120.01 0.05\n0.04 0.01 \u22120.07 0.08 0.05 0.05 0.36 0.05 0.10 0.06 0.14 0.06 0.08 0.05 0.04 0.07 0.02 0.02 0.00 0.11 0.06 0.08 0.07 0.06"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a193b-6db4-11ee-9e70-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-36", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "0.01 \u22120.03 \u22120.21 0.06 0.01 0.05 0.10 0.06 0.30 0.10 0.09 0.05 0.68 0.01 \u22120.02 0.11 0.03 0.00 0.11 0.00 0.04 0.09 \u22120.00 0.12\n0.01 \u22120.03 0.03 0.09 0.08 0.03 0.04 0.06 0.01 0.07 \u22120.00 0.07 0.02 0.06 0.01 0.05 0.05 0.22 0.06 0.04 0.00 0.06 0.05 0.07\n0.02 \u22120.03 \u22120.08 0.09 0.14 0.04 0.05 0.06 0.23 0.12 0.06 0.09 0.05 0.06 0.00 0.22 0.10 0.03 0.08 0.09 0.06 0.00 0.06 0.10"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a193c-6db4-11ee-b899-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-37", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "0.04 0.05 0.06 0.08 0.06 0.07 0.04 0.02 0.05 0.07 0.03 0.05 \u22120.07 0.04 0.06 \u22120.01 0.03 \u22120.01 0.07 \u22120.00 0.05 0.06 0.00 0.07\n0.02 \u22120.03 \u22120.03 0.10 0.10 0.05 0.07 0.07 0.11 0.18 0.02 0.09 0.10 0.07 0.01 0.10 0.27 0.05 0.06 0.12 0.07 0.10 0.07 0.00female\npeople\nindoor\nbaby\nsea\nportrait\ntransport\nflower\nsky\nlake\nstructures\nbird\nplant life\nfood\nmale\nclouds\nwater\nanimals\ncar\ntree\ndog\nsunset\nnight\nriver\nfemalepeopleindoorbabyseaportraittransportflowerskylakestructuresbirdplant lifefoodmalecloudswateranimalscartreedogsunsetnightriver\n0 5000 100000246810x 105\nTime [s]Neg. Log\u2212Likelihood\n  \nw/o blend\nw blend\n0 5000 100000200040006000800010000\nTime [s]Training error"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a193d-6db4-11ee-9538-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-38", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "Time [s]Neg. Log\u2212Likelihood\n  \nw/o blend\nw blend\n0 5000 100000200040006000800010000\nTime [s]Training error\n  \nw/o blend\nw blend\n(a) (b)\nFigure 7: (a) Correlation matrix ( i.e., pairwise potentials) learned on the Flickr dataset. (b) Blending\nlearning and inference speeds-up training signi\ufb01cantly.\nDeep helps: We tested our models using one layer and two-layer perceptrons with both shortrange and long-range connections in the MRF. For the two-layer MLP, the number of hidden units\nin the \ufb01rst layer is \ufb01xed to H1= 512 , and we varied the number of hidden units H2in the second\nlayer. As shown in Table 1 we observe that the deeper and the more structured the model, the better\nthe performance we achieve. As expected, performance also grows with the number of hidden units.\nEf\ufb01ciency: Using GPUs, it takes on average 0.064s per iteration for the 1st order Markov model\nand 0.104s for the 2nd order Markov model. The time employed for training the one layer vs. the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a193e-6db4-11ee-b114-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-39", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "and 0.104s for the 2nd order Markov model. The time employed for training the one layer vs. the\nmulti-layer models is approximately the same. Note that our approach is very ef\ufb01cient, as this is the\ntime per iteration to train 831,166 weights.\nLearned parameters: As shown in the left column of Fig. 4, the learned unary weights resemble\ncharacter strokes. The middle two panels show the learned pairwise weights for distance-1 edges\n(i.e., edges with only neighboring connections) and distance-2 edges ( i.e., edges connecting every\nother variable). For example, it shows that \u2018q\u2019 is likely to be followed by \u2018u,\u2019 and \u2018e\u2019 is likely to\nbe distance-2 away from \u2018q\u2019 in this dataset. On the right-most panel, we also show the negative\nlog-likelihood as a function of the number of joint training iterations. PreTrainJoint can achieve the\nlowest cost value, while PwTrain has the highest value.\nNon-linear pairwise functions: To further demonstrate the generality of our approach, we replaced\nthe linear pairwise function in Eq. (7) by a one-layer MLP, while keeping the other settings identical."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a193f-6db4-11ee-9a51-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-40", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "the linear pairwise function in Eq. (7) by a one-layer MLP, while keeping the other settings identical.\nFor this experiment we utilize a 1st order Markov model. As shown in Fig. 5, our model attains best\nperformance when using a non-linear pairwise function. We found 16 to 64 hidden units for the\nnon-linear pairwise function to be suf\ufb01cient for modeling the bi-gram combinations in this dataset.\nIn this case the largest model has 974,846 weights and training takes on average 0.068s per iteration.\n3.2 I MAGE CLASSIFICATION : FLICKR\nWe next evaluate the importance of blending learning and inference. Towards this goal, we make\nuse of the Flickr dataset, which consists of 10;000training and 10;000test images from Flickr. The\ntask is to predict which of 38 possible tags should be assigned to each image. Fig. 6 shows some\nexamples. The graphical model has 38 binary random variables, each denoting the presence/absence\nof a particular tag. We de\ufb01ne the non-linear unaries fr(x;yi;wu)using the 8-layer deep-net architecture from Krizhevsky et al. (2013) followed by a 76-dimensional top layer. Hence the function"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1940-6db4-11ee-a4d1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-41", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "is composed out of two subsequent stacks of convolution, recti\ufb01ed linear (ReLU), pooling and local\nresponse normalization units. Those are followed by three convolution\u2013ReLU function pairs. Afterwards pooling is applied before two fully-connected\u2013ReLU\u2013dropout combinations are employed to\nyield the input into a fully connected layer which \ufb01nally computes the unary potentials. We employ\npairwise potentials similar to Eq. (7) which now fully model the correlations between any pair of\n8\nAccepted as a workshop contribution at ICLR 2015\noutput variables. This amounts to a total of 57;182;408parameters arising from the convolutional\nunits, fully connected units and corresponding biases as well as the pairwise weights.\nWe use a momentum based sub-gradient method for training with a mini-batch size of 300, a step\nsize of 0:0001 , a momentum of 0:95and set\u000f= 1 andcr= 18r. We initialized the deep-net\nparameters using a model pre-trained on ImageNet (Deng et al., 2009). Our error metric is the\nclassi\ufb01cation error ( i.e., Hamming loss).\nJoint training helps: The mean error for unary only potentials (\u2018Unary only\u2019), piecewise training"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1941-6db4-11ee-a69b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-42", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "Joint training helps: The mean error for unary only potentials (\u2018Unary only\u2019), piecewise training\n(\u2018PwTrain\u2019) and joint pretraining (\u2018PreTrainJoint\u2019) is 9.36%, 7.70% and 7.25% respectively. Similar\nto the Word50 dataset we observe that joint training is bene\ufb01cial. We provide examples for perfect\n(two left-most images), roughly accurate and failing predictions (right image) in Fig. 6.\nLearned pairwise weights: In Fig. 7(a) we illustrate the learned correlations for a subset of the 38\nclasses. We observe that the class \u2018people\u2019 correlates highly with \u2018female,\u2019 \u2018male,\u2019 and \u2018portrait.\u2019\nThe \u2018indoor\u2019 tag does not co-occur with \u2018sky,\u2019 \u2018structures,\u2019 \u2018plant life\u2019 and \u2018tree.\u2019 \u2018Sea\u2019 appears\ntypically with \u2018water,\u2019 \u2018clouds,\u2019 \u2018lake\u2019 and \u2018sky.\u2019\nEf\ufb01ciency of Blending: To illustrate that blending is indeed bene\ufb01cial we compare the negative"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1942-6db4-11ee-8759-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-43", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "Ef\ufb01ciency of Blending: To illustrate that blending is indeed bene\ufb01cial we compare the negative\nlog-likelihood and the training error as a function of run-time in Fig. 7(b). The standard approach\nis limited to 20 iterations of message passing to avoid time-consuming, repeated computation of a\nstopping criterion involving both the approximated log-partition function and its dual. As show in\nFig. 7(b) blending learning and inference speeds up parameter estimation signi\ufb01cantly. For larger\ngraphical models, we expect the differences to be even more signi\ufb01cant.\n4 D ISCUSSION & C ONCLUSION\nJoint training of neural networks and graphical models: Neural Networks have been incorporated as unary potentials in graphical models. One of the earliest works by Bridle (1990) jointly optimizes a system consisting of multilayer perceptrons and hidden Markov models for speech recognition. For document processing systems, Bottou et al. (1997) propose Graph Transformer Networks\nto jointly optimize sub-tasks, such as word segmentation and character recognition. Several works\n(Collobert et al., 2011; Peng et al., 2009; Ma et al., 2012; Do & Artieres, 2010; Prabhavalkar &"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1943-6db4-11ee-8a5a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-44", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "(Collobert et al., 2011; Peng et al., 2009; Ma et al., 2012; Do & Artieres, 2010; Prabhavalkar &\nFosler-Lussier, 2010; Morris & Fosler-Lussier, 2008) have extended the linear unary potential in\nMRFs to incorporate non-linearities. However, they assume that exact inference can be performed\neither via a forward-backward pass within the graphical model or dynamic programming. In contrast, in this paper we present learning algorithms for general graphical models, where inference is\nhard. Moreover, all the previous works (except Do & Artieres (2010)) do not consider max-margin\nloss during training which is incorporated into our framework by choosing \u000f= 0. More recently, Li\n& Zemel (2014) use a hinge loss to learn the unary term de\ufb01ned as a neural net, but keep the pairwise\npotentials \ufb01xed ( i.e., no joint training). Domke (2013) considers non-linear structured prediction and\ndecomposes the learning problem into a subset of logistic regressors, which require the parameter\nupdates to be run till convergence before updating the messages. Tompson et al. (2014) also jointly\ntrain convolutional neural networks and a graphical model for pose estimation. However, the MRF"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1944-6db4-11ee-b5d2-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-45", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "train convolutional neural networks and a graphical model for pose estimation. However, the MRF\ninference procedure is approximated by their Spatial-Model which ignores the partition function.\nBlending learning and inference: In this paper we de\ufb01ned learning to be a min-max task. The\nblending strategy, which was previously employed for learning log-linear models by (Meshi et al.,\n2010; Hazan & Urtasun, 2010), amounts to converting the maximization task into a minimization\nproblem using its dual. Subsequently we make use of block-coordinate descent strategies to obtain\na more ef\ufb01cient algorithm. Importantly any order of block-updates is possible. It remains an open\nproblem to \ufb01nd the optimal tradeoff.\nWe have proposed an ef\ufb01cient algorithm to learn deep models enriched to capture the dependencies\nbetween the output variables. Our experiments on word prediction from noisy images and multiclass image classi\ufb01cation showed that the deeper and the more structured the model, the better the\nperformance we achieve. Furthermore, joint learning of all weights outperforms all other strategies. In the future we plan to learn deeper models in applications such as holistic semantic scene\nunderstanding. We will also extend our approach to deal with hidden variables.\n9\nAccepted as a workshop contribution at ICLR 2015\nACKNOWLEDGMENTS"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1945-6db4-11ee-9ad1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-46", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "understanding. We will also extend our approach to deal with hidden variables.\n9\nAccepted as a workshop contribution at ICLR 2015\nACKNOWLEDGMENTS\nWe thank NVIDIA Corporation for the donation of GPUs used in this research. This work was\npartially funded by ONR-N00014-14-1-0232.\nREFERENCES\nBengio, Y ., Lamblin, P., Popovici, D., and Larochelle, H. Greedy Layer-Wise Training of Deep Networks. In\nProc. NIPS , 2007.\nBottou, L., Bengio, Y ., and LeCun, Y . Global training of document processing systems using graph transformer\nnetworks. In Proc. CVPR , 1997.\nBridle, J. S. Training stochastic model recognition algorithms as networks can lead to maximum mutual information estimation of parameters. In Proc. NIPS , 1990.\nCollobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., and Kuksa, P. Natural language processing\n(almost) from scratch. JMLR , 2011.\nde Campos, T. E., Babu, B. R., and Varma, M. Character recognition in natural images. In Proc. VISAPP , 2009."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1946-6db4-11ee-b8ff-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-47", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "de Campos, T. E., Babu, B. R., and Varma, M. Character recognition in natural images. In Proc. VISAPP , 2009.\nDeng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image\ndatabase. In Proc. CVPR , 2009.\nDeng, J., Ding, N., Jia, Y ., Frome, A., Murphy, K., Bengio, S., Li, Y ., Neven, H., and Adam, H. Large-Scale\nObject Classi\ufb01cation using Label Relation Graphs. In Proc. ECCV , 2014.\nDo, T.-M.-T. and Artieres, T. Neural conditional random \ufb01elds. In Proc. AISTATS , 2010.\nDomke, J. Structured Learning via Logistic Regression. In Proc. NIPS , 2013.\nEigen, D., Rolfe, J., Fergus, R., and LeCun, Y . Understanding Deep Architectures using a Recursive Convolutional Network. In Proc. ICLR , 2014.\nGirshick, R., Donahue, J., Darrell, T., and Malik, J. Rich feature hierarchies for accurate object detection and"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1947-6db4-11ee-abe5-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-48", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "Girshick, R., Donahue, J., Darrell, T., and Malik, J. Rich feature hierarchies for accurate object detection and\nsemantic segmentation. In Proc. CVPR , 2014.\nHariharan, B., Arbel \u00b4aez, P., Girshick, R., and Malik, J. Simultaneous detection and segmentation. In Proc.\nECCV , 2014.\nHazan, T. and Urtasun, R. A Primal-Dual Message-Passing Algorithm for Approximated Large Scale Structured Prediction. In Proc. NIPS , 2010.\nHinton, G. E. and Salakhutdinov, R. R. Reducing the dimensionality of data with neural networks. Science ,\n2006.\nHinton, G. E., Sejnowski, T. J., and Ackley, D. H. Boltzmann Machines: Constraint Satisfaction Networks that\nLearn. Technical report, University of Toronto, 1984.\nJia, Y . Caffe: An Open Source Convolutional Architecture for Fast Feature Embedding. http://caffe.\nberkeleyvision.org/ , 2013.\nKoller, D. and Friedman, N. Probabilistic Graphical Models: Principles and Techniques . MIT Press, 2009."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1948-6db4-11ee-9f0f-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-49", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "berkeleyvision.org/ , 2013.\nKoller, D. and Friedman, N. Probabilistic Graphical Models: Principles and Techniques . MIT Press, 2009.\nKrizhevsky, A., Sutskever, I., and Hinton, G. E. ImageNet Classi\ufb01cation with Deep Convolutional Neural\nNetworks. In Proc. NIPS , 2013.\nLarochelle, H., Erhan, D., Courville, A., Bergstra, J., and Bengio, Y . An empirical evaluation of deep architectures on problems with many factors of variation. In Proc. ICML , 2007.\nLeCun, Y ., Bottou, L., Bengio, Y ., and Haffner, P. Gradient-based learning applied to document recognition.\nProceedings of the IEEE , 1998.\nLee, H., Grosse, R., Ranganath, R., and Ng, A. Y . Convolutional deep belief networks for scalable unsupervised\nlearning of hierarchical representations. In Proc. ICML , 2009.\nLi, Y . and Zemel, R. High Order Regularization for Semi-Supervised Learning of Structured Output Problems.\nInProc. ICML , 2014.\n10\nAccepted as a workshop contribution at ICLR 2015"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1949-6db4-11ee-8c10-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-50", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "InProc. ICML , 2014.\n10\nAccepted as a workshop contribution at ICLR 2015\nMa, J., Peng, J., Wang, S., and Xu, J. A conditional neural \ufb01elds model for protein threading. Bioinformatics ,\n2012.\nMeltzer, T., Globerson, A., and Weiss, Y . Convergent Message Passing Algorithms: a unifying view. In Proc.\nUAI, 2009.\nMeshi, O., Sontag, D., Jaakkola, T., and Globerson, A. Learning Ef\ufb01ciently with Approximate inference via\nDual Losses. In Proc. ICML , 2010.\nMorris, J. and Fosler-Lussier, E. Conditional random \ufb01elds for integrating local discriminative classi\ufb01ers. IEEE\nTrans. Audio, Speech, and Language Processing , 2008.\nNowozin, S., Rother, C., Bagon, S., Sharp, T., Yao, B., and Kohli, P. Decision tree \ufb01elds. In Proc. ICCV , 2011.\nPearl, J. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference . Morgan Kaufmann,\n1988."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a194a-6db4-11ee-9100-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-51", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "Pearl, J. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference . Morgan Kaufmann,\n1988.\nPeng, J., Bo, L., and Xu, J. Conditional Neural Fields. In Proc. NIPS , 2009.\nPrabhavalkar, R. and Fosler-Lussier, E. Backpropagation training for multilayer conditional random \ufb01eld based\nphone recognition. In Proc. ICASSP , 2010.\nSalakhutdinov, R. R. and Hinton, G. E. An Ef\ufb01cient Learning Procedure for Deep Boltzmann Machines. Neural\nComputation , 2012.\nSchwing, A. G. Inference and Learning Algorithms with Applications to 3D Indoor Scene Understanding . PhD\nthesis, ETH Zurich, 2013.\nSocher, R., Huval, B., Bhat, B., Manning, C. D., and Ng, A. Y . Convolutional-Recursive Deep Learning for 3D\nObject Classi\ufb01cation. In Proc. NIPS , 2012.\nTompson, J., Jain, A., LeCun, Y ., and Bregler, C. Joint Training of a Convolutional Network and a Graphical\nModel for Human Pose Estimation. In Proc. NIPS , 2014."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a194b-6db4-11ee-80dd-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1407.2538-52", "title": "Learning Deep Structured Models", "source": "http://arxiv.org/pdf/1407.2538", "start_index": 0, "chroma:document": "Model for Human Pose Estimation. In Proc. NIPS , 2014.\nWainwright, M. J. and Jordan, M. I. Graphical Models, Exponential Families and Variational Inference . Foundations and Trends in Machine Learning, 2008.\nWainwright, M. J., Jaakkola, T., and Willsky, A. S. A new class of upper bounds on the log partition function.\nTrans. Information Theory , 2005.\nWeiss, Y ., Yanover, C., and Meltzer, T. MAP Estimation, Linear Programming and Belief Propagation with\nConvex Free Energies. In Proc. UAI , 2007.\nWiegerinck, W. and Heskes, T. Fractional belief propagation. In Proc. NIPS , 2003.\nXu, J., Schwing, A. G., and Urtasun, R. Tell me what you see and I will show you where it is. In Proc. CVPR ,\n2014.\nYedidia, J. S., Freeman, W. T., and Weiss, Y . Constructing free-energy approximations and generalized belief\npropagation algorithms. Trans. Information Theory , 2005.\nZeiler, M. D. and Fergus, R. Visualizing and Understanding Convolutional Networks. In Proc. ECCV , 2014.\n11"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a194c-6db4-11ee-9bf4-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-0", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "Published as a conference paper at ICLR 2015\nNEURAL MACHINE TRANSLATION\nBYJOINTLY LEARNING TO ALIGN AND TRANSLATE\nDzmitry Bahdanau\nJacobs University Bremen, Germany\nKyungHyun Cho Yoshua Bengio\u0003\nUniversit \u00b4e de Montr \u00b4eal\nABSTRACT\nNeural machine translation is a recently proposed approach to machine translation. Unlike the traditional statistical machine translation, the neural machine\ntranslation aims at building a single neural network that can be jointly tuned to\nmaximize the translation performance. The models proposed recently for neural machine translation often belong to a family of encoder\u2013decoders and encode\na source sentence into a \ufb01xed-length vector from which a decoder generates a\ntranslation. In this paper, we conjecture that the use of a \ufb01xed-length vector is a\nbottleneck in improving the performance of this basic encoder\u2013decoder architecture, and propose to extend this by allowing a model to automatically (soft-)search\nfor parts of a source sentence that are relevant to predicting a target word, without\nhaving to form these parts as a hard segment explicitly. With this new approach,\nwe achieve a translation performance comparable to the existing state-of-the-art\nphrase-based system on the task of English-to-French translation. Furthermore,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a194d-6db4-11ee-b4a6-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-1", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "we achieve a translation performance comparable to the existing state-of-the-art\nphrase-based system on the task of English-to-French translation. Furthermore,\nqualitative analysis reveals that the (soft-)alignments found by the model agree\nwell with our intuition.\n1 I NTRODUCTION\nNeural machine translation is a newly emerging approach to machine translation, recently proposed\nby Kalchbrenner and Blunsom (2013), Sutskever et al. (2014) and Cho et al. (2014b). Unlike the\ntraditional phrase-based translation system (see, e.g., Koehn et al. , 2003) which consists of many\nsmall sub-components that are tuned separately, neural machine translation attempts to build and\ntrain a single, large neural network that reads a sentence and outputs a correct translation.\nMost of the proposed neural machine translation models belong to a family of encoder\u2013\ndecoders (Sutskever et al. , 2014; Cho et al. , 2014a), with an encoder and a decoder for each language, or involve a language-speci\ufb01c encoder applied to each sentence whose outputs are then compared (Hermann and Blunsom, 2014). An encoder neural network reads and encodes a source sentence into a \ufb01xed-length vector. A decoder then outputs a translation from the encoded vector. The"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a194e-6db4-11ee-9a84-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-2", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "whole encoder\u2013decoder system, which consists of the encoder and the decoder for a language pair,\nis jointly trained to maximize the probability of a correct translation given a source sentence.\nA potential issue with this encoder\u2013decoder approach is that a neural network needs to be able to\ncompress all the necessary information of a source sentence into a \ufb01xed-length vector. This may\nmake it dif\ufb01cult for the neural network to cope with long sentences, especially those that are longer\nthan the sentences in the training corpus. Cho et al. (2014b) showed that indeed the performance of\na basic encoder\u2013decoder deteriorates rapidly as the length of an input sentence increases.\nIn order to address this issue, we introduce an extension to the encoder\u2013decoder model which learns\nto align and translate jointly. Each time the proposed model generates a word in a translation, it\n(soft-)searches for a set of positions in a source sentence where the most relevant information is\nconcentrated. The model then predicts a target word based on the context vectors associated with\nthese source positions and all the previous generated target words.\n\u0003CIFAR Senior Fellow\n1arXiv:1409.0473v7  [cs.CL]  19 May 2016\nPublished as a conference paper at ICLR 2015"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a194f-6db4-11ee-8617-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-3", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "\u0003CIFAR Senior Fellow\n1arXiv:1409.0473v7  [cs.CL]  19 May 2016\nPublished as a conference paper at ICLR 2015\nThe most important distinguishing feature of this approach from the basic encoder\u2013decoder is that\nit does not attempt to encode a whole input sentence into a single \ufb01xed-length vector. Instead, it encodes the input sentence into a sequence of vectors and chooses a subset of these vectors adaptively\nwhile decoding the translation. This frees a neural translation model from having to squash all the\ninformation of a source sentence, regardless of its length, into a \ufb01xed-length vector. We show this\nallows a model to cope better with long sentences.\nIn this paper, we show that the proposed approach of jointly learning to align and translate achieves\nsigni\ufb01cantly improved translation performance over the basic encoder\u2013decoder approach. The improvement is more apparent with longer sentences, but can be observed with sentences of any\nlength. On the task of English-to-French translation, the proposed approach achieves, with a single\nmodel, a translation performance comparable, or close, to the conventional phrase-based system.\nFurthermore, qualitative analysis reveals that the proposed model \ufb01nds a linguistically plausible\n(soft-)alignment between a source sentence and the corresponding target sentence."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1950-6db4-11ee-bac7-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-4", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "Furthermore, qualitative analysis reveals that the proposed model \ufb01nds a linguistically plausible\n(soft-)alignment between a source sentence and the corresponding target sentence.\n2 B ACKGROUND : NEURAL MACHINE TRANSLATION\nFrom a probabilistic perspective, translation is equivalent to \ufb01nding a target sentence ythat maximizes the conditional probability of ygiven a source sentence x, i.e., arg maxyp(yjx). In\nneural machine translation, we \ufb01t a parameterized model to maximize the conditional probability\nof sentence pairs using a parallel training corpus. Once the conditional distribution is learned by a\ntranslation model, given a source sentence a corresponding translation can be generated by searching\nfor the sentence that maximizes the conditional probability.\nRecently, a number of papers have proposed the use of neural networks to directly learn this conditional distribution (see, e.g., Kalchbrenner and Blunsom, 2013; Cho et al. , 2014a; Sutskever et al. ,\n2014; Cho et al. , 2014b; Forcada and \u02dcNeco, 1997). This neural machine translation approach typically consists of two components, the \ufb01rst of which encodes a source sentence xand the second\ndecodes to a target sentence y. For instance, two recurrent neural networks (RNN) were used by"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1951-6db4-11ee-83d1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-5", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "decodes to a target sentence y. For instance, two recurrent neural networks (RNN) were used by\n(Cho et al. , 2014a) and (Sutskever et al. , 2014) to encode a variable-length source sentence into a\n\ufb01xed-length vector and to decode the vector into a variable-length target sentence.\nDespite being a quite new approach, neural machine translation has already shown promising results.\nSutskever et al. (2014) reported that the neural machine translation based on RNNs with long shortterm memory (LSTM) units achieves close to the state-of-the-art performance of the conventional\nphrase-based machine translation system on an English-to-French translation task.1Adding neural\ncomponents to existing translation systems, for instance, to score the phrase pairs in the phrase\ntable (Cho et al. , 2014a) or to re-rank candidate translations (Sutskever et al. , 2014), has allowed to\nsurpass the previous state-of-the-art performance level.\n2.1 RNN E NCODER \u2013DECODER\nHere, we describe brie\ufb02y the underlying framework, called RNN Encoder\u2013Decoder , proposed by\nCho et al. (2014a) and Sutskever et al. (2014) upon which we build a novel architecture that learns"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1952-6db4-11ee-a309-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-6", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "Cho et al. (2014a) and Sutskever et al. (2014) upon which we build a novel architecture that learns\nto align and translate simultaneously.\nIn the Encoder\u2013Decoder framework, an encoder reads the input sentence, a sequence of vectors\nx= (x1;\u0001\u0001\u0001;xTx), into a vector c.2The most common approach is to use an RNN such that\nht=f(xt;ht\u00001) (1)\nand\nc=q(fh1;\u0001\u0001\u0001;hTxg);\nwhereht2Rnis a hidden state at time t, andcis a vector generated from the sequence of the\nhidden states. fandqare some nonlinear functions. Sutskever et al. (2014) used an LSTM as fand\nq(fh1;\u0001\u0001\u0001;hTg) =hT, for instance.\n1We mean by the state-of-the-art performance, the performance of the conventional phrase-based system\nwithout using any neural network-based component.\n2Although most of the previous works (see, e.g., Cho et al. , 2014a; Sutskever et al. , 2014; Kalchbrenner and"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a1953-6db4-11ee-9b24-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-7", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "2Although most of the previous works (see, e.g., Cho et al. , 2014a; Sutskever et al. , 2014; Kalchbrenner and\nBlunsom, 2013) used to encode a variable-length input sentence into a \ufb01xed-length vector, it is not necessary,\nand even it may be bene\ufb01cial to have a variable-length vector, as we will show later.\n2\nPublished as a conference paper at ICLR 2015\nThe decoder is often trained to predict the next word yt0given the context vector cand all the\npreviously predicted words fy1;\u0001\u0001\u0001;yt0\u00001g. In other words, the decoder de\ufb01nes a probability over\nthe translation yby decomposing the joint probability into the ordered conditionals:\np(y) =TY\nt=1p(ytjfy1;\u0001\u0001\u0001;yt\u00001g;c); (2)\nwhere y=\u0000\ny1;\u0001\u0001\u0001;yTy\u0001\n. With an RNN, each conditional probability is modeled as\np(ytjfy1;\u0001\u0001\u0001;yt\u00001g;c) =g(yt\u00001;st;c); (3)\nwheregis a nonlinear, potentially multi-layered, function that outputs the probability of yt, andstis"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f71-6db4-11ee-bc5c-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-8", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "wheregis a nonlinear, potentially multi-layered, function that outputs the probability of yt, andstis\nthe hidden state of the RNN. It should be noted that other architectures such as a hybrid of an RNN\nand a de-convolutional neural network can be used (Kalchbrenner and Blunsom, 2013).\n3 L EARNING TO ALIGN AND TRANSLATE\nIn this section, we propose a novel architecture for neural machine translation. The new architecture\nconsists of a bidirectional RNN as an encoder (Sec. 3.2) and a decoder that emulates searching\nthrough a source sentence during decoding a translation (Sec. 3.1).\n3.1 D ECODER : GENERAL DESCRIPTION\nx1x2x3xT+\n\u03b1t,1\n\u03b1t,2 \u03b1t,3\u03b1t,Tyt-1yt\nh1h2h3 hTh1h2h3 hTst-1st\nFigure 1: The graphical illustration of the proposed model\ntrying to generate the t-th target wordytgiven a source\nsentence (x1;x2;:::;x T).In a new model architecture, we de\ufb01ne each conditional probability\nin Eq. (2) as:"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f72-6db4-11ee-bf5f-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-9", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "sentence (x1;x2;:::;x T).In a new model architecture, we de\ufb01ne each conditional probability\nin Eq. (2) as:\np(yijy1;:::;y i\u00001;x) =g(yi\u00001;si;ci); (4)\nwheresiis an RNN hidden state for time i, computed by\nsi=f(si\u00001;yi\u00001;ci):\nIt should be noted that unlike the existing encoder\u2013decoder approach (see Eq. (2)), here the probability is conditioned on a distinct\ncontext vector cifor each target word yi.\nThe context vector cidepends on a sequence of annotations\n(h1;\u0001\u0001\u0001;hTx)to which an encoder maps the input sentence. Each\nannotationhicontains information about the whole input sequence\nwith a strong focus on the parts surrounding the i-th word of the\ninput sequence. We explain in detail how the annotations are computed in the next section.\nThe context vector ciis, then, computed as a weighted sum of these\nannotationshi:\nci=TxX\nj=1\u000bijhj: (5)\nThe weight\u000bijof each annotation hjis computed by\n\u000bij=exp (eij)PTx"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f73-6db4-11ee-a437-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-10", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "ci=TxX\nj=1\u000bijhj: (5)\nThe weight\u000bijof each annotation hjis computed by\n\u000bij=exp (eij)PTx\nk=1exp (eik); (6)\nwhere\neij=a(si\u00001;hj)\nis an alignment model which scores how well the inputs around position jand the output at position\nimatch. The score is based on the RNN hidden state si\u00001(just before emitting yi, Eq. (4)) and the\nj-th annotation hjof the input sentence.\nWe parametrize the alignment model aas a feedforward neural network which is jointly trained with\nall the other components of the proposed system. Note that unlike in traditional machine translation,\n3\nPublished as a conference paper at ICLR 2015\nthe alignment is not considered to be a latent variable. Instead, the alignment model directly computes a soft alignment, which allows the gradient of the cost function to be backpropagated through.\nThis gradient can be used to train the alignment model as well as the whole translation model jointly.\nWe can understand the approach of taking a weighted sum of all the annotations as computing an\nexpected annotation , where the expectation is over possible alignments. Let \u000bijbe a probability that"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f74-6db4-11ee-b111-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-11", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "We can understand the approach of taking a weighted sum of all the annotations as computing an\nexpected annotation , where the expectation is over possible alignments. Let \u000bijbe a probability that\nthe target word yiis aligned to, or translated from, a source word xj. Then, thei-th context vector\nciis the expected annotation over all the annotations with probabilities \u000bij.\nThe probability \u000bij, or its associated energy eij, re\ufb02ects the importance of the annotation hjwith\nrespect to the previous hidden state si\u00001in deciding the next state siand generating yi. Intuitively,\nthis implements a mechanism of attention in the decoder. The decoder decides parts of the source\nsentence to pay attention to. By letting the decoder have an attention mechanism, we relieve the\nencoder from the burden of having to encode all information in the source sentence into a \ufb01xedlength vector. With this new approach the information can be spread throughout the sequence of\nannotations, which can be selectively retrieved by the decoder accordingly.\n3.2 E NCODER : BIDIRECTIONAL RNN FOR ANNOTATING SEQUENCES\nThe usual RNN, described in Eq. (1), reads an input sequence xin order starting from the \ufb01rst"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f75-6db4-11ee-b796-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-12", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "The usual RNN, described in Eq. (1), reads an input sequence xin order starting from the \ufb01rst\nsymbolx1to the last one xTx. However, in the proposed scheme, we would like the annotation\nof each word to summarize not only the preceding words, but also the following words. Hence,\nwe propose to use a bidirectional RNN (BiRNN, Schuster and Paliwal, 1997), which has been\nsuccessfully used recently in speech recognition (see, e.g., Graves et al. , 2013).\nA BiRNN consists of forward and backward RNN\u2019s. The forward RNN\u0000 !freads the input sequence\nas it is ordered (from x1toxTx) and calculates a sequence of forward hidden states (\u0000 !h1;\u0001\u0001\u0001;\u0000 !hTx).\nThe backward RNN \u0000freads the sequence in the reverse order (from xTxtox1), resulting in a\nsequence of backward hidden states ( \u0000h1;\u0001\u0001\u0001; \u0000hTx).\nWe obtain an annotation for each word xjby concatenating the forward hidden state\u0000 !hjand the\nbackward one \u0000hj, i.e.,hj=h\u0000 !h>\nj; \u0000h>\nji>"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f76-6db4-11ee-b808-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-13", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "backward one \u0000hj, i.e.,hj=h\u0000 !h>\nj; \u0000h>\nji>\n. In this way, the annotation hjcontains the summaries\nof both the preceding words and the following words. Due to the tendency of RNNs to better\nrepresent recent inputs, the annotation hjwill be focused on the words around xj. This sequence\nof annotations is used by the decoder and the alignment model later to compute the context vector\n(Eqs. (5)\u2013(6)).\nSee Fig. 1 for the graphical illustration of the proposed model.\n4 E XPERIMENT SETTINGS\nWe evaluate the proposed approach on the task of English-to-French translation. We use the bilingual, parallel corpora provided by ACL WMT \u201914.3As a comparison, we also report the performance of an RNN Encoder\u2013Decoder which was proposed recently by Cho et al. (2014a). We use\nthe same training procedures and the same dataset for both models.4\n4.1 D ATASET\nWMT \u201914 contains the following English-French parallel corpora: Europarl (61M words), news\ncommentary (5.5M), UN (421M) and two crawled corpora of 90M and 272.5M words respectively,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f77-6db4-11ee-b3eb-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-14", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "commentary (5.5M), UN (421M) and two crawled corpora of 90M and 272.5M words respectively,\ntotaling 850M words. Following the procedure described in Cho et al. (2014a), we reduce the size of\nthe combined corpus to have 348M words using the data selection method by Axelrod et al. (2011).5\nWe do not use any monolingual data other than the mentioned parallel corpora, although it may be\npossible to use a much larger monolingual corpus to pretrain an encoder. We concatenate news-test3http://www.statmt.org/wmt14/translation-task.html\n4Implementations are available at https://github.com/lisa-groundhog/GroundHog .\n5Available online at http://www-lium.univ-lemans.fr/ \u02dcschwenk/cslm_joint_paper/ .\n4\nPublished as a conference paper at ICLR 2015\n0 10 20 30 40 50 60\nSentence length051015202530BLEU score RNNsearch-50\nRNNsearch-30\nRNNenc-50\nRNNenc-30\nFigure 2: The BLEU scores\nof the generated translations\non the test set with respect\nto the lengths of the sentences. The results are on"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f78-6db4-11ee-9e54-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-15", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "RNNenc-30\nFigure 2: The BLEU scores\nof the generated translations\non the test set with respect\nto the lengths of the sentences. The results are on\nthe full test set which includes sentences having unknown words to the models.\n2012 and news-test-2013 to make a development (validation) set, and evaluate the models on the test\nset (news-test-2014) from WMT \u201914, which consists of 3003 sentences not present in the training\ndata.\nAfter a usual tokenization6, we use a shortlist of 30,000 most frequent words in each language to\ntrain our models. Any word not included in the shortlist is mapped to a special token ( [UNK ]). We\ndo not apply any other special preprocessing, such as lowercasing or stemming, to the data.\n4.2 M ODELS\nWe train two types of models. The \ufb01rst one is an RNN Encoder\u2013Decoder (RNNencdec, Cho et al. ,\n2014a), and the other is the proposed model, to which we refer as RNNsearch. We train each model\ntwice: \ufb01rst with the sentences of length up to 30 words (RNNencdec-30, RNNsearch-30) and then"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f79-6db4-11ee-a54d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-16", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "twice: \ufb01rst with the sentences of length up to 30 words (RNNencdec-30, RNNsearch-30) and then\nwith the sentences of length up to 50 word (RNNencdec-50, RNNsearch-50).\nThe encoder and decoder of the RNNencdec have 1000 hidden units each.7The encoder of the\nRNNsearch consists of forward and backward recurrent neural networks (RNN) each having 1000\nhidden units. Its decoder has 1000 hidden units. In both cases, we use a multilayer network with a\nsingle maxout (Goodfellow et al. , 2013) hidden layer to compute the conditional probability of each\ntarget word (Pascanu et al. , 2014).\nWe use a minibatch stochastic gradient descent (SGD) algorithm together with Adadelta (Zeiler,\n2012) to train each model. Each SGD update direction is computed using a minibatch of 80 sentences. We trained each model for approximately 5 days.\nOnce a model is trained, we use a beam search to \ufb01nd a translation that approximately maximizes the\nconditional probability (see, e.g., Graves, 2012; Boulanger-Lewandowski et al. , 2013). Sutskever\net al. (2014) used this approach to generate translations from their neural machine translation model."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f7a-6db4-11ee-86fc-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-17", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "et al. (2014) used this approach to generate translations from their neural machine translation model.\nFor more details on the architectures of the models and training procedure used in the experiments,\nsee Appendices A and B.\n5 R ESULTS\n5.1 Q UANTITATIVE RESULTS\nIn Table 1, we list the translation performances measured in BLEU score. It is clear from the table\nthat in all the cases, the proposed RNNsearch outperforms the conventional RNNencdec. More\nimportantly, the performance of the RNNsearch is as high as that of the conventional phrase-based\ntranslation system (Moses), when only the sentences consisting of known words are considered.\nThis is a signi\ufb01cant achievement, considering that Moses uses a separate monolingual corpus (418M\nwords) in addition to the parallel corpora we used to train the RNNsearch and RNNencdec.\n6We used the tokenization script from the open-source machine translation package, Moses.\n7In this paper, by a \u2019hidden unit\u2019, we always mean the gated hidden unit (see Appendix A.1.1).\n5\nPublished as a conference paper at ICLR 2015\nThe\nagreement\non\nthe\nEuropean\nEconomic\nArea\nwas\nsigned\nin\nAugust\n1992\n.\n<end>\nL'\naccord"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f7b-6db4-11ee-9df7-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-18", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "The\nagreement\non\nthe\nEuropean\nEconomic\nArea\nwas\nsigned\nin\nAugust\n1992\n.\n<end>\nL'\naccord\nsur\nla\nzone\n\u00e9conomique\neurop\u00e9enne\na\n\u00e9t\u00e9\nsign\u00e9\nen\nao\u00fbt\n1992\n.\n<end>\nIt\nshould\nbe\nnoted\nthat\nthe\nmarine\nenvironment\nis\nthe\nleast\nknown\nof\nenvironments\n.\n<end>\nIl\nconvient\nde\nnoter\nque\nl'\nenvironnement\nmarin\nest\nle\nmoins\nconnu\nde\nl'\nenvironnement\n.\n<end>\n(a) (b)\nDestruction\nof\nthe\nequipment\nmeans\nthat\nSyria\ncan\nno\nlonger\nproduce\nnew\nchemical\nweapons\n.\n<end>\nLa\ndestruction\nde\nl'\n\u00e9quipement\nsignifie\nque\nla\nSyrie\nne\npeut\nplus\nproduire\nde\nnouvelles\narmes\nchimiques\n.\n<end>\n\"\nThis\nwill\nchange\nmy\nfuture\nwith\nmy\nfamily\n,\n\"\nthe\nman\nsaid\n.\n<end>\n\"\nCela"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f7c-6db4-11ee-8df5-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-19", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "\"\nThis\nwill\nchange\nmy\nfuture\nwith\nmy\nfamily\n,\n\"\nthe\nman\nsaid\n.\n<end>\n\"\nCela\nva\nchanger\nmon\navenir\navec\nma\nfamille\n\"\n,\na\ndit\nl'\nhomme\n.\n<end>\n(c) (d)\nFigure 3: Four sample alignments found by RNNsearch-50. The x-axis and y-axis of each plot\ncorrespond to the words in the source sentence (English) and the generated translation (French),\nrespectively. Each pixel shows the weight \u000bijof the annotation of the j-th source word for the i-th\ntarget word (see Eq. (6)), in grayscale ( 0: black, 1: white). (a) an arbitrary sentence. (b\u2013d) three\nrandomly selected samples among the sentences without any unknown words and of length between\n10 and 20 words from the test set.\nOne of the motivations behind the proposed approach was the use of a \ufb01xed-length context vector\nin the basic encoder\u2013decoder approach. We conjectured that this limitation may make the basic"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f7d-6db4-11ee-a180-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-20", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "in the basic encoder\u2013decoder approach. We conjectured that this limitation may make the basic\nencoder\u2013decoder approach to underperform with long sentences. In Fig. 2, we see that the performance of RNNencdec dramatically drops as the length of the sentences increases. On the other hand,\nboth RNNsearch-30 and RNNsearch-50 are more robust to the length of the sentences. RNNsearch50, especially, shows no performance deterioration even with sentences of length 50 or more. This\nsuperiority of the proposed model over the basic encoder\u2013decoder is further con\ufb01rmed by the fact\nthat the RNNsearch-30 even outperforms RNNencdec-50 (see Table 1).\n6\nPublished as a conference paper at ICLR 2015\nModel All No UNK\u000e\nRNNencdec-30 13.93 24.19\nRNNsearch-30 21.50 31.44\nRNNencdec-50 17.82 26.71\nRNNsearch-50 26.75 34.16\nRNNsearch-50?28.45 36.15\nMoses 33.30 35.63Table 1: BLEU scores of the trained models computed on the test set. The second and third columns\nshow respectively the scores on all the sentences and,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f7e-6db4-11ee-bfac-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-21", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "show respectively the scores on all the sentences and,\non the sentences without any unknown word in themselves and in the reference translations. Note that\nRNNsearch-50?was trained much longer until the\nperformance on the development set stopped improving. (\u000e) We disallowed the models to generate [UNK]\ntokens when only the sentences having no unknown\nwords were evaluated (last column).\n5.2 Q UALITATIVE ANALYSIS\n5.2.1 A LIGNMENT\nThe proposed approach provides an intuitive way to inspect the (soft-)alignment between the words\nin a generated translation and those in a source sentence. This is done by visualizing the annotation\nweights\u000bijfrom Eq. (6), as in Fig. 3. Each row of a matrix in each plot indicates the weights\nassociated with the annotations. From this we see which positions in the source sentence were\nconsidered more important when generating the target word.\nWe can see from the alignments in Fig. 3 that the alignment of words between English and French\nis largely monotonic. We see strong weights along the diagonal of each matrix. However, we also\nobserve a number of non-trivial, non-monotonic alignments. Adjectives and nouns are typically\nordered differently between French and English, and we see an example in Fig. 3 (a). From this"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f7f-6db4-11ee-b250-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-22", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "ordered differently between French and English, and we see an example in Fig. 3 (a). From this\n\ufb01gure, we see that the model correctly translates a phrase [European Economic Area] into [zone\n\u00b4economique europ \u00b4een]. The RNNsearch was able to correctly align [zone] with [Area], jumping\nover the two words ([European] and [Economic]), and then looked one word back at a time to\ncomplete the whole phrase [zone \u00b4economique europ \u00b4eenne].\nThe strength of the soft-alignment, opposed to a hard-alignment, is evident, for instance, from\nFig. 3 (d). Consider the source phrase [the man] which was translated into [l\u2019 homme]. Any hard\nalignment will map [the] to [l\u2019] and [man] to [homme]. This is not helpful for translation, as one\nmust consider the word following [the] to determine whether it should be translated into [le], [la],\n[les] or [l\u2019]. Our soft-alignment solves this issue naturally by letting the model look at both [the] and\n[man], and in this example, we see that the model was able to correctly translate [the] into [l\u2019]. We"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f80-6db4-11ee-8dd5-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-23", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "[man], and in this example, we see that the model was able to correctly translate [the] into [l\u2019]. We\nobserve similar behaviors in all the presented cases in Fig. 3. An additional bene\ufb01t of the soft alignment is that it naturally deals with source and target phrases of different lengths, without requiring a\ncounter-intuitive way of mapping some words to or from nowhere ([NULL]) (see, e.g., Chapters 4\nand 5 of Koehn, 2010).\n5.2.2 L ONG SENTENCES\nAs clearly visible from Fig. 2 the proposed model (RNNsearch) is much better than the conventional\nmodel (RNNencdec) at translating long sentences. This is likely due to the fact that the RNNsearch\ndoes not require encoding a long sentence into a \ufb01xed-length vector perfectly, but only accurately\nencoding the parts of the input sentence that surround a particular word.\nAs an example, consider this source sentence from the test set:\nAn admitting privilege is the right of a doctor to admit a patient to a hospital or\na medical centre tocarry outadiagnosis oraprocedure, based onhisstatus asa\nhealth care worker atahospital.\nThe RNNencdec-50 translated this sentence into:"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f81-6db4-11ee-85ca-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-24", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "health care worker atahospital.\nThe RNNencdec-50 translated this sentence into:\nUn privil `ege d\u2019admission est le droit d\u2019un m \u00b4edecin de reconna \u02c6\u0131tre un patient `a\nl\u2019h\u02c6opital ou un centre m \u00b4edical d\u2019un diagnostic oudeprendre undiagnostic en\nfonction deson\u00b4etatdesant\u00b4e.\n7\nPublished as a conference paper at ICLR 2015\nThe RNNencdec-50 correctly translated the source sentence until [a medical center]. However, from\nthere on (underlined), it deviated from the original meaning of the source sentence. For instance, it\nreplaced [based on his status as a health care worker at a hospital] in the source sentence with [en\nfonction de son \u00b4etat de sant \u00b4e] (\u201cbased on his state of health\u201d).\nOn the other hand, the RNNsearch-50 generated the following correct translation, preserving the\nwhole meaning of the input sentence without omitting any details:\nUn privil `ege d\u2019admission est le droit d\u2019un m \u00b4edecin d\u2019admettre un patient `a un"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f82-6db4-11ee-aabf-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-25", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "Un privil `ege d\u2019admission est le droit d\u2019un m \u00b4edecin d\u2019admettre un patient `a un\nh\u02c6opital ou un centre m \u00b4edical pour effectuer undiagnostic ouuneproc\u00b4edure, selon\nsonstatut detravailleur dessoins desant\u00b4e`al\u2019h\u02c6opital.\nLet us consider another sentence from the test set:\nThis kind of experience is part of Disney\u2019s efforts to \u201dextend the lifetime of its\nseries and build new relationships with audiences viadigital platforms that are\nbecoming ever more important, \u201d headded.\nThe translation by the RNNencdec-50 is\nCe type d\u2019exp \u00b4erience fait partie des initiatives du Disney pour \u201dprolonger la dur \u00b4ee\nde vie de ses nouvelles et de d \u00b4evelopper des liens avec les lecteurs num\u00b4eriques qui\ndeviennent plus complexes.\nAs with the previous example, the RNNencdec began deviating from the actual meaning of the\nsource sentence after generating approximately 30 words (see the underlined phrase). After that"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f83-6db4-11ee-8778-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-26", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "As with the previous example, the RNNencdec began deviating from the actual meaning of the\nsource sentence after generating approximately 30 words (see the underlined phrase). After that\npoint, the quality of the translation deteriorates, with basic mistakes such as the lack of a closing\nquotation mark.\nAgain, the RNNsearch-50 was able to translate this long sentence correctly:\nCe genre d\u2019exp \u00b4erience fait partie des efforts de Disney pour \u201dprolonger la dur \u00b4ee\nde vie de ses s \u00b4eries et cr \u00b4eer de nouvelles relations avec des publics viades\nplateformes num\u00b4eriques deplus enplus importantes\u201d, a-t-il ajout \u00b4e.\nIn conjunction with the quantitative results presented already, these qualitative observations con\ufb01rm our hypotheses that the RNNsearch architecture enables far more reliable translation of long\nsentences than the standard RNNencdec model.\nIn Appendix C, we provide a few more sample translations of long source sentences generated by\nthe RNNencdec-50, RNNsearch-50 and Google Translate along with the reference translations.\n6 R ELATED WORK\n6.1 L EARNING TO ALIGN\nA similar approach of aligning an output symbol with an input symbol was proposed recently by"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f84-6db4-11ee-b4f6-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-27", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "6 R ELATED WORK\n6.1 L EARNING TO ALIGN\nA similar approach of aligning an output symbol with an input symbol was proposed recently by\nGraves (2013) in the context of handwriting synthesis. Handwriting synthesis is a task where the\nmodel is asked to generate handwriting of a given sequence of characters. In his work, he used a\nmixture of Gaussian kernels to compute the weights of the annotations, where the location, width\nand mixture coef\ufb01cient of each kernel was predicted from an alignment model. More speci\ufb01cally,\nhis alignment was restricted to predict the location such that the location increases monotonically.\nThe main difference from our approach is that, in (Graves, 2013), the modes of the weights of the\nannotations only move in one direction. In the context of machine translation, this is a severe limitation, as (long-distance) reordering is often needed to generate a grammatically correct translation\n(for instance, English-to-German).\nOur approach, on the other hand, requires computing the annotation weight of every word in the\nsource sentence for each word in the translation. This drawback is not severe with the task of\ntranslation in which most of input and output sentences are only 15\u201340 words. However, this may\nlimit the applicability of the proposed scheme to other tasks.\n8"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f85-6db4-11ee-93bd-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-28", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "translation in which most of input and output sentences are only 15\u201340 words. However, this may\nlimit the applicability of the proposed scheme to other tasks.\n8\nPublished as a conference paper at ICLR 2015\n6.2 N EURAL NETWORKS FOR MACHINE TRANSLATION\nSince Bengio et al. (2003) introduced a neural probabilistic language model which uses a neural network to model the conditional probability of a word given a \ufb01xed number of the preceding words,\nneural networks have widely been used in machine translation. However, the role of neural networks has been largely limited to simply providing a single feature to an existing statistical machine\ntranslation system or to re-rank a list of candidate translations provided by an existing system.\nFor instance, Schwenk (2012) proposed using a feedforward neural network to compute the score of\na pair of source and target phrases and to use the score as an additional feature in the phrase-based\nstatistical machine translation system. More recently, Kalchbrenner and Blunsom (2013) and Devlin\net al. (2014) reported the successful use of the neural networks as a sub-component of the existing\ntranslation system. Traditionally, a neural network trained as a target-side language model has been\nused to rescore or rerank a list of candidate translations (see, e.g., Schwenk et al. , 2006)."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f86-6db4-11ee-b462-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-29", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "used to rescore or rerank a list of candidate translations (see, e.g., Schwenk et al. , 2006).\nAlthough the above approaches were shown to improve the translation performance over the stateof-the-art machine translation systems, we are more interested in a more ambitious objective of\ndesigning a completely new translation system based on neural networks. The neural machine translation approach we consider in this paper is therefore a radical departure from these earlier works.\nRather than using a neural network as a part of the existing system, our model works on its own and\ngenerates a translation from a source sentence directly.\n7 C ONCLUSION\nThe conventional approach to neural machine translation, called an encoder\u2013decoder approach, encodes a whole input sentence into a \ufb01xed-length vector from which a translation will be decoded.\nWe conjectured that the use of a \ufb01xed-length context vector is problematic for translating long sentences, based on a recent empirical study reported by Cho et al. (2014b) and Pouget-Abadie et al.\n(2014).\nIn this paper, we proposed a novel architecture that addresses this issue. We extended the basic\nencoder\u2013decoder by letting a model (soft-)search for a set of input words, or their annotations computed by an encoder, when generating each target word. This frees the model from having to encode"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f87-6db4-11ee-96db-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-30", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "a whole source sentence into a \ufb01xed-length vector, and also lets the model focus only on information\nrelevant to the generation of the next target word. This has a major positive impact on the ability\nof the neural machine translation system to yield good results on longer sentences. Unlike with\nthe traditional machine translation systems, all of the pieces of the translation system, including\nthe alignment mechanism, are jointly trained towards a better log-probability of producing correct\ntranslations.\nWe tested the proposed model, called RNNsearch, on the task of English-to-French translation. The\nexperiment revealed that the proposed RNNsearch outperforms the conventional encoder\u2013decoder\nmodel (RNNencdec) signi\ufb01cantly, regardless of the sentence length and that it is much more robust to the length of a source sentence. From the qualitative analysis where we investigated the\n(soft-)alignment generated by the RNNsearch, we were able to conclude that the model can correctly align each target word with the relevant words, or their annotations, in the source sentence as\nit generated a correct translation.\nPerhaps more importantly, the proposed approach achieved a translation performance comparable to\nthe existing phrase-based statistical machine translation. It is a striking result, considering that the\nproposed architecture, or the whole family of neural machine translation, has only been proposed"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f88-6db4-11ee-8cc7-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-31", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "the existing phrase-based statistical machine translation. It is a striking result, considering that the\nproposed architecture, or the whole family of neural machine translation, has only been proposed\nas recently as this year. We believe the architecture proposed here is a promising step toward better\nmachine translation and a better understanding of natural languages in general.\nOne of challenges left for the future is to better handle unknown, or rare words. This will be required\nfor the model to be more widely used and to match the performance of current state-of-the-art\nmachine translation systems in all contexts.\n9\nPublished as a conference paper at ICLR 2015\nACKNOWLEDGMENTS\nThe authors would like to thank the developers of Theano (Bergstra et al. , 2010; Bastien et al. ,\n2012). We acknowledge the support of the following agencies for research funding and computing\nsupport: NSERC, Calcul Qu \u00b4ebec, Compute Canada, the Canada Research Chairs and CIFAR. Bahdanau thanks the support from Planet Intelligent Systems GmbH. We also thank Felix Hill, Bart van\nMerri \u00b4enboer, Jean Pouget-Abadie, Coline Devin and Tae-Ho Kim.\nREFERENCES\nAxelrod, A., He, X., and Gao, J. (2011). Domain adaptation via pseudo in-domain data selection."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f89-6db4-11ee-9b45-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-32", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "REFERENCES\nAxelrod, A., He, X., and Gao, J. (2011). Domain adaptation via pseudo in-domain data selection.\nInProceedings of the ACL Conference on Empirical Methods in Natural Language Processing\n(EMNLP) , pages 355\u2013362. Association for Computational Linguistics.\nBastien, F., Lamblin, P., Pascanu, R., Bergstra, J., Goodfellow, I. J., Bergeron, A., Bouchard, N.,\nand Bengio, Y . (2012). Theano: new features and speed improvements. Deep Learning and\nUnsupervised Feature Learning NIPS 2012 Workshop.\nBengio, Y ., Simard, P., and Frasconi, P. (1994). Learning long-term dependencies with gradient\ndescent is dif\ufb01cult. IEEE Transactions on Neural Networks ,5(2), 157\u2013166.\nBengio, Y ., Ducharme, R., Vincent, P., and Janvin, C. (2003). A neural probabilistic language model.\nJ. Mach. Learn. Res. ,3, 1137\u20131155."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f8a-6db4-11ee-b4b0-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-33", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "J. Mach. Learn. Res. ,3, 1137\u20131155.\nBergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., WardeFarley, D., and Bengio, Y . (2010). Theano: a CPU and GPU math expression compiler. In\nProceedings of the Python for Scienti\ufb01c Computing Conference (SciPy) . Oral Presentation.\nBoulanger-Lewandowski, N., Bengio, Y ., and Vincent, P. (2013). Audio chord recognition with\nrecurrent neural networks. In ISMIR .\nCho, K., van Merrienboer, B., Gulcehre, C., Bougares, F., Schwenk, H., and Bengio, Y . (2014a).\nLearning phrase representations using RNN encoder-decoder for statistical machine translation.\nInProceedings of the Empiricial Methods in Natural Language Processing (EMNLP 2014) . to\nappear.\nCho, K., van Merri \u00a8enboer, B., Bahdanau, D., and Bengio, Y . (2014b). On the properties of neural\nmachine translation: Encoder\u2013Decoder approaches. In Eighth Workshop on Syntax, Semantics"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f8b-6db4-11ee-9ff9-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-34", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "machine translation: Encoder\u2013Decoder approaches. In Eighth Workshop on Syntax, Semantics\nand Structure in Statistical Translation . to appear.\nDevlin, J., Zbib, R., Huang, Z., Lamar, T., Schwartz, R., and Makhoul, J. (2014). Fast and robust\nneural network joint models for statistical machine translation. In Association for Computational\nLinguistics .\nForcada, M. L. and \u02dcNeco, R. P. (1997). Recursive hetero-associative memories for translation. In\nJ. Mira, R. Moreno-D \u00b4\u0131az, and J. Cabestany, editors, Biological and Arti\ufb01cial Computation: From\nNeuroscience to Technology , volume 1240 of Lecture Notes in Computer Science , pages 453\u2013462.\nSpringer Berlin Heidelberg.\nGoodfellow, I., Warde-Farley, D., Mirza, M., Courville, A., and Bengio, Y . (2013). Maxout networks. In Proceedings of The 30th International Conference on Machine Learning , pages 1319\u2013\n1327.\nGraves, A. (2012). Sequence transduction with recurrent neural networks. In Proceedings of the\n29th International Conference on Machine Learning (ICML 2012) ."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f8c-6db4-11ee-a392-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-35", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "1327.\nGraves, A. (2012). Sequence transduction with recurrent neural networks. In Proceedings of the\n29th International Conference on Machine Learning (ICML 2012) .\nGraves, A. (2013). Generating sequences with recurrent neural networks. arXiv: 1308.0850\n[cs.NE] .\nGraves, A., Jaitly, N., and Mohamed, A.-R. (2013). Hybrid speech recognition with deep bidirectional LSTM. In Automatic Speech Recognition and Understanding (ASRU), 2013 IEEE Workshop on , pages 273\u2013278.\n10\nPublished as a conference paper at ICLR 2015\nHermann, K. and Blunsom, P. (2014). Multilingual distributed representations without word alignment. In Proceedings of the Second International Conference on Learning Representations (ICLR\n2014) .\nHochreiter, S. (1991). Untersuchungen zu dynamischen neuronalen Netzen. Diploma thesis, Institut\nf\u00a8ur Informatik, Lehrstuhl Prof. Brauer, Technische Universit \u00a8at M \u00a8unchen.\nHochreiter, S. and Schmidhuber, J. (1997). Long short-term memory. Neural Computation ,9(8),\n1735\u20131780."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f8d-6db4-11ee-85e0-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-36", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "Hochreiter, S. and Schmidhuber, J. (1997). Long short-term memory. Neural Computation ,9(8),\n1735\u20131780.\nKalchbrenner, N. and Blunsom, P. (2013). Recurrent continuous translation models. In Proceedings\nof the ACL Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages\n1700\u20131709. Association for Computational Linguistics.\nKoehn, P. (2010). Statistical Machine Translation . Cambridge University Press, New York, NY ,\nUSA.\nKoehn, P., Och, F. J., and Marcu, D. (2003). Statistical phrase-based translation. In Proceedings\nof the 2003 Conference of the North American Chapter of the Association for Computational\nLinguistics on Human Language Technology - Volume 1 , NAACL \u201903, pages 48\u201354, Stroudsburg,\nPA, USA. Association for Computational Linguistics.\nPascanu, R., Mikolov, T., and Bengio, Y . (2013a). On the dif\ufb01culty of training recurrent neural\nnetworks. In ICML\u20192013 .\nPascanu, R., Mikolov, T., and Bengio, Y . (2013b). On the dif\ufb01culty of training recurrent neural"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f8e-6db4-11ee-bb7c-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-37", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "Pascanu, R., Mikolov, T., and Bengio, Y . (2013b). On the dif\ufb01culty of training recurrent neural\nnetworks. In Proceedings of the 30th International Conference on Machine Learning (ICML\n2013) .\nPascanu, R., Gulcehre, C., Cho, K., and Bengio, Y . (2014). How to construct deep recurrent neural\nnetworks. In Proceedings of the Second International Conference on Learning Representations\n(ICLR 2014) .\nPouget-Abadie, J., Bahdanau, D., van Merri \u00a8enboer, B., Cho, K., and Bengio, Y . (2014). Overcoming\nthe curse of sentence length for neural machine translation using automatic segmentation. In\nEighth Workshop on Syntax, Semantics and Structure in Statistical Translation . to appear.\nSchuster, M. and Paliwal, K. K. (1997). Bidirectional recurrent neural networks. Signal Processing,\nIEEE Transactions on ,45(11), 2673\u20132681.\nSchwenk, H. (2012). Continuous space translation models for phrase-based statistical machine\ntranslation. In M. Kay and C. Boitet, editors, Proceedings of the 24th International Conference on"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f8f-6db4-11ee-8e36-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-38", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "translation. In M. Kay and C. Boitet, editors, Proceedings of the 24th International Conference on\nComputational Linguistics (COLIN) , pages 1071\u20131080. Indian Institute of Technology Bombay.\nSchwenk, H., Dchelotte, D., and Gauvain, J.-L. (2006). Continuous space language models for\nstatistical machine translation. In Proceedings of the COLING/ACL on Main conference poster\nsessions , pages 723\u2013730. Association for Computational Linguistics.\nSutskever, I., Vinyals, O., and Le, Q. (2014). Sequence to sequence learning with neural networks.\nInAdvances in Neural Information Processing Systems (NIPS 2014) .\nZeiler, M. D. (2012). ADADELTA: An adaptive learning rate method. arXiv: 1212.5701\n[cs.LG] .\n11\nPublished as a conference paper at ICLR 2015\nA M ODEL ARCHITECTURE\nA.1 A RCHITECTURAL CHOICES\nThe proposed scheme in Section 3 is a general framework where one can freely de\ufb01ne, for instance,\nthe activation functions fof recurrent neural networks (RNN) and the alignment model a. Here, we\ndescribe the choices we made for the experiments in this paper."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f90-6db4-11ee-ad86-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-39", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "the activation functions fof recurrent neural networks (RNN) and the alignment model a. Here, we\ndescribe the choices we made for the experiments in this paper.\nA.1.1 R ECURRENT NEURAL NETWORK\nFor the activation function fof an RNN, we use the gated hidden unit recently proposed by Cho\net al. (2014a). The gated hidden unit is an alternative to the conventional simple units such as an\nelement-wise tanh . This gated unit is similar to a long short-term memory (LSTM) unit proposed\nearlier by Hochreiter and Schmidhuber (1997), sharing with it the ability to better model and learn\nlong-term dependencies. This is made possible by having computation paths in the unfolded RNN\nfor which the product of derivatives is close to 1. These paths allow gradients to \ufb02ow backward\neasily without suffering too much from the vanishing effect (Hochreiter, 1991; Bengio et al. , 1994;\nPascanu et al. , 2013a). It is therefore possible to use LSTM units instead of the gated hidden unit\ndescribed here, as was done in a similar context by Sutskever et al. (2014).\nThe new state siof the RNN employing ngated hidden units8is computed by"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f91-6db4-11ee-af50-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-40", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "described here, as was done in a similar context by Sutskever et al. (2014).\nThe new state siof the RNN employing ngated hidden units8is computed by\nsi=f(si\u00001;yi\u00001;ci) = (1\u0000zi)\u000esi\u00001+zi\u000e~si;\nwhere\u000eis an element-wise multiplication, and ziis the output of the update gates (see below). The\nproposed updated state ~siis computed by\n~si= tanh (We(yi\u00001) +U[ri\u000esi\u00001] +Cci);\nwheree(yi\u00001)2Rmis anm-dimensional embedding of a word yi\u00001, andriis the output of the\nreset gates (see below). When yiis represented as a 1-of-Kvector,e(yi)is simply a column of an\nembedding matrix E2Rm\u0002K. Whenever possible, we omit bias terms to make the equations less\ncluttered.\nThe update gates ziallow each hidden unit to maintain its previous activation, and the reset gates ri\ncontrol how much and what information from the previous state should be reset. We compute them\nby\nzi=\u001b(Wze(yi\u00001) +Uzsi\u00001+Czci);"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f92-6db4-11ee-a741-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-41", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "by\nzi=\u001b(Wze(yi\u00001) +Uzsi\u00001+Czci);\nri=\u001b(Wre(yi\u00001) +Ursi\u00001+Crci);\nwhere\u001b(\u0001)is a logistic sigmoid function.\nAt each step of the decoder, we compute the output probability (Eq. (4)) as a multi-layered function (Pascanu et al. , 2014). We use a single hidden layer of maxout units (Goodfellow et al. , 2013)\nand normalize the output probabilities (one for each word) with a softmax function (see Eq. (6)).\nA.1.2 A LIGNMENT MODEL\nThe alignment model should be designed considering that the model needs to be evaluated Tx\u0002Ty\ntimes for each sentence pair of lengths TxandTy. In order to reduce computation, we use a singlelayer multilayer perceptron such that\na(si\u00001;hj) =v>\natanh (Wasi\u00001+Uahj);\nwhereWa2Rn\u0002n;Ua2Rn\u00022nandva2Rnare the weight matrices. Since Uahjdoes not\ndepend oni, we can pre-compute it in advance to minimize the computational cost."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f93-6db4-11ee-9bd3-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-42", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "depend oni, we can pre-compute it in advance to minimize the computational cost.\n8Here, we show the formula of the decoder. The same formula can be used in the encoder by simply\nignoring the context vector ciand the related terms.\n12\nPublished as a conference paper at ICLR 2015\nA.2 D ETAILED DESCRIPTION OF THE MODEL\nA.2.1 E NCODER\nIn this section, we describe in detail the architecture of the proposed model (RNNsearch) used in the\nexperiments (see Sec. 4\u20135). From here on, we omit all bias terms in order to increase readability.\nThe model takes a source sentence of 1-of-K coded word vectors as input\nx= (x1;:::;x Tx); xi2RKx\nand outputs a translated sentence of 1-of-K coded word vectors\ny= (y1;:::;y Ty); yi2RKy;\nwhereKxandKyare the vocabulary sizes of source and target languages, respectively. TxandTy\nrespectively denote the lengths of source and target sentences.\nFirst, the forward states of the bidirectional recurrent neural network (BiRNN) are computed:\n\u0000 !hi=("}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f94-6db4-11ee-95be-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-43", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "respectively denote the lengths of source and target sentences.\nFirst, the forward states of the bidirectional recurrent neural network (BiRNN) are computed:\n\u0000 !hi=(\n(1\u0000\u0000 !zi)\u000e\u0000 !hi\u00001+\u0000 !zi\u000e\u0000 !hi, ifi>0\n0 , ifi= 0\nwhere\n\u0000 !hi= tanh\u0010\u0000 !WExi+\u0000 !Uh\u0000 !ri\u000e\u0000 !hi\u00001i\u0011\n\u0000 !zi=\u001b\u0010\u0000 !WzExi+\u0000 !Uz\u0000 !hi\u00001\u0011\n\u0000 !ri=\u001b\u0010\u0000 !WrExi+\u0000 !Ur\u0000 !hi\u00001\u0011\n:\nE2Rm\u0002Kxis the word embedding matrix.\u0000 !W;\u0000 !Wz;\u0000 !Wr2Rn\u0002m,\u0000 !U;\u0000 !Uz;\u0000 !Ur2Rn\u0002nare\nweight matrices. mandnare the word embedding dimensionality and the number of hidden units,\nrespectively. \u001b(\u0001)is as usual a logistic sigmoid function.\nThe backward states ( \u0000h1;\u0001\u0001\u0001; \u0000hTx)are computed similarly. We share the word embedding matrix\nEbetween the forward and backward RNNs, unlike the weight matrices."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f95-6db4-11ee-af1c-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-44", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "Ebetween the forward and backward RNNs, unlike the weight matrices.\nWe concatenate the forward and backward states to to obtain the annotations (h1;h2;\u0001\u0001\u0001;hTx),\nwhere\nhi=\"\u0000 !hi \u0000hi#\n(7)\nA.2.2 D ECODER\nThe hidden state siof the decoder given the annotations from the encoder is computed by\nsi=(1\u0000zi)\u000esi\u00001+zi\u000e~si;\nwhere\n~si= tanh (WEy i\u00001+U[ri\u000esi\u00001] +Cci)\nzi=\u001b(WzEyi\u00001+Uzsi\u00001+Czci)\nri=\u001b(WrEyi\u00001+Ursi\u00001+Crci)\nEis the word embedding matrix for the target language. W;W z;Wr2Rn\u0002m,U;Uz;Ur2Rn\u0002n,\nandC;C z;Cr2Rn\u00022nare weights. Again, mandnare the word embedding dimensionality\nand the number of hidden units, respectively. The initial hidden state s0is computed by s0=\ntanh\u0010\nWs \u0000h1\u0011\n;whereWs2Rn\u0002n."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f96-6db4-11ee-ae68-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-45", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "tanh\u0010\nWs \u0000h1\u0011\n;whereWs2Rn\u0002n.\nThe context vector ciare recomputed at each step by the alignment model:\nci=TxX\nj=1\u000bijhj;\n13\nPublished as a conference paper at ICLR 2015\nModel Updates (\u0002105)Epochs Hours GPU Train NLL Dev. NLL\nRNNenc-30 8.46 6.4 109 TITAN BLACK 28.1 53.0\nRNNenc-50 6.00 4.5 108 Quadro K-6000 44.0 43.6\nRNNsearch-30 4.71 3.6 113 TITAN BLACK 26.7 47.2\nRNNsearch-50 2.88 2.2 111 Quadro K-6000 40.7 38.1\nRNNsearch-50?6.67 5.0 252 Quadro K-6000 36.7 35.2\nTable 2: Learning statistics and relevant information. Each update corresponds to updating the\nparameters once using a single minibatch. One epoch is one pass through the training set. NLL is\nthe average conditional log-probabilities of the sentences in either the training set or the development\nset. Note that the lengths of the sentences differ.\nwhere\n\u000bij=exp (eij)PTx\nk=1exp (eik)\neij=v>"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f97-6db4-11ee-8175-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-46", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "set. Note that the lengths of the sentences differ.\nwhere\n\u000bij=exp (eij)PTx\nk=1exp (eik)\neij=v>\natanh (Wasi\u00001+Uahj);\nandhjis thej-th annotation in the source sentence (see Eq. (7)). va2Rn0;Wa2Rn0\u0002nand\nUa2Rn0\u00022nare weight matrices. Note that the model becomes RNN Encoder\u2013Decoder (Cho\net al. , 2014a), if we \ufb01x cito\u0000 !hTx.\nWith the decoder state si\u00001, the context ciand the last generated word yi\u00001, we de\ufb01ne the probability\nof a target word yias\np(yijsi;yi\u00001;ci)/exp\u0000\ny>\niWoti\u0001\n;\nwhere\nti=\u0002\nmax\b~ti;2j\u00001;~ti;2j\t\u0003>\nj=1;:::;l\nand~ti;kis thek-th element of a vector ~tiwhich is computed by\n~ti=Uosi\u00001+VoEyi\u00001+Coci:"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f98-6db4-11ee-b047-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-47", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "and~ti;kis thek-th element of a vector ~tiwhich is computed by\n~ti=Uosi\u00001+VoEyi\u00001+Coci:\nWo2RKy\u0002l,Uo2R2l\u0002n,Vo2R2l\u0002mandCo2R2l\u00022nare weight matrices. This can be understood as having a deep output (Pascanu et al. , 2014) with a single maxout hidden layer (Goodfellow\net al. , 2013).\nA.2.3 M ODEL SIZE\nFor all the models used in this paper, the size of a hidden layer nis 1000, the word embedding\ndimensionality mis 620 and the size of the maxout hidden layer in the deep output lis 500. The\nnumber of hidden units in the alignment model n0is 1000.\nB T RAINING PROCEDURE\nB.1 P ARAMETER INITIALIZATION\nWe initialized the recurrent weight matrices U;Uz;Ur; \u0000U; \u0000Uz; \u0000Ur;\u0000 !U;\u0000 !Uzand\u0000 !Uras random orthogonal matrices. For WaandUa, we initialized them by sampling each element from the Gaussian"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f99-6db4-11ee-b24a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-48", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "distribution of mean 0and variance 0:0012. All the elements of Vaand all the bias vectors were initialized to zero. Any other weight matrix was initialized by sampling from the Gaussian distribution\nof mean 0and variance 0:012.\nB.2 T RAINING\nWe used the stochastic gradient descent (SGD) algorithm. Adadelta (Zeiler, 2012) was used to\nautomatically adapt the learning rate of each parameter ( \u000f= 10\u00006and\u001a= 0:95). We explicitly\n14\nPublished as a conference paper at ICLR 2015\nnormalized the L2-norm of the gradient of the cost function each time to be at most a prede\ufb01ned\nthreshold of 1, when the norm was larger than the threshold (Pascanu et al. , 2013b). Each SGD\nupdate direction was computed with a minibatch of 80 sentences.\nAt each update our implementation requires time proportional to the length of the longest sentence in\na minibatch. Hence, to minimize the waste of computation, before every 20-th update, we retrieved\n1600 sentence pairs, sorted them according to the lengths and split them into 20 minibatches. The\ntraining data was shuf\ufb02ed once before training and was traversed sequentially in this manner.\nIn Tables 2 we present the statistics related to training all the models used in the experiments."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f9a-6db4-11ee-9a04-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-49", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "In Tables 2 we present the statistics related to training all the models used in the experiments.\nC T RANSLATIONS OF LONG SENTENCES\nSource An admitting privilege is the right of a doctor to admit a patient to a hospital or a medical centre\nto carry out a diagnosis or a procedure, based on his status as a health care worker at a hospital.\nReference Le privil `ege d\u2019admission est le droit d\u2019un m \u00b4edecin, en vertu de son statut de membre soignant\nd\u2019un h \u02c6opital, d\u2019admettre un patient dans un h \u02c6opital ou un centre m \u00b4edical a\ufb01n d\u2019y d \u00b4elivrer un\ndiagnostic ou un traitement.\nRNNenc-50 Un privil `ege d\u2019admission est le droit d\u2019un m \u00b4edecin de reconna \u02c6\u0131tre un patient `a l\u2019h \u02c6opital ou un\ncentre m \u00b4edical d\u2019un diagnostic ou de prendre un diagnostic en fonction de son \u00b4etat de sant \u00b4e."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f9b-6db4-11ee-9584-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-50", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "centre m \u00b4edical d\u2019un diagnostic ou de prendre un diagnostic en fonction de son \u00b4etat de sant \u00b4e.\nRNNsearch-50 Un privil `ege d\u2019admission est le droit d\u2019un m \u00b4edecin d\u2019admettre un patient `a un h \u02c6opital ou un\ncentre m \u00b4edical pour effectuer un diagnostic ou une proc \u00b4edure, selon son statut de travailleur des\nsoins de sant \u00b4e`a l\u2019h \u02c6opital.\nGoogle\nTranslateUn privil `ege admettre est le droit d\u2019un m \u00b4edecin d\u2019admettre un patient dans un h \u02c6opital ou un\ncentre m \u00b4edical pour effectuer un diagnostic ou une proc \u00b4edure, fond \u00b4ee sur sa situation en tant\nque travailleur de soins de sant \u00b4e dans un h \u02c6opital.\nSource This kind of experience is part of Disney\u2019s efforts to \u201dextend the lifetime of its series and build\nnew relationships with audiences via digital platforms that are becoming ever more important,\u201d\nhe added."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f9c-6db4-11ee-b96b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-51", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "new relationships with audiences via digital platforms that are becoming ever more important,\u201d\nhe added.\nReference Ce type d\u2019exp \u00b4erience entre dans le cadre des efforts de Disney pour \u201d \u00b4etendre la dur \u00b4ee de\nvie de ses s \u00b4eries et construire de nouvelles relations avec son public gr \u02c6ace`a des plateformes\nnum\u00b4eriques qui sont de plus en plus importantes\u201d, a-t-il ajout \u00b4e.\nRNNenc-50 Ce type d\u2019exp \u00b4erience fait partie des initiatives du Disney pour \u201dprolonger la dur \u00b4ee de vie de\nses nouvelles et de d \u00b4evelopper des liens avec les lecteurs num \u00b4eriques qui deviennent plus complexes.\nRNNsearch-50 Ce genre d\u2019exp \u00b4erience fait partie des efforts de Disney pour \u201dprolonger la dur \u00b4ee de vie de ses\ns\u00b4eries et cr \u00b4eer de nouvelles relations avec des publics via des plateformes num \u00b4eriques de plus\nen plus importantes\u201d, a-t-il ajout \u00b4e.\nGoogle"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f9d-6db4-11ee-90cf-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-52", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "en plus importantes\u201d, a-t-il ajout \u00b4e.\nGoogle\nTranslateCe genre d\u2019exp \u00b4erience fait partie des efforts de Disney `a \u201c\u00b4etendre la dur \u00b4ee de vie de sa s \u00b4erie et\nconstruire de nouvelles relations avec le public par le biais des plates-formes num \u00b4eriques qui\ndeviennent de plus en plus important\u201d, at-il ajout \u00b4e.\nSource In a press conference on Thursday, Mr Blair stated that there was nothing in this video that might\nconstitute a \u201dreasonable motive\u201d that could lead to criminal charges being brought against the\nmayor.\nReference En conf \u00b4erence de presse, jeudi, M. Blair a af\ufb01rm \u00b4e qu\u2019il n\u2019y avait rien dans cette vid \u00b4eo qui puisse\nconstituer des \u201dmotifs raisonnables\u201d pouvant mener au d \u00b4ep\u02c6ot d\u2019une accusation criminelle contre\nle maire."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f9e-6db4-11ee-ace2-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-53", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "le maire.\nRNNenc-50 Lors de la conf \u00b4erence de presse de jeudi, M. Blair a dit qu\u2019il n\u2019y avait rien dans cette vid \u00b4eo qui\npourrait constituer une \u201dmotivation raisonnable\u201d pouvant entra \u02c6\u0131ner des accusations criminelles\nport\u00b4ees contre le maire.\nRNNsearch-50 Lors d\u2019une conf \u00b4erence de presse jeudi, M. Blair a d \u00b4eclar \u00b4e qu\u2019il n\u2019y avait rien dans cette vid \u00b4eo qui\npourrait constituer un \u201dmotif raisonnable\u201d qui pourrait conduire `a des accusations criminelles\ncontre le maire.\nGoogle\nTranslateLors d\u2019une conf \u00b4erence de presse jeudi, M. Blair a d \u00b4eclar \u00b4e qu\u2019il n\u2019y avait rien dans cette vido\nqui pourrait constituer un \u201dmotif raisonnable\u201d qui pourrait mener `a des accusations criminelles\nportes contre le maire."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3f9f-6db4-11ee-931d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.0473-54", "title": "Neural Machine Translation by Jointly Learning to Align and Translate", "source": "http://arxiv.org/pdf/1409.0473", "start_index": 0, "chroma:document": "qui pourrait constituer un \u201dmotif raisonnable\u201d qui pourrait mener `a des accusations criminelles\nportes contre le maire.\nTable 3: The translations generated by RNNenc-50 and RNNsearch-50 from long source sentences\n(30 words or more) selected from the test set. For each source sentence, we also show the goldstandard translation. The translations by Google Translate were made on 27 August 2014.\n15"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fa0-6db4-11ee-b98b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-0", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "arXiv:1409.1556v6  [cs.CV]  10 Apr 2015Publishedasa conferencepaperat ICLR2015\nVERYDEEPCONVOLUTIONAL NETWORKS\nFORLARGE-SCALEIMAGERECOGNITION\nKarenSimonyan\u2217& AndrewZisserman+\nVisualGeometryGroup,DepartmentofEngineeringScience, UniversityofOxford\n{karen,az }@robots.ox.ac.uk\nABSTRACT\nIn this work we investigate the effect of the convolutional n etwork depth on its\naccuracy in the large-scale image recognition setting. Our main contribution is\na thorough evaluation of networks of increasing depth using an architecture with\nverysmall( 3\u00d73)convolution\ufb01lters,whichshowsthatasigni\ufb01cantimprove ment\non the prior-art con\ufb01gurations can be achieved by pushing th e depth to 16\u201319\nweight layers. These \ufb01ndings were the basis of our ImageNet C hallenge 2014\nsubmission,whereourteamsecuredthe\ufb01rstandthesecondpl acesinthelocalisation and classi\ufb01cation tracks respectively. We also show th at our representations\ngeneralise well to other datasets, where they achieve state -of-the-art results. We"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fa1-6db4-11ee-8cff-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-1", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "generalise well to other datasets, where they achieve state -of-the-art results. We\nhave made our two best-performingConvNet models publicly a vailable to facilitate furtherresearchontheuse ofdeepvisualrepresentati onsincomputervision.\n1 INTRODUCTION\nConvolutional networks (ConvNets) have recently enjoyed a great success in large-scale image and video recognition (Krizhevskyetal., 2012; Zeiler& Fergus, 2013; Sermanetet al., 2014;\nSimonyan& Zisserman, 2014) which has become possible due to the large public image repositories,suchasImageNet(Denget al.,2009),andhigh-perform ancecomputingsystems,suchasGPUs\norlarge-scaledistributedclusters(Deanet al., 2012). In particular,animportantroleintheadvance\nofdeepvisualrecognitionarchitectureshasbeenplayedby theImageNetLarge-ScaleVisualRecognition Challenge (ILSVRC) (Russakovskyet al., 2014), whic h has served as a testbed for a few"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fa2-6db4-11ee-9dcf-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-2", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "generationsof large-scale image classi\ufb01cation systems, f rom high-dimensionalshallow feature encodings(Perronninetal.,2010)(thewinnerofILSVRC-2011 )todeepConvNets(Krizhevskyet al.,\n2012)(thewinnerofILSVRC-2012).\nWith ConvNets becoming more of a commodity in the computer vi sion \ufb01eld, a number of attempts have been made to improve the original architecture o f Krizhevskyet al. (2012) in a\nbid to achieve better accuracy. For instance, the best-perf orming submissions to the ILSVRC2013 (Zeiler&Fergus, 2013; Sermanetetal., 2014) utilised smaller receptive window size and\nsmaller stride of the \ufb01rst convolutional layer. Another lin e of improvements dealt with training\nand testing the networks densely over the whole image and ove r multiple scales (Sermanetet al.,\n2014; Howard, 2014). In this paper, we address another impor tant aspect of ConvNet architecture\ndesign\u2013itsdepth. Tothisend,we \ufb01xotherparametersofthea rchitecture,andsteadilyincreasethe\ndepth of the network by adding more convolutionallayers, wh ich is feasible due to the use of very"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fa3-6db4-11ee-8dd3-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-3", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "depth of the network by adding more convolutionallayers, wh ich is feasible due to the use of very\nsmall (3\u00d73)convolution\ufb01ltersinall layers.\nAs a result, we come up with signi\ufb01cantly more accurate ConvN et architectures, which not only\nachieve the state-of-the-art accuracy on ILSVRC classi\ufb01ca tion and localisation tasks, but are also\napplicabletootherimagerecognitiondatasets,wherethey achieveexcellentperformanceevenwhen\nusedasa partofa relativelysimple pipelines(e.g.deepfea turesclassi\ufb01ed byalinearSVM without\n\ufb01ne-tuning). We havereleasedourtwobest-performingmode ls1tofacilitatefurtherresearch.\nThe rest of the paper is organised as follows. In Sect. 2, we de scribe our ConvNet con\ufb01gurations.\nThe details of the image classi\ufb01cation trainingand evaluat ionare then presented in Sect. 3, and the\n\u2217current af\ufb01liation: Google DeepMind+current af\ufb01liation: Universityof Oxfordand Google DeepMi nd\n1http://www.robots.ox.ac.uk/ \u02dcvgg/research/very_deep/\n1"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fa4-6db4-11ee-97bc-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-4", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "1http://www.robots.ox.ac.uk/ \u02dcvgg/research/very_deep/\n1\nPublishedasa conferencepaperat ICLR2015\ncon\ufb01gurations are compared on the ILSVRC classi\ufb01cation tas k in Sect. 4. Sect. 5 concludes the\npaper. For completeness,we also describeand assess our ILS VRC-2014object localisationsystem\ninAppendixA,anddiscussthegeneralisationofverydeepfe aturestootherdatasetsinAppendixB.\nFinally,AppendixCcontainsthelist ofmajorpaperrevisio ns.\n2 CONVNETCONFIGURATIONS\nTo measure the improvement brought by the increased ConvNet depth in a fair setting, all our\nConvNet layer con\ufb01gurations are designed using the same pri nciples, inspired by Ciresan etal.\n(2011); Krizhevskyet al. (2012). In this section, we \ufb01rst de scribe a generic layout of our ConvNet\ncon\ufb01gurations(Sect.2.1)andthendetailthespeci\ufb01ccon\ufb01g urationsusedintheevaluation(Sect.2.2)."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fa5-6db4-11ee-9602-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-5", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "Ourdesignchoicesarethendiscussedandcomparedtothepri orart inSect. 2.3.\n2.1 A RCHITECTURE\nDuring training, the input to our ConvNets is a \ufb01xed-size 224\u00d7224RGB image. The only preprocessingwedoissubtractingthemeanRGBvalue,computed onthetrainingset,fromeachpixel.\nTheimageispassedthroughastackofconvolutional(conv.) layers,whereweuse\ufb01lterswithavery\nsmall receptive \ufb01eld: 3\u00d73(which is the smallest size to capture the notion of left/rig ht, up/down,\ncenter). In one of the con\ufb01gurationswe also utilise 1\u00d71convolution\ufb01lters, which can be seen as\na linear transformationof the input channels (followed by n on-linearity). The convolutionstride is\n\ufb01xedto1pixel;thespatialpaddingofconv.layerinputissuchthatt hespatialresolutionispreserved\nafterconvolution,i.e. the paddingis 1pixel for3\u00d73conv.layers. Spatial poolingis carriedoutby"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fa6-6db4-11ee-b316-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-6", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "afterconvolution,i.e. the paddingis 1pixel for3\u00d73conv.layers. Spatial poolingis carriedoutby\n\ufb01vemax-poolinglayers,whichfollowsomeoftheconv.layer s(notalltheconv.layersarefollowed\nbymax-pooling). Max-poolingisperformedovera 2\u00d72pixelwindow,withstride 2.\nAstackofconvolutionallayers(whichhasadifferentdepth indifferentarchitectures)isfollowedby\nthree Fully-Connected(FC) layers: the \ufb01rst two have4096ch annelseach,the thirdperforms1000way ILSVRC classi\ufb01cation and thus contains1000channels(o ne foreach class). The \ufb01nal layer is\nthesoft-maxlayer. Thecon\ufb01gurationofthefullyconnected layersis thesameinall networks.\nAllhiddenlayersareequippedwiththerecti\ufb01cation(ReLU( Krizhevskyetal.,2012))non-linearity.\nWe note that none of our networks (except for one) contain Loc al Response Normalisation\n(LRN) normalisation (Krizhevskyet al., 2012): as will be sh own in Sect. 4, such normalisation"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fa7-6db4-11ee-841a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-7", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "(LRN) normalisation (Krizhevskyet al., 2012): as will be sh own in Sect. 4, such normalisation\ndoes not improve the performance on the ILSVRC dataset, but l eads to increased memory consumption and computation time. Where applicable, the param eters for the LRN layer are those\nof(Krizhevskyetal., 2012).\n2.2 C ONFIGURATIONS\nThe ConvNet con\ufb01gurations, evaluated in this paper, are out lined in Table 1, one per column. In\nthe following we will refer to the nets by their names (A\u2013E). A ll con\ufb01gurationsfollow the generic\ndesign presented in Sect. 2.1, and differ only in the depth: f rom 11 weight layers in the network A\n(8conv.and3FClayers)to19weightlayersinthenetworkE(1 6conv.and3FClayers). Thewidth\nof conv.layers (the number of channels) is rather small, sta rting from 64in the \ufb01rst layer and then\nincreasingbyafactorof 2aftereachmax-poolinglayer,untilit reaches 512.\nIn Table 2 we reportthe numberof parametersfor each con\ufb01gur ation. In spite of a large depth, the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fa8-6db4-11ee-9ba5-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-8", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "In Table 2 we reportthe numberof parametersfor each con\ufb01gur ation. In spite of a large depth, the\nnumberof weights in our netsis not greater thanthe numberof weightsin a moreshallow net with\nlargerconv.layerwidthsandreceptive\ufb01elds(144Mweights in(Sermanetet al., 2014)).\n2.3 D ISCUSSION\nOur ConvNet con\ufb01gurations are quite different from the ones used in the top-performing entries\nof the ILSVRC-2012 (Krizhevskyetal., 2012) and ILSVRC-201 3 competitions (Zeiler& Fergus,\n2013;Sermanetet al.,2014). Ratherthanusingrelativelyl argereceptive\ufb01eldsinthe\ufb01rstconv.layers(e.g.11\u00d711withstride 4in(Krizhevskyet al.,2012),or 7\u00d77withstride 2in(Zeiler& Fergus,\n2013; Sermanetet al., 2014)), we use very small 3\u00d73receptive \ufb01elds throughout the whole net,\nwhichareconvolvedwiththeinputateverypixel(withstrid e1). Itiseasytoseethatastackoftwo"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fa9-6db4-11ee-b0dd-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-9", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "whichareconvolvedwiththeinputateverypixel(withstrid e1). Itiseasytoseethatastackoftwo\n3\u00d73conv.layers(withoutspatialpoolinginbetween)hasaneff ectivereceptive\ufb01eldof 5\u00d75;three\n2\nPublishedasa conferencepaperat ICLR2015\nTable 1:ConvNet con\ufb01gurations (shown in columns). The depth of the con\ufb01gurations increase s\nfromtheleft(A)totheright(E),asmorelayersareadded(th eaddedlayersareshowninbold). The\nconvolutional layer parameters are denoted as \u201cconv /an}bracketle{treceptive \ufb01eld size /an}bracketri}ht-/an}bracketle{tnumber of channels /an}bracketri}ht\u201d.\nTheReLU activationfunctionisnotshownforbrevity.\nConvNet Con\ufb01guration\nA A-LRN B C D E\n11weight 11weight 13 weight 16weight 16weight 19 weight\nlayers layers layers layers layers layers\ninput (224\u00d7224RGBimage)\nconv3-64 conv3-64 conv3-64 conv3-64 conv3-64 conv3-64"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3faa-6db4-11ee-a8e1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-10", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "input (224\u00d7224RGBimage)\nconv3-64 conv3-64 conv3-64 conv3-64 conv3-64 conv3-64\nLRN conv3-64 conv3-64 conv3-64 conv3-64\nmaxpool\nconv3-128 conv3-128 conv3-128 conv3-128 conv3-128 conv3-128\nconv3-128 conv3-128 conv3-128 conv3-128\nmaxpool\nconv3-256 conv3-256 conv3-256 conv3-256 conv3-256 conv3-256\nconv3-256 conv3-256 conv3-256 conv3-256 conv3-256 conv3-256\nconv1-256 conv3-256 conv3-256\nconv3-256\nmaxpool\nconv3-512 conv3-512 conv3-512 conv3-512 conv3-512 conv3-512\nconv3-512 conv3-512 conv3-512 conv3-512 conv3-512 conv3-512\nconv1-512 conv3-512 conv3-512\nconv3-512\nmaxpool\nconv3-512 conv3-512 conv3-512 conv3-512 conv3-512 conv3-512\nconv3-512 conv3-512 conv3-512 conv3-512 conv3-512 conv3-512"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fab-6db4-11ee-a30d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-11", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "conv3-512 conv3-512 conv3-512 conv3-512 conv3-512 conv3-512\nconv1-512 conv3-512 conv3-512\nconv3-512\nmaxpool\nFC-4096\nFC-4096\nFC-1000\nsoft-max\nTable2:Number ofparameters (inmillions).\nNetwork A,A-LRN BCDE\nNumber of parameters 133 133134138144\nsuch layers have a 7\u00d77effectivereceptive \ufb01eld. So what have we gainedby using, fo r instance, a\nstackofthree 3\u00d73conv.layersinsteadofasingle 7\u00d77layer? First,weincorporatethreenon-linear\nrecti\ufb01cation layers instead of a single one, which makes the decision functionmore discriminative.\nSecond, we decrease the number of parameters: assuming that both the input and the output of a\nthree-layer 3\u00d73convolutionstack has Cchannels,the stack is parametrisedby 3/parenleftbig\n32C2/parenrightbig\n= 27C2\nweights; at the same time, a single 7\u00d77conv. layer would require 72C2= 49C2parameters, i.e."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fac-6db4-11ee-b749-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-12", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "= 27C2\nweights; at the same time, a single 7\u00d77conv. layer would require 72C2= 49C2parameters, i.e.\n81%more. Thiscan be seen as imposinga regularisationon the 7\u00d77conv.\ufb01lters, forcingthemto\nhaveadecompositionthroughthe 3\u00d73\ufb01lters(withnon-linearityinjectedin between).\nThe incorporation of 1\u00d71conv. layers (con\ufb01guration C, Table 1) is a way to increase th e nonlinearity of the decision function without affecting the re ceptive \ufb01elds of the conv. layers. Even\nthoughinourcasethe 1\u00d71convolutionisessentiallyalinearprojectionontothespa ceofthesame\ndimensionality(thenumberofinputandoutputchannelsist hesame),anadditionalnon-linearityis\nintroducedbytherecti\ufb01cationfunction. Itshouldbenoted that1\u00d71conv.layershaverecentlybeen\nutilisedin the\u201cNetworkinNetwork\u201darchitectureofLinet a l.(2014).\nSmall-size convolution \ufb01lters have been previously used by Ciresan etal. (2011), but their nets"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fad-6db4-11ee-900a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-13", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "Small-size convolution \ufb01lters have been previously used by Ciresan etal. (2011), but their nets\nare signi\ufb01cantly less deep than ours, and they did not evalua te on the large-scale ILSVRC\ndataset. Goodfellowet al. (2014) applied deep ConvNets ( 11weight layers) to the task of\nstreet number recognition, and showed that the increased de pth led to better performance.\nGoogLeNet(Szegedyet al., 2014), a top-performingentryof the ILSVRC-2014classi\ufb01cation task,\nwas developed independentlyof our work, but is similar in th at it is based on very deep ConvNets\n3\nPublishedasa conferencepaperat ICLR2015\n(22 weight layers) and small convolution \ufb01lters (apart from 3\u00d73, they also use 1\u00d71and5\u00d75\nconvolutions). Their network topology is, however, more co mplex than ours, and the spatial resolution of the feature maps is reduced more aggressively in th e \ufb01rst layers to decrease the amount\nof computation. As will be shown in Sect. 4.5, our model is out performing that of Szegedyetal."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fae-6db4-11ee-b023-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-14", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "of computation. As will be shown in Sect. 4.5, our model is out performing that of Szegedyetal.\n(2014)intermsofthesingle-networkclassi\ufb01cationaccura cy.\n3 CLASSIFICATION FRAMEWORK\nIn the previous section we presented the details of our netwo rk con\ufb01gurations. In this section, we\ndescribethe detailsofclassi\ufb01cationConvNettrainingand evaluation.\n3.1 T RAINING\nThe ConvNet training procedure generally follows Krizhevs kyetal. (2012) (except for sampling\ntheinputcropsfrommulti-scaletrainingimages,asexplai nedlater). Namely,thetrainingiscarried\nout by optimising the multinomial logistic regression obje ctive using mini-batch gradient descent\n(based on back-propagation(LeCunet al., 1989)) with momen tum. The batch size was set to 256,\nmomentum to 0.9. The training was regularised by weight decay (the L2penalty multiplier set to\n5\u00b710\u22124)anddropoutregularisationforthe\ufb01rsttwofully-connect edlayers(dropoutratiosetto 0.5)."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3faf-6db4-11ee-9af8-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-15", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "5\u00b710\u22124)anddropoutregularisationforthe\ufb01rsttwofully-connect edlayers(dropoutratiosetto 0.5).\nThelearningrate wasinitially setto 10\u22122,andthendecreasedbyafactorof 10whenthevalidation\nset accuracy stopped improving. In total, the learning rate was decreased 3 times, and the learning\nwas stopped after 370K iterations (74 epochs). We conjecture that in spite of the l arger number of\nparametersandthegreaterdepthofournetscomparedto(Kri zhevskyetal.,2012),thenetsrequired\nlessepochstoconvergedueto(a)implicitregularisationi mposedbygreaterdepthandsmallerconv.\n\ufb01lter sizes; (b)pre-initialisationofcertainlayers.\nThe initialisation of the networkweightsis important,sin ce bad initialisation can stall learningdue\nto the instability of gradient in deep nets. To circumvent th is problem, we began with training\nthe con\ufb01gurationA (Table 1), shallow enoughto be trained wi th randominitialisation. Then,when"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fb0-6db4-11ee-972c-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-16", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "the con\ufb01gurationA (Table 1), shallow enoughto be trained wi th randominitialisation. Then,when\ntrainingdeeperarchitectures,weinitialisedthe\ufb01rstfou rconvolutionallayersandthelastthreefullyconnectedlayerswiththelayersofnetA(theintermediatel ayerswereinitialisedrandomly). Wedid\nnotdecreasethelearningrateforthepre-initialisedlaye rs,allowingthemtochangeduringlearning.\nFor random initialisation (where applicable), we sampled t he weights from a normal distribution\nwith thezeromeanand 10\u22122variance. The biaseswere initialisedwith zero. It isworth notingthat\nafter the paper submission we found that it is possible to ini tialise the weights without pre-training\nbyusingthe randominitialisationprocedureofGlorot&Ben gio(2010).\nToobtainthe\ufb01xed-size 224\u00d7224ConvNetinputimages,theywererandomlycroppedfromresca led\ntraining images (one crop per image per SGD iteration). To fu rther augment the training set, the\ncropsunderwentrandomhorizontal\ufb02ippingandrandomRGBco lourshift(Krizhevskyet al.,2012).\nTrainingimagerescalingisexplainedbelow."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fb1-6db4-11ee-808d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-17", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "Trainingimagerescalingisexplainedbelow.\nTraining image size. LetSbe the smallest side of an isotropically-rescaledtraining image, from\nwhich the ConvNet input is cropped (we also refer to Sas the training scale). While the crop size\nis \ufb01xed to 224\u00d7224, in principle Scan take on any value not less than 224: forS= 224the crop\nwill capture whole-image statistics, completely spanning the smallest side of a training image; for\nS\u226b224thecropwillcorrespondtoasmallpartoftheimage,contain ingasmallobjectoranobject\npart.\nWe considertwoapproachesforsettingthetrainingscale S. The\ufb01rst isto\ufb01x S,whichcorresponds\nto single-scale training (note that image content within th e sampled crops can still represent multiscale image statistics). In our experiments, we evaluated m odels trained at two \ufb01xed scales: S=\n256(which has been widely used in the prior art (Krizhevskyet al ., 2012; Zeiler&Fergus, 2013;\nSermanetet al., 2014)) and S= 384. Given a ConvNet con\ufb01guration,we \ufb01rst trained the network"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fb2-6db4-11ee-9c79-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-18", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "Sermanetet al., 2014)) and S= 384. Given a ConvNet con\ufb01guration,we \ufb01rst trained the network\nusingS= 256. To speed-up training of the S= 384network, it was initialised with the weights\npre-trainedwith S= 256,andwe useda smallerinitiallearningrateof 10\u22123.\nThe second approachto setting Sis multi-scale training, where each training image is indiv idually\nrescaled by randomly sampling Sfrom a certain range [Smin,Smax](we used Smin= 256and\nSmax= 512). Sinceobjectsinimagescanbeofdifferentsize,itisbene \ufb01cialtotakethisintoaccount\nduringtraining. Thiscanalso beseen astrainingset augmen tationbyscale jittering,wherea single\n4\nPublishedasa conferencepaperat ICLR2015\nmodel is trained to recognise objects over a wide range of sca les. For speed reasons, we trained\nmulti-scale models by \ufb01ne-tuning all layers of a single-sca le model with the same con\ufb01guration,\npre-trainedwith\ufb01xed S= 384.\n3.2 T ESTING"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fb3-6db4-11ee-8c0a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-19", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "pre-trainedwith\ufb01xed S= 384.\n3.2 T ESTING\nAttest time,givena trainedConvNetandaninputimage,itis classi\ufb01ed inthefollowingway. First,\nit is isotropically rescaled to a pre-de\ufb01ned smallest image side, denoted as Q(we also refer to it\nas the test scale). We note that Qis not necessarily equal to the training scale S(as we will show\nin Sect. 4, usingseveralvaluesof QforeachSleadsto improvedperformance). Then,the network\nis applied densely overthe rescaled test image in a way simil ar to (Sermanetet al., 2014). Namely,\nthe fully-connected layers are \ufb01rst converted to convoluti onal layers (the \ufb01rst FC layer to a 7\u00d77\nconv. layer, the last two FC layers to 1\u00d71conv. layers). The resulting fully-convolutional net is\nthen applied to the whole (uncropped) image. The result is a c lass score map with the number of\nchannels equal to the number of classes, and a variable spati al resolution, dependent on the input\nimagesize. Finally,toobtaina\ufb01xed-sizevectorofclasssc oresfortheimage,theclassscoremapis"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fb4-6db4-11ee-9f17-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-20", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "imagesize. Finally,toobtaina\ufb01xed-sizevectorofclasssc oresfortheimage,theclassscoremapis\nspatially averaged(sum-pooled). We also augmentthe test s et by horizontal\ufb02ippingof the images;\nthesoft-maxclassposteriorsoftheoriginaland\ufb02ippedima gesareaveragedtoobtainthe\ufb01nalscores\nfortheimage.\nSince the fully-convolutionalnetwork is applied over the w hole image, there is no need to sample\nmultiple crops at test time (Krizhevskyetal., 2012), which is less ef\ufb01cient as it requires network\nre-computationforeachcrop. Atthesametime,usingalarge setofcrops,asdonebySzegedyetal.\n(2014),canleadtoimprovedaccuracy,asit resultsin a\ufb01ner samplingoftheinputimagecompared\ntothefully-convolutionalnet. Also,multi-cropevaluati oniscomplementarytodenseevaluationdue\nto different convolution boundary conditions: when applyi ng a ConvNet to a crop, the convolved\nfeature mapsare paddedwith zeros, while in the case of dense evaluationthe paddingfor the same"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fb5-6db4-11ee-bac8-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-21", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "feature mapsare paddedwith zeros, while in the case of dense evaluationthe paddingfor the same\ncrop naturally comes from the neighbouring parts of an image (due to both the convolutions and\nspatial pooling), which substantially increases the overa ll network receptive \ufb01eld, so more context\niscaptured. Whilewebelievethatinpracticetheincreased computationtimeofmultiplecropsdoes\nnotjustifythepotentialgainsinaccuracy,forreferencew ealsoevaluateournetworksusing 50crops\nperscale( 5\u00d75regulargridwith 2\ufb02ips),foratotalof 150cropsover 3scales,whichiscomparable\nto144cropsover 4scalesusedbySzegedyetal. (2014).\n3.3 IMPLEMENTATION DETAILS\nOurimplementationisderivedfromthepubliclyavailableC ++ Caffetoolbox(Jia,2013)(branched\nout in December 2013), but contains a number of signi\ufb01cant mo di\ufb01cations, allowing us to perform\ntrainingandevaluationonmultipleGPUsinstalledinasing lesystem,aswellastrainandevaluateon\nfull-size (uncropped) images at multiple scales (as descri bed above). Multi-GPU training exploits"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fb6-6db4-11ee-a771-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-22", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "full-size (uncropped) images at multiple scales (as descri bed above). Multi-GPU training exploits\ndata parallelism, and is carried out by splitting each batch of training images into several GPU\nbatches, processed in parallel on each GPU. After the GPU bat ch gradientsare computed, they are\naveraged to obtain the gradient of the full batch. Gradient c omputation is synchronous across the\nGPUs, sothe resultisexactlythesame aswhentrainingona si ngleGPU.\nWhile more sophisticated methods of speeding up ConvNet tra ining have been recently proposed (Krizhevsky, 2014), which employmodeland data paral lelism for differentlayersof the net,\nwehavefoundthatourconceptuallymuchsimplerschemealre adyprovidesaspeedupof 3.75times\non an off-the-shelf4-GPU system, as comparedto using a sing le GPU. On a system equippedwith\nfourNVIDIATitanBlackGPUs,trainingasinglenettook2\u20133w eeksdependingonthearchitecture.\n4 CLASSIFICATION EXPERIMENTS\nDataset. In this section, we present the image classi\ufb01cation results achieved by the described"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fb7-6db4-11ee-86de-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-23", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "4 CLASSIFICATION EXPERIMENTS\nDataset. In this section, we present the image classi\ufb01cation results achieved by the described\nConvNetarchitecturesontheILSVRC-2012dataset(whichwa susedforILSVRC2012\u20132014challenges). The dataset includes images of 1000 classes, and is split into three sets: training ( 1.3M\nimages), validation ( 50K images), and testing ( 100K images with held-out class labels). The classi\ufb01cation performanceis evaluated using two measures: the top-1 and top-5 error. The former is a\nmulti-class classi\ufb01cation error, i.e. the proportion of in correctly classi\ufb01ed images; the latter is the\n5\nPublishedasa conferencepaperat ICLR2015\nmain evaluation criterion used in ILSVRC, and is computed as the proportion of images such that\ntheground-truthcategoryisoutsidethetop-5predictedca tegories.\nForthemajorityofexperiments,weusedthevalidationseta sthetestset. Certainexperimentswere\nalso carried out on the test set and submitted to the of\ufb01cial I LSVRC server as a \u201cVGG\u201d team entry\ntothe ILSVRC-2014competition(Russakovskyet al., 2014)."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fb8-6db4-11ee-be8b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-24", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "tothe ILSVRC-2014competition(Russakovskyet al., 2014).\n4.1 SINGLESCALEEVALUATION\nWe begin with evaluating the performanceof individual Conv Net models at a single scale with the\nlayercon\ufb01gurationsdescribedin Sect. 2.2. The test images ize was set as follows: Q=Sfor\ufb01xed\nS,andQ= 0.5(Smin+Smax)forjittered S\u2208[Smin,Smax]. Theresultsofareshownin Table3.\nFirst, we note that using local response normalisation (A-L RN network) does not improve on the\nmodel A without any normalisation layers. We thus do not empl oy normalisation in the deeper\narchitectures(B\u2013E).\nSecond, we observe that the classi\ufb01cation error decreases w ith the increased ConvNet depth: from\n11 layers in A to 19 layers in E. Notably, in spite of the same de pth, the con\ufb01guration C (which\ncontainsthree 1\u00d71conv.layers),performsworsethanthecon\ufb01gurationD,whic huses3\u00d73conv.\nlayersthroughoutthenetwork. Thisindicatesthatwhileth e additionalnon-linearitydoeshelp(Cis"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fb9-6db4-11ee-82e1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-25", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "layersthroughoutthenetwork. Thisindicatesthatwhileth e additionalnon-linearitydoeshelp(Cis\nbetter than B), it is also important to capture spatial conte xt by using conv. \ufb01lters with non-trivial\nreceptive \ufb01elds (D is better than C). The error rate of our arc hitecture saturates when the depth\nreaches19layers,butevendeepermodelsmightbebene\ufb01cialforlarger datasets. Wealsocompared\nthe net B with a shallow net with \ufb01ve 5\u00d75conv. layers, which was derived from B by replacing\neachpairof 3\u00d73conv. layerswithasingle 5\u00d75conv. layer(whichhasthesamereceptive\ufb01eldas\nexplained in Sect. 2.3). The top-1 error of the shallow net wa s measured to be 7%higher than that\nof B (on a center crop),which con\ufb01rmsthat a deepnet with smal l \ufb01lters outperformsa shallow net\nwithlarger\ufb01lters.\nFinally, scale jittering at training time ( S\u2208[256;512] ) leads to signi\ufb01cantly better results than"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fba-6db4-11ee-bc3a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-26", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "Finally, scale jittering at training time ( S\u2208[256;512] ) leads to signi\ufb01cantly better results than\ntraining on images with \ufb01xed smallest side ( S= 256orS= 384), even though a single scale is\nusedattesttime. Thiscon\ufb01rmsthattrainingsetaugmentati onbyscalejitteringisindeedhelpfulfor\ncapturingmulti-scaleimagestatistics.\nTable3:ConvNetperformanceatasingle testscale.\nConvNet con\ufb01g. (Table 1) smallest image side top-1 val.error (%) top-5 val.error (%)\ntrain(S)test (Q)\nA 256 256 29.6 10.4\nA-LRN 256 256 29.7 10.5\nB 256 256 28.7 9.9\nC256 256 28.1 9.4\n384 384 28.1 9.3\n[256;512] 384 27.3 8.8\nD256 256 27.0 8.8\n384 384 26.8 8.7\n[256;512] 384 25.6 8.1\nE256 256 27.3 9.0\n384 384 26.9 8.7\n[256;512] 384 25.5 8.0\n4.2 M ULTI-SCALEEVALUATION"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fbb-6db4-11ee-bdb0-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-27", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "384 384 26.9 8.7\n[256;512] 384 25.5 8.0\n4.2 M ULTI-SCALEEVALUATION\nHavingevaluatedtheConvNetmodelsatasinglescale,wenow assesstheeffectofscalejitteringat\ntesttime. Itconsistsofrunningamodeloverseveralrescal edversionsofatestimage(corresponding\nto different values of Q), followed by averaging the resulting class posteriors. Co nsidering that a\nlarge discrepancy between training and testing scales lead s to a drop in performance, the models\ntrained with \ufb01xed Swere evaluated over three test image sizes, close to the trai ning one: Q=\n{S\u221232,S,S+ 32}. At the same time, scale jittering at training time allows th e network to be\nappliedto a widerrangeofscales at test time,so the modeltr ainedwithvariable S\u2208[Smin;Smax]\nwasevaluatedoveralargerrangeofsizes Q={Smin,0.5(Smin+Smax),Smax}.\n6\nPublishedasa conferencepaperat ICLR2015\nTheresults,presentedinTable4,indicatethatscalejitte ringattest timeleadstobetterperformance"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fbc-6db4-11ee-81a8-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-28", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "6\nPublishedasa conferencepaperat ICLR2015\nTheresults,presentedinTable4,indicatethatscalejitte ringattest timeleadstobetterperformance\n(as compared to evaluating the same model at a single scale, s hown in Table 3). As before, the\ndeepest con\ufb01gurations(D and E) perform the best, and scale j ittering is better than training with a\n\ufb01xed smallest side S. Our best single-network performance on the validation set is24.8%/7.5%\ntop-1/top-5error(highlightedinboldinTable4). Onthete stset,thecon\ufb01gurationEachieves 7.3%\ntop-5error.\nTable4:ConvNetperformanceatmultiple test scales.\nConvNet con\ufb01g. (Table 1) smallest image side top-1val. error (%) top-5val. error (%)\ntrain(S)test(Q)\nB 256 224,256,288 28.2 9.6\nC256 224,256,288 27.7 9.2\n384 352,384,416 27.8 9.2\n[256;512] 256,384,512 26.3 8.2\nD256 224,256,288 26.6 8.6"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fbd-6db4-11ee-88f4-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-29", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "[256;512] 256,384,512 26.3 8.2\nD256 224,256,288 26.6 8.6\n384 352,384,416 26.5 8.6\n[256;512] 256,384,512 24.8 7.5\nE256 224,256,288 26.9 8.7\n384 352,384,416 26.7 8.6\n[256;512] 256,384,512 24.8 7.5\n4.3 M ULTI-CROP EVALUATION\nIn Table 5 we compare dense ConvNet evaluation with mult-cro p evaluation (see Sect. 3.2 for details). We also assess the complementarityof thetwo evalua tiontechniquesbyaveragingtheirsoftmax outputs. As can be seen, using multiple crops performs sl ightly better than dense evaluation,\nandthe two approachesareindeedcomplementary,astheir co mbinationoutperformseach ofthem.\nAs noted above, we hypothesize that this is due to a different treatment of convolution boundary\nconditions.\nTable 5:ConvNetevaluationtechniques comparison. Inall experimentsthe trainingscale Swas\nsampledfrom [256;512] ,andthreetest scales Qwereconsidered: {256,384,512}."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fbe-6db4-11ee-a31a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-30", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "sampledfrom [256;512] ,andthreetest scales Qwereconsidered: {256,384,512}.\nConvNet con\ufb01g. (Table 1) Evaluationmethod top-1 val. error(%) top-5 val. error (%)\nDdense 24.8 7.5\nmulti-crop 24.6 7.5\nmulti-crop &dense 24.4 7.2\nEdense 24.8 7.5\nmulti-crop 24.6 7.4\nmulti-crop &dense 24.4 7.1\n4.4 C ONVNETFUSION\nUpuntilnow,weevaluatedtheperformanceofindividualCon vNetmodels. Inthispartoftheexperiments,wecombinetheoutputsofseveralmodelsbyaveragin gtheirsoft-maxclassposteriors. This\nimprovesthe performancedueto complementarityof the mode ls, andwas used in the top ILSVRC\nsubmissions in 2012 (Krizhevskyet al., 2012) and 2013 (Zeil er&Fergus, 2013; Sermanetet al.,\n2014).\nThe results are shown in Table 6. By the time of ILSVRC submiss ion we had only trained the\nsingle-scale networks, as well as a multi-scale model D (by \ufb01 ne-tuning only the fully-connected"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fbf-6db4-11ee-aa1f-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-31", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "single-scale networks, as well as a multi-scale model D (by \ufb01 ne-tuning only the fully-connected\nlayers rather than all layers). The resulting ensemble of 7 n etworks has 7.3%ILSVRC test error.\nAfter the submission, we considered an ensemble of only two b est-performing multi-scale models\n(con\ufb01gurations D and E), which reduced the test error to 7.0%using dense evaluation and 6.8%\nusing combined dense and multi-crop evaluation. For refere nce, our best-performingsingle model\nachieves7.1%error(modelE, Table5).\n4.5 C OMPARISON WITH THE STATE OF THE ART\nFinally, we compare our results with the state of the art in Ta ble 7. In the classi\ufb01cation task of\nILSVRC-2014 challenge (Russakovskyet al., 2014), our \u201cVGG \u201d team secured the 2nd place with\n7\nPublishedasa conferencepaperat ICLR2015\nTable6:Multiple ConvNetfusion results.\nCombined ConvNet modelsError\ntop-1 val top-5val top-5test\nILSVRCsubmission"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fc0-6db4-11ee-830c-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-32", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "Table6:Multiple ConvNetfusion results.\nCombined ConvNet modelsError\ntop-1 val top-5val top-5test\nILSVRCsubmission\n(D/256/224,256,288), (D/384/352,384,416), (D/[256;512 ]/256,384,512)\n(C/256/224,256,288), (C/384/352,384,416)\n(E/256/224,256,288), (E/384/352,384,416)24.7 7.5 7.3\npost-submission\n(D/[256;512]/256,384,512), (E/[256;512]/256,384,512) ,dense eval. 24.0 7.1 7.0\n(D/[256;512]/256,384,512), (E/[256;512]/256,384,512) ,multi-crop 23.9 7.2 (D/[256;512]/256,384,512), (E/[256;512]/256,384,512) ,multi-crop &dense eval. 23.7 6.8 6.8\n7.3%test errorusinganensembleof7 models. Afterthesubmissio n,we decreasedtheerrorrateto"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fc1-6db4-11ee-b09a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-33", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "7.3%test errorusinganensembleof7 models. Afterthesubmissio n,we decreasedtheerrorrateto\n6.8%usinganensembleof2models.\nAs can be seen from Table 7, our very deep ConvNetssigni\ufb01cant ly outperformthe previousgeneration of models, which achieved the best results in the ILSVR C-2012 and ILSVRC-2013 competitions. Our result is also competitivewith respect to the cla ssi\ufb01cation task winner(GoogLeNetwith\n6.7%error) and substantially outperforms the ILSVRC-2013 winn ing submission Clarifai, which\nachieved 11.2%with outside training data and 11.7%without it. This is remarkable, considering\nthat our best result is achievedby combiningjust two models \u2013 signi\ufb01cantly less than used in most\nILSVRC submissions. In terms of the single-net performance , our architecture achieves the best\nresult (7.0%test error), outperforming a single GoogLeNet by 0.9%. Notably, we did not depart\nfrom the classical ConvNet architecture of LeCunetal. (198 9), but improved it by substantially\nincreasingthedepth."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fc2-6db4-11ee-8048-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-34", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "from the classical ConvNet architecture of LeCunetal. (198 9), but improved it by substantially\nincreasingthedepth.\nTable 7:Comparison with the state of the art in ILSVRC classi\ufb01cation . Our methodis denoted\nas\u201cVGG\u201d.Onlytheresultsobtainedwithoutoutsidetrainin gdataarereported.\nMethod top-1 val. error(%) top-5val. error (%) top-5testerror (%)\nVGG(2nets, multi-crop& dense eval.) 23.7 6.8 6.8\nVGG(1net, multi-crop& dense eval.) 24.4 7.1 7.0\nVGG(ILSVRCsubmission, 7nets, dense eval.) 24.7 7.5 7.3\nGoogLeNet (Szegedy et al., 2014) (1net) - 7.9\nGoogLeNet (Szegedy et al., 2014) (7nets) - 6.7\nMSRA(He et al., 2014) (11nets) - - 8.1\nMSRA(He et al., 2014) (1net) 27.9 9.1 9.1\nClarifai(Russakovsky et al., 2014) (multiplenets) - - 11.7"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fc3-6db4-11ee-9c60-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-35", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "Clarifai(Russakovsky et al., 2014) (multiplenets) - - 11.7\nClarifai(Russakovsky et al., 2014) (1net) - - 12.5\nZeiler& Fergus (Zeiler&Fergus, 2013) (6nets) 36.0 14.7 14.8\nZeiler& Fergus (Zeiler&Fergus, 2013) (1net) 37.5 16.0 16.1\nOverFeat (Sermanetet al.,2014) (7nets) 34.0 13.2 13.6\nOverFeat (Sermanetet al.,2014) (1net) 35.7 14.2 Krizhevsky et al.(Krizhevsky et al., 2012) (5nets) 38.1 16.4 16.4\nKrizhevsky et al.(Krizhevsky et al., 2012) (1net) 40.7 18.2 5 CONCLUSION\nIn this work we evaluated very deep convolutional networks ( up to 19 weight layers) for largescale image classi\ufb01cation. It was demonstrated that the rep resentation depth is bene\ufb01cial for the\nclassi\ufb01cationaccuracy,andthatstate-of-the-artperfor manceontheImageNetchallengedatasetcan"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fc4-6db4-11ee-a033-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-36", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "classi\ufb01cationaccuracy,andthatstate-of-the-artperfor manceontheImageNetchallengedatasetcan\nbeachievedusingaconventionalConvNetarchitecture(LeC unet al.,1989;Krizhevskyet al.,2012)\nwithsubstantiallyincreaseddepth. Intheappendix,weals oshowthatourmodelsgeneralisewellto\na wide range of tasks and datasets, matchingor outperformin gmore complexrecognitionpipelines\nbuiltaroundlessdeepimagerepresentations. Ourresultsy etagaincon\ufb01rmtheimportanceof depth\ninvisualrepresentations.\nACKNOWLEDGEMENTS\nThisworkwassupportedbyERCgrantVisRecno.228180. Wegra tefullyacknowledgethesupport\nofNVIDIACorporationwiththedonationoftheGPUsusedfort hisresearch.\n8\nPublishedasa conferencepaperat ICLR2015\nREFERENCES\nBell, S., Upchurch, P.,Snavely, N., and Bala, K. Material re cognition inthe wild withthe materials in context\ndatabase. CoRR,abs/1412.0623, 2014."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fc5-6db4-11ee-8aa1-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-37", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "database. CoRR,abs/1412.0623, 2014.\nChat\ufb01eld, K., Simonyan, K., Vedaldi, A., and Zisserman, A. R eturn of the devil in the details: Delving deep\nintoconvolutional nets. In Proc.BMVC. ,2014.\nCimpoi,M.,Maji,S.,andVedaldi,A. Deepconvolutional\ufb01lt erbanksfortexturerecognitionandsegmentation.\nCoRR,abs/1411.6836, 2014.\nCiresan, D. C., Meier, U., Masci, J., Gambardella, L. M., and Schmidhuber, J. Flexible, high performance\nconvolutional neural networks for image classi\ufb01cation. In IJCAI,pp. 1237\u20131242, 2011.\nDean, J., Corrado, G., Monga, R., Chen, K., Devin, M., Mao, M. , Ranzato, M., Senior, A., Tucker, P., Yang,\nK.,Le,Q. V.,andNg, A.Y. Large scale distributeddeepnetwo rks. InNIPS,pp. 1232\u20131240, 2012."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fc6-6db4-11ee-b26a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-38", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "K.,Le,Q. V.,andNg, A.Y. Large scale distributeddeepnetwo rks. InNIPS,pp. 1232\u20131240, 2012.\nDeng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei , L. Imagenet: A large-scale hierarchical image\ndatabase. In Proc.CVPR ,2009.\nDonahue,J.,Jia,Y.,Vinyals,O.,Hoffman,J.,Zhang,N.,Tz eng,E.,andDarrell,T.Decaf: Adeepconvolutional\nactivation feature for generic visual recognition. CoRR,abs/1310.1531, 2013.\nEveringham, M., Eslami, S.M. A., Van Gool, L., Williams,C., Winn, J., and Zisserman, A. The Pascal visual\nobject classes challenge: Aretrospective. IJCV,111(1):98\u2013136, 2015.\nFei-Fei, L., Fergus, R., and Perona, P. Learning generative visual models from few training examples: An\nincremental bayesian approach tested on 101 object categor ies. InIEEE CVPR Workshop of Generative\nModel BasedVision , 2004."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fc7-6db4-11ee-bfd6-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-39", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "incremental bayesian approach tested on 101 object categor ies. InIEEE CVPR Workshop of Generative\nModel BasedVision , 2004.\nGirshick, R. B., Donahue, J., Darrell, T., and Malik, J. Rich feature hierarchies for accurate object detection\nand semantic segmentation. CoRR,abs/1311.2524v5, 2014. PublishedinProc.CVPR,2014.\nGkioxari, G.,Girshick, R.,and Malik, J. Actions and attrib utes from wholes and parts. CoRR,abs/1412.2604,\n2014.\nGlorot, X. andBengio, Y. Understanding the dif\ufb01cultyof tra iningdeep feedforward neural networks. In Proc.\nAISTATS,volume 9, pp. 249\u2013256, 2010.\nGoodfellow, I. J., Bulatov, Y., Ibarz, J., Arnoud, S., and Sh et, V. Multi-digit number recognition from street\nview imagery usingdeep convolutional neural networks. In Proc.ICLR ,2014.\nGrif\ufb01n, G., Holub, A., and Perona, P. Caltech-256 object cat egory dataset. Technical Report 7694, California\nInstitute of Technology, 2007."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fc8-6db4-11ee-9b54-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-40", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "Institute of Technology, 2007.\nHe, K., Zhang, X., Ren, S., and Sun, J. Spatial pyramid poolin g in deep convolutional networks for visual\nrecognition. CoRR,abs/1406.4729v2, 2014.\nHoai, M. Regularizedmax pooling forimage categorization. InProc. BMVC. ,2014.\nHoward, A.G. Someimprovements ondeepconvolutional neura l networkbasedimageclassi\ufb01cation. In Proc.\nICLR,2014.\nJia, Y. Caffe: An open source convolutional architecture fo r fast feature embedding.\nhttp://caffe.berkeleyvision.org/ ,2013.\nKarpathy, A. and Fei-Fei, L. Deep visual-semantic alignmen ts for generating image descriptions. CoRR,\nabs/1412.2306, 2014.\nKiros, R., Salakhutdinov, R., and Zemel, R. S. Unifying visu al-semantic embeddings with multimodal neural\nlanguage models. CoRR,abs/1411.2539, 2014.\nKrizhevsky, A. One weirdtrickfor parallelizingconvoluti onal neural networks. CoRR,abs/1404.5997, 2014."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fc9-6db4-11ee-898a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-41", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "Krizhevsky, A. One weirdtrickfor parallelizingconvoluti onal neural networks. CoRR,abs/1404.5997, 2014.\nKrizhevsky, A., Sutskever, I., and Hinton, G. E. ImageNet cl assi\ufb01cation with deep convolutional neural networks. In NIPS,pp. 1106\u20131114, 2012.\nLeCun,Y.,Boser, B.,Denker, J.S.,Henderson, D.,Howard, R .E.,Hubbard, W.,andJackel, L.D. Backpropagationapplied tohandwrittenzipcode recognition. Neural Computation , 1(4):541\u2013551, 1989.\nLin,M., Chen, Q.,andYan, S. Networkinnetwork. In Proc.ICLR ,2014.\nLong, J., Shelhamer, E., and Darrell, T. Fully convolutiona l networks for semantic segmentation. CoRR,\nabs/1411.4038, 2014.\nOquab, M., Bottou, L., Laptev, I., and Sivic, J. Learning and Transferring Mid-Level Image Representations\nusing Convolutional Neural Networks. In Proc.CVPR ,2014."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fca-6db4-11ee-ba52-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-42", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "using Convolutional Neural Networks. In Proc.CVPR ,2014.\nPerronnin, F.,S\u00b4 anchez, J.,andMensink, T. Improving theF isherkernel forlarge-scale image classi\ufb01cation. In\nProc.ECCV ,2010.\nRazavian, A.,Azizpour, H.,Sullivan, J.,andCarlsson,S. C NNFeaturesoff-the-shelf: anAstounding Baseline\nfor Recognition. CoRR,abs/1403.6382, 2014.\n9\nPublishedasa conferencepaperat ICLR2015\nRussakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A.,\nBernstein, M., Berg, A. C., and Fei-Fei, L. ImageNet large sc ale visual recognition challenge. CoRR,\nabs/1409.0575, 2014.\nSermanet,P.,Eigen, D.,Zhang, X.,Mathieu, M.,Fergus,R., andLeCun,Y. OverFeat: IntegratedRecognition,\nLocalizationand Detectionusing Convolutional Networks. InProc.ICLR ,2014."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fcb-6db4-11ee-9a60-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-43", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "Localizationand Detectionusing Convolutional Networks. InProc.ICLR ,2014.\nSimonyan, K. and Zisserman, A. Two-stream convolutional ne tworks for action recognition in videos. CoRR,\nabs/1406.2199, 2014. Published inProc.NIPS,2014.\nSzegedy, C., Liu, W.,Jia, Y., Sermanet, P.,Reed, S.,Anguel ov, D.,Erhan, D., Vanhoucke, V., and Rabinovich,\nA. Goingdeeper withconvolutions. CoRR,abs/1409.4842, 2014.\nWei, Y., Xia, W., Huang, J., Ni, B., Dong, J., Zhao, Y., and Yan , S. CNN: Single-label to multi-label. CoRR,\nabs/1406.5726, 2014.\nZeiler, M. D. and Fergus, R. Visualizing and understanding c onvolutional networks. CoRR, abs/1311.2901,\n2013. PublishedinProc. ECCV,2014.\nA LOCALISATION\nIn the main bodyof the paper we have consideredthe classi\ufb01ca tion task of the ILSVRC challenge,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fcc-6db4-11ee-a4cd-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-44", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "A LOCALISATION\nIn the main bodyof the paper we have consideredthe classi\ufb01ca tion task of the ILSVRC challenge,\nand performed a thorough evaluation of ConvNet architectur es of different depth. In this section,\nwe turn to the localisation task of the challenge, which we ha ve won in 2014 with 25.3%error. It\ncan be seen as a special case of object detection, where a sing le object bounding box should be\npredictedforeach ofthe top-5classes, irrespectiveof the actual numberofobjectsof the class. For\nthiswe adoptthe approachof Sermanetet al. (2014), the winn ersof the ILSVRC-2013localisation\nchallenge,withafewmodi\ufb01cations. Ourmethodisdescribed inSect.A.1andevaluatedinSect.A.2.\nA.1 L OCALISATION CONVNET\nTo perform object localisation, we use a very deep ConvNet, w here the last fully connected layer\npredicts the bounding box location instead of the class scor es. A bounding box is represented by\na 4-D vector storing its center coordinates, width, and heig ht. There is a choice of whether the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fcd-6db4-11ee-a09b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-45", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "a 4-D vector storing its center coordinates, width, and heig ht. There is a choice of whether the\nboundingbox prediction is shared across all classes (singl e-class regression, SCR (Sermanetet al.,\n2014))orisclass-speci\ufb01c(per-classregression,PCR).In theformercase,thelastlayeris4-D,while\nin the latter it is 4000-D (since there are 1000 classes in the dataset). Apart from the last bounding\nboxpredictionlayer,weuse theConvNetarchitectureD (Tab le1),whichcontains16weightlayers\nandwasfoundtobe thebest-performingin theclassi\ufb01cation task (Sect.4).\nTraining. Training of localisation ConvNets is similar to that of the c lassi\ufb01cation ConvNets\n(Sect.3.1). Themaindifferenceisthatwereplacethelogis ticregressionobjectivewithaEuclidean\nloss,whichpenalisesthedeviationofthepredictedboundi ngboxparametersfromtheground-truth.\nWe trainedtwo localisation models, each on a single scale: S= 256andS= 384(due to the time"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fce-6db4-11ee-9d15-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-46", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "We trainedtwo localisation models, each on a single scale: S= 256andS= 384(due to the time\nconstraints,we didnot use trainingscale jitteringforour ILSVRC-2014submission). Trainingwas\ninitialised with the correspondingclassi\ufb01cation models ( trained on the same scales), and the initial\nlearning rate was set to 10\u22123. We exploredboth \ufb01ne-tuningall layers and \ufb01ne-tuningonly the \ufb01rst\ntwo fully-connected layers, as done in (Sermanetetal., 201 4). The last fully-connected layer was\ninitialisedrandomlyandtrainedfromscratch.\nTesting. We consider two testing protocols. The \ufb01rst is used for compa ring different network\nmodi\ufb01cations on the validation set, and considers only the b oundingbox prediction for the ground\ntruth class (to factor out the classi\ufb01cation errors). The bo unding box is obtained by applying the\nnetworkonlyto thecentralcropoftheimage.\nThe second, fully-\ufb02edged, testing procedure is based on the dense application of the localisation\nConvNet to the whole image, similarly to the classi\ufb01cation t ask (Sect. 3.2). The difference is that"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fcf-6db4-11ee-bc59-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-47", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "ConvNet to the whole image, similarly to the classi\ufb01cation t ask (Sect. 3.2). The difference is that\ninstead of the class score map, the output of the last fully-c onnected layer is a set of bounding\nbox predictions. To come up with the \ufb01nal prediction, we util ise the greedy merging procedure\nof Sermanetetal. (2014), which \ufb01rst merges spatially close predictions (by averaging their coordinates), and then rates them based on the class scores, obta ined from the classi\ufb01cation ConvNet.\nWhen several localisation ConvNets are used, we \ufb01rst take th e union of their sets of boundingbox\npredictions, and then run the mergingprocedureon the union . We did not use the multiple pooling\n10\nPublishedasa conferencepaperat ICLR2015\noffsets technique of Sermanetetal. (2014), which increase s the spatial resolution of the bounding\nboxpredictionsandcanfurtherimprovetheresults.\nA.2 L OCALISATION EXPERIMENTS"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fd0-6db4-11ee-8108-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-48", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "boxpredictionsandcanfurtherimprovetheresults.\nA.2 L OCALISATION EXPERIMENTS\nIn this section we \ufb01rst determine the best-performinglocal isation setting (using the \ufb01rst test protocol), and then evaluate it in a fully-\ufb02edged scenario (the se cond protocol). The localisation error\nis measured according to the ILSVRC criterion (Russakovsky et al., 2014), i.e. the bounding box\npredictionis deemed correctif its intersectionoverunion ratio with the ground-truthboundingbox\nisabove0.5.\nSettings comparison. As can be seen from Table 8, per-class regression (PCR) outpe rforms the\nclass-agnostic single-class regression (SCR), which diff ers from the \ufb01ndings of Sermanetetal.\n(2014), where PCR was outperformed by SCR. We also note that \ufb01 ne-tuning all layers for the localisation task leads to noticeablybetter results than \ufb01ne -tuningonly the fully-connectedlayers(as\ndonein(Sermanetet al.,2014)). Intheseexperiments,thes mallestimagessidewassetto S= 384;\ntheresultswith S= 256exhibitthesamebehaviourandarenotshownforbrevity."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fd1-6db4-11ee-b211-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-49", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "theresultswith S= 256exhibitthesamebehaviourandarenotshownforbrevity.\nTable 8:Localisation error for different modi\ufb01cations with the simpli\ufb01ed testing protocol: the\nboundingbox is predictedfrom a single central image crop, a nd the ground-truthclass is used. All\nConvNet layers (except for the last one) have the con\ufb01gurati on D (Table 1), while the last layer\nperformseithersingle-classregression(SCR) orper-clas sregression(PCR).\nFine-tunedlayers regression type GTclass localisationerror\n1st and2nd FCSCR 36.4\nPCR 34.3\nall PCR 33.1\nFully-\ufb02edgedevaluation. Havingdeterminedthebestlocalisationsetting(PCR,\ufb01ne- tuningofall\nlayers),we nowapply it in the fully-\ufb02edgedscenario,where the top-5class labelsare predictedusing our best-performingclassi\ufb01cation system (Sect. 4.5), and multiple densely-computedbounding"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fd2-6db4-11ee-83b5-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-50", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "box predictions are merged using the method of Sermanetetal . (2014). As can be seen from Table 9, applicationof the localisationConvNetto the whole i magesubstantiallyimprovesthe results\ncompared to using a center crop (Table 8), despite using the t op-5 predicted class labels instead of\nthegroundtruth. Similarlytotheclassi\ufb01cationtask(Sect .4),testingatseveralscalesandcombining\nthepredictionsofmultiplenetworksfurtherimprovesthep erformance.\nTable9:Localisationerror\nsmallestimage side top-5localisationerror (%)\ntrain(S) test(Q) val. test.\n256 256 29.5 384 384 28.2 26.7\n384 352,384 27.5 fusion: 256/256 and 384/352,384 26.9 25.3\nComparison with the state of the art. We compare our best localisation result with the state\nof the art in Table 10. With 25.3%test error, our \u201cVGG\u201d team won the localisation challenge of\nILSVRC-2014 (Russakovskyet al., 2014). Notably, our resul ts are considerably better than those\nof the ILSVRC-2013winnerOverfeat(Sermanetet al., 2014), even thoughwe used less scales and"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fd3-6db4-11ee-8337-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-51", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "of the ILSVRC-2013winnerOverfeat(Sermanetet al., 2014), even thoughwe used less scales and\ndid not employ their resolution enhancement technique. We e nvisage that better localisation performance can be achieved if this technique is incorporated i nto our method. This indicates the\nperformanceadvancementbroughtbyourverydeepConvNets\u2013 wegotbetterresultswithasimpler\nlocalisationmethod,buta morepowerfulrepresentation.\nB GENERALISATION OF VERYDEEPFEATURES\nIn the previous sections we have discussed training and eval uation of very deep ConvNets on the\nILSVRC dataset. In this section, we evaluate our ConvNets, p re-trained on ILSVRC, as feature\n11\nPublishedasa conferencepaperat ICLR2015\nTable 10: Comparison with the state of the art in ILSVRC localisation . Our methodis denoted\nas\u201cVGG\u201d.\nMethod top-5val. error (%) top-5 testerror (%)\nVGG 26.9 25.3\nGoogLeNet (Szegedyet al., 2014) - 26.7\nOverFeat (Sermanet etal.,2014) 30.0 29.9\nKrizhevsky et al.(Krizhevsky et al.,2012) - 34.2"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fd4-6db4-11ee-8c76-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-52", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "Krizhevsky et al.(Krizhevsky et al.,2012) - 34.2\nextractors on other, smaller, datasets, where training lar ge models from scratch is not feasible due\nto over-\ufb01tting. Recently, there has been a lot of interest in such a use case (Zeiler&Fergus, 2013;\nDonahueet al., 2013; Razavianet al., 2014; Chat\ufb01eldet al., 2014), as it turns out that deep image\nrepresentations,learntonILSVRC,generalisewelltoothe rdatasets,wheretheyhaveoutperformed\nhand-crafted representations by a large margin. Following that line of work, we investigate if our\nmodelsleadtobetterperformancethanmoreshallowmodelsu tilisedinthestate-of-the-artmethods.\nIn this evaluation, we consider two models with the best clas si\ufb01cation performance on ILSVRC\n(Sect.4)\u2013con\ufb01gurations\u201cNet-D\u201dand\u201cNet-E\u201d(whichwemade publiclyavailable).\nTo utilise the ConvNets, pre-trained on ILSVRC, for image cl assi\ufb01cation on other datasets, we"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fd5-6db4-11ee-969b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-53", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "To utilise the ConvNets, pre-trained on ILSVRC, for image cl assi\ufb01cation on other datasets, we\nremove the last fully-connected layer (which performs 1000 -way ILSVRC classi\ufb01cation), and use\n4096-Dactivationsofthepenultimatelayerasimagefeatur es,whichareaggregatedacrossmultiple\nlocations and scales. The resulting image descriptor is L2-normalised and combined with a linear\nSVM classi\ufb01er, trained on the target dataset. For simplicit y, pre-trained ConvNet weights are kept\n\ufb01xed(no\ufb01ne-tuningisperformed).\nAggregation of features is carried out in a similar manner to our ILSVRC evaluation procedure\n(Sect. 3.2). Namely, an image is \ufb01rst rescaled so that its sma llest side equals Q, and then the network is densely applied over the image plane (which is possib le when all weight layers are treated\nas convolutional). We then perform global average pooling o n the resulting feature map, which\nproducesa 4096-Dimage descriptor. The descriptor is then a veraged with the descriptor of a horizontally \ufb02ipped image. As was shown in Sect. 4.2, evaluation over multiple scales is bene\ufb01cial, so"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fd6-6db4-11ee-aded-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-54", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "we extract features over several scales Q. The resulting multi-scale features can be either stacked\nor pooled across scales. Stacking allows a subsequent class i\ufb01er to learn how to optimally combine\nimage statistics over a range of scales; this, however, come s at the cost of the increased descriptor\ndimensionality. We returntothediscussionofthisdesignc hoicein theexperimentsbelow. We also\nassess late fusion of features, computed using two networks , which is performed by stacking their\nrespectiveimagedescriptors.\nTable11: Comparisonwiththestateoftheartinimageclassi\ufb01cationo nVOC-2007,VOC-2012,\nCaltech-101, and Caltech-256 . Our models are denoted as \u201cVGG\u201d. Results marked with * were\nachievedusingConvNetspre-trainedonthe extended ILSVRCdataset(2000classes).\nMethodVOC-2007 VOC-2012 Caltech-101 Caltech-256\n(meanAP) (mean AP) (meanclass recall) (mean class recall)\nZeiler& Fergus (Zeiler&Fergus, 2013) - 79.0 86.5\u00b10.5 74.2\u00b10.3"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fd7-6db4-11ee-9f95-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-55", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "Zeiler& Fergus (Zeiler&Fergus, 2013) - 79.0 86.5\u00b10.5 74.2\u00b10.3\nChat\ufb01eldetal. (Chat\ufb01eldet al., 2014) 82.4 83.2 88.4\u00b10.6 77.6\u00b10.1\nHe etal. (Heet al.,2014) 82.4 - 93.4\u00b10.5 Weiet al.(Weiet al., 2014) 81.5(85.2\u2217)81.7 (90.3\u2217) - VGGNet-D (16layers) 89.3 89.0 91.8\u00b11.0 85.0\u00b10.2\nVGGNet-E(19 layers) 89.3 89.0 92.3\u00b10.5 85.1\u00b10.3\nVGGNet-D & Net-E 89.7 89.3 92.7\u00b10.5 86.2\u00b10.3\nImage Classi\ufb01cation on VOC-2007and VOC-2012. We beginwith the evaluationon the image\nclassi\ufb01cation task of PASCAL VOC-2007 and VOC-2012 benchma rks (Everinghametal., 2015)."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fd8-6db4-11ee-bc11-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-56", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "classi\ufb01cation task of PASCAL VOC-2007 and VOC-2012 benchma rks (Everinghametal., 2015).\nThese datasets contain 10K and 22.5K images respectively, a nd each image is annotated with one\nor several labels, correspondingto 20 object categories. T he VOC organisersprovidea pre-de\ufb01ned\nsplit into training, validation, and test data (the test dat a for VOC-2012 is not publicly available;\ninstead,anof\ufb01cialevaluationserverisprovided). Recogn itionperformanceismeasuredusingmean\naverageprecision(mAP)acrossclasses.\nNotably, by examining the performance on the validation set s of VOC-2007 and VOC-2012, we\nfoundthat aggregatingimage descriptors,computedat mult iple scales, by averagingperformssim12\nPublishedasa conferencepaperat ICLR2015\nilarly to the aggregation by stacking. We hypothesize that t his is due to the fact that in the VOC"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fd9-6db4-11ee-b24d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-57", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "Publishedasa conferencepaperat ICLR2015\nilarly to the aggregation by stacking. We hypothesize that t his is due to the fact that in the VOC\ndataset the objects appear over a variety of scales, so there is no particular scale-speci\ufb01c semantics which a classi\ufb01er could exploit. Since averaging has a b ene\ufb01t of not in\ufb02ating the descriptor dimensionality, we were able to aggregated image descri ptors over a wide range of scales:\nQ\u2208 {256,384,512,640,768}. It is worth noting though that the improvement over a smalle r\nrangeof{256,384,512}wasrathermarginal( 0.3%).\nThetestsetperformanceisreportedandcomparedwithother approachesinTable11. Ournetworks\n\u201cNet-D\u201dand\u201cNet-E\u201dexhibitidenticalperformanceonVOCda tasets,andtheircombinationslightly\nimproves the results. Our methods set the new state of the art across image representations, pretrained on the ILSVRC dataset, outperformingthe previousb est result of Chat\ufb01eldet al. (2014) by\nmore than 6%. It should be noted that the method of Wei et al. (2014), which achieves1%better"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fda-6db4-11ee-9b7b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-58", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "more than 6%. It should be noted that the method of Wei et al. (2014), which achieves1%better\nmAP on VOC-2012, is pre-trained on an extended 2000-class IL SVRC dataset, which includes\nadditional 1000 categories, semantically close to those in VOC datasets. It also bene\ufb01ts from the\nfusionwith anobjectdetection-assistedclassi\ufb01cation pi peline.\nImageClassi\ufb01cationonCaltech-101andCaltech-256. InthissectionweevaluateverydeepfeaturesonCaltech-101(Fei-Feiet al.,2004)andCaltech-256 (Grif\ufb01net al.,2007)imageclassi\ufb01cation\nbenchmarks. Caltech-101contains9Kimageslabelledinto1 02classes(101objectcategoriesanda\nbackgroundclass), while Caltech-256 is larger with 31K ima ges and 257 classes. A standard evaluation protocolon these datasets is to generateseveral ran domsplits into training and test data and\nreport the average recognition performance across the spli ts, which is measured by the mean class\nrecall(whichcompensatesforadifferentnumberoftestima gesperclass). FollowingChat\ufb01eld etal."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fdb-6db4-11ee-95f6-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-59", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "recall(whichcompensatesforadifferentnumberoftestima gesperclass). FollowingChat\ufb01eld etal.\n(2014); Zeiler&Fergus(2013); He etal. (2014),onCaltech- 101we generated3 randomsplits into\ntraining and test data, so that each split contains 30 traini ng images per class, and up to 50 test\nimages per class. On Caltech-256 we also generated 3 splits, each of which contains 60 training\nimages per class (and the rest is used for testing). In each sp lit, 20% of training images were used\nasa validationset forhyper-parameterselection.\nWe found that unlike VOC, on Caltech datasets the stacking of descriptors, computed over multiple scales, performs better than averaging or max-pooling. This can be explained by the fact that\nin Caltech images objects typically occupy the whole image, so multi-scale image features are semanticallydifferent(capturingthe wholeobject vs. object parts), andstacking allows a classi\ufb01er to\nexploitsuchscale-speci\ufb01crepresentations. We usedthree scalesQ\u2208 {256,384,512}.\nOurmodelsarecomparedtoeachotherandthestateofthearti nTable11. Ascanbeseen,thedeeper"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fdc-6db4-11ee-868a-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-60", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "Ourmodelsarecomparedtoeachotherandthestateofthearti nTable11. Ascanbeseen,thedeeper\n19-layerNet-Eperformsbetterthanthe16-layerNet-D,and theircombinationfurtherimprovesthe\nperformance. On Caltech-101, our representations are comp etitive with the approach of He etal.\n(2014),which,however,performssigni\ufb01cantlyworsethano urnetsonVOC-2007. OnCaltech-256,\nourfeaturesoutperformthestate oftheart (Chat\ufb01eldetal. , 2014)byalargemargin( 8.6%).\nAction Classi\ufb01cation on VOC-2012. We also evaluated our best-performing image representation (the stacking of Net-D and Net-E features) on the PASCAL VOC-2012 action classi\ufb01cation\ntask (Everinghamet al., 2015), which consists in predictin g an action class from a single image,\ngiven a bounding box of the person performing the action. The dataset contains 4.6K training images,labelledinto11classes. SimilarlytotheVOC-2012ob jectclassi\ufb01cationtask,theperformance"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fdd-6db4-11ee-aea0-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-61", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "is measured using the mAP. We considered two training settin gs: (i) computing the ConvNet featuresonthewholeimageandignoringtheprovidedboundingb ox;(ii)computingthefeaturesonthe\nwholeimageandontheprovidedboundingbox,andstackingth emtoobtainthe\ufb01nalrepresentation.\nTheresultsarecomparedtootherapproachesinTable12.\nOurrepresentationachievesthestateofartontheVOCactio nclassi\ufb01cationtaskevenwithoutusing\nthe provided bounding boxes, and the results are further imp roved when using both images and\nbounding boxes. Unlike other approaches, we did not incorpo rate any task-speci\ufb01c heuristics, but\nreliedontherepresentationpowerofverydeepconvolution alfeatures.\nOther Recognition Tasks. Since the public release of our models, they have been active ly used\nby the research community for a wide range of image recogniti on tasks, consistently outperforming more shallow representations. For instance, Girshicke t al. (2014) achieve the state of the\nobject detection results by replacing the ConvNet of Krizhe vskyet al. (2012) with our 16-layer"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fde-6db4-11ee-b689-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-62", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "object detection results by replacing the ConvNet of Krizhe vskyet al. (2012) with our 16-layer\nmodel. Similar gains over a more shallow architecture of Kri zhevskyet al. (2012) have been ob13\nPublishedasa conferencepaperat ICLR2015\nTable 12: Comparison with the state of the art in single-image action c lassi\ufb01cation on VOC2012. Our models are denoted as \u201cVGG\u201d. Results marked with * were a chieved using ConvNets\npre-trainedonthe extended ILSVRCdataset (1512classes).\nMethod VOC-2012 (meanAP)\n(Oquab et al., 2014) 70.2\u2217\n(Gkioxari etal.,2014) 73.6\n(Hoai,2014) 76.3\nVGG Net-D& Net-E,image-only 79.2\nVGG Net-D& Net-E,image and bounding box 84.0\nserved in semantic segmentation (Longet al., 2014), image c aption generation (Kirosetal., 2014;\nKarpathy& Fei-Fei, 2014),textureandmaterialrecognitio n(Cimpoiet al., 2014; Bell etal., 2014).\nC PAPERREVISIONS"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fdf-6db4-11ee-bc06-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1409.1556-63", "title": "Very Deep Convolutional Networks for Large-Scale Image Recognition", "source": "http://arxiv.org/pdf/1409.1556", "start_index": 0, "chroma:document": "C PAPERREVISIONS\nHere we present the list of major paper revisions, outlining the substantial changes for the convenienceofthe reader.\nv1Initialversion. Presentstheexperimentscarriedoutbefo rethe ILSVRCsubmission.\nv2Addspost-submissionILSVRCexperimentswithtrainingset augmentationusingscalejittering,\nwhichimprovestheperformance.\nv3Addsgeneralisationexperiments(AppendixB) on PASCAL VOC andCaltech image classi\ufb01cationdatasets. Themodelsusedforthese experimentsarepub liclyavailable.\nv4The paper is converted to ICLR-2015 submission format. Also adds experiments with multiple\ncropsforclassi\ufb01cation.\nv6Camera-readyICLR-2015conferencepaper. Addsa compariso nof the net B with a shallow net\nandtheresultsonPASCAL VOCactionclassi\ufb01cationbenchmar k.\n14"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fe0-6db4-11ee-8f61-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-0", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "Fully Convolutional Networks for Semantic Segmentation\nJonathan Long\u0003Evan Shelhamer\u0003Trevor Darrell\nUC Berkeley\nfjonlong,shelhamer,trevor g@cs.berkeley.edu\nAbstract\nConvolutional networks are powerful visual models that\nyield hierarchies of features. We show that convolutional networks by themselves, trained end-to-end, pixelsto-pixels, exceed the state-of-the-art in semantic segmentation. Our key insight is to build \u201cfully convolutional\u201d\nnetworks that take input of arbitrary size and produce\ncorrespondingly-sized output with ef\ufb01cient inference and\nlearning. We de\ufb01ne and detail the space of fully convolutional networks, explain their application to spatially dense\nprediction tasks, and draw connections to prior models. We\nadapt contemporary classi\ufb01cation networks (AlexNet [19],\nthe VGG net [31], and GoogLeNet [32]) into fully convolutional networks and transfer their learned representations\nby \ufb01ne-tuning [4] to the segmentation task. We then de\ufb01ne a novel architecture that combines semantic information from a deep, coarse layer with appearance information\nfrom a shallow, \ufb01ne layer to produce accurate and detailed"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fe1-6db4-11ee-b273-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-1", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "from a shallow, \ufb01ne layer to produce accurate and detailed\nsegmentations. Our fully convolutional network achieves\nstate-of-the-art segmentation of PASCAL VOC (20% relative improvement to 62.2% mean IU on 2012), NYUDv2,\nand SIFT Flow, while inference takes less than one \ufb01fth of a\nsecond for a typical image.\n1. Introduction\nConvolutional networks are driving advances in recognition. Convnets are not only improving for whole-image\nclassi\ufb01cation [19, 31, 32], but also making progress on local tasks with structured output. These include advances in\nbounding box object detection [29, 12, 17], part and keypoint prediction [39, 24], and local correspondence [24, 9].\nThe natural next step in the progression from coarse to\n\ufb01ne inference is to make a prediction at every pixel. Prior\napproaches have used convnets for semantic segmentation\n[27, 2, 8, 28, 16, 14, 11], in which each pixel is labeled with\nthe class of its enclosing object or region, but with shortcomings that this work addresses.\n\u0003Authors contributed equally\n963842564096409621\n21backward/learningforward/inference\npixelwise predictionsegmentation g.t."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fe2-6db4-11ee-8387-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-2", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "\u0003Authors contributed equally\n963842564096409621\n21backward/learningforward/inference\npixelwise predictionsegmentation g.t.\n256384Figure 1. Fully convolutional networks can ef\ufb01ciently learn to\nmake dense predictions for per-pixel tasks like semantic segmentation.\nWe show that a fully convolutional network (FCN),\ntrained end-to-end, pixels-to-pixels on semantic segmentation exceeds the state-of-the-art without further machinery. To our knowledge, this is the \ufb01rst work to train FCNs\nend-to-end (1) for pixelwise prediction and (2) from supervised pre-training. Fully convolutional versions of existing\nnetworks predict dense outputs from arbitrary-sized inputs.\nBoth learning and inference are performed whole-image-ata-time by dense feedforward computation and backpropagation. In-network upsampling layers enable pixelwise prediction and learning in nets with subsampled pooling.\nThis method is ef\ufb01cient, both asymptotically and absolutely, and precludes the need for the complications in other\nworks. Patchwise training is common [27, 2, 8, 28, 11], but"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fe3-6db4-11ee-bd70-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-3", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "works. Patchwise training is common [27, 2, 8, 28, 11], but\nlacks the ef\ufb01ciency of fully convolutional training. Our approach does not make use of pre- and post-processing complications, including superpixels [8, 16], proposals [16, 14],\nor post-hoc re\ufb01nement by random \ufb01elds or local classi\ufb01ers\n[8, 16]. Our model transfers recent success in classi\ufb01cation [19, 31, 32] to dense prediction by reinterpreting classi\ufb01cation nets as fully convolutional and \ufb01ne-tuning from\ntheir learned representations. In contrast, previous works\nhave applied small convnets without supervised pre-training\n[8, 28, 27].\nSemantic segmentation faces an inherent tension between semantics and location: global information resolves\nwhat while local information resolves where. Deep feature\n1arXiv:1411.4038v2  [cs.CV]  8 Mar 2015\nhierarchies jointly encode location and semantics in a localto-global pyramid. We de\ufb01ne a novel \u201cskip\u201d architecture\nto combine deep, coarse, semantic information and shallow,\n\ufb01ne, appearance information in Section 4.2 (see Figure 3)."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fe4-6db4-11ee-9cb6-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-4", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "to combine deep, coarse, semantic information and shallow,\n\ufb01ne, appearance information in Section 4.2 (see Figure 3).\nIn the next section, we review related work on deep classi\ufb01cation nets, FCNs, and recent approaches to semantic\nsegmentation using convnets. The following sections explain FCN design and dense prediction tradeoffs, introduce\nour architecture with in-network upsampling and multilayer combinations, and describe our experimental framework. Finally, we demonstrate state-of-the-art results on\nPASCAL VOC 2011-2, NYUDv2, and SIFT Flow.\n2. Related work\nOur approach draws on recent successes of deep nets\nfor image classi\ufb01cation [19, 31, 32] and transfer learning\n[4, 38]. Transfer was \ufb01rst demonstrated on various visual\nrecognition tasks [4, 38], then on detection, and on both\ninstance and semantic segmentation in hybrid proposalclassi\ufb01er models [12, 16, 14]. We now re-architect and \ufb01netune classi\ufb01cation nets to direct, dense prediction of semantic segmentation. We chart the space of FCNs and situate\nprior models, both historical and recent, in this framework.\nFully convolutional networks To our knowledge, the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fe5-6db4-11ee-b684-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-5", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "prior models, both historical and recent, in this framework.\nFully convolutional networks To our knowledge, the\nidea of extending a convnet to arbitrary-sized inputs \ufb01rst\nappeared in Matan et al. [25], which extended the classic\nLeNet [21] to recognize strings of digits. Because their net\nwas limited to one-dimensional input strings, Matan et al.\nused Viterbi decoding to obtain their outputs. Wolf and Platt\n[37] expand convnet outputs to 2-dimensional maps of detection scores for the four corners of postal address blocks.\nBoth of these historical works do inference and learning\nfully convolutionally for detection. Ning et al. [27] de\ufb01ne\na convnet for coarse multiclass segmentation of C. elegans\ntissues with fully convolutional inference.\nFully convolutional computation has also been exploited\nin the present era of many-layered nets. Sliding window\ndetection by Sermanet et al . [29], semantic segmentation\nby Pinheiro and Collobert [28], and image restoration by\nEigen et al. [5] do fully convolutional inference. Fully convolutional training is rare, but used effectively by Tompson\net al. [35] to learn an end-to-end part detector and spatial"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fe6-6db4-11ee-8d81-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-6", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "et al. [35] to learn an end-to-end part detector and spatial\nmodel for pose estimation, although they do not exposit on\nor analyze this method.\nAlternatively, He et al . [17] discard the nonconvolutional portion of classi\ufb01cation nets to make a\nfeature extractor. They combine proposals and spatial\npyramid pooling to yield a localized, \ufb01xed-length feature\nfor classi\ufb01cation. While fast and effective, this hybrid\nmodel cannot be learned end-to-end.\nDense prediction with convnets Several recent works\nhave applied convnets to dense prediction problems, including semantic segmentation by Ning et al. [27], Farabet et al.[8], and Pinheiro and Collobert [28]; boundary prediction\nfor electron microscopy by Ciresan et al. [2] and for natural\nimages by a hybrid neural net/nearest neighbor model by\nGanin and Lempitsky [11]; and image restoration and depth\nestimation by Eigen et al. [5, 6]. Common elements of these\napproaches include\n\u000fsmall models restricting capacity and receptive \ufb01elds;\n\u000fpatchwise training [27, 2, 8, 28, 11];\n\u000fpost-processing by superpixel projection, random \ufb01eld"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fe7-6db4-11ee-9143-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-7", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "\u000fpatchwise training [27, 2, 8, 28, 11];\n\u000fpost-processing by superpixel projection, random \ufb01eld\nregularization, \ufb01ltering, or local classi\ufb01cation [8, 2,\n11];\n\u000finput shifting and output interlacing for dense output\n[28, 11] as introduced by OverFeat [29];\n\u000fmulti-scale pyramid processing [8, 28, 11];\n\u000fsaturating tanh nonlinearities [8, 5, 28]; and\n\u000fensembles [2, 11],\nwhereas our method does without this machinery. However,\nwe do study patchwise training 3.4 and \u201cshift-and-stitch\u201d\ndense output 3.2 from the perspective of FCNs. We also\ndiscuss in-network upsampling 3.3, of which the fully connected prediction by Eigen et al. [6] is a special case.\nUnlike these existing methods, we adapt and extend deep\nclassi\ufb01cation architectures, using image classi\ufb01cation as supervised pre-training, and \ufb01ne-tune fully convolutionally to\nlearn simply and ef\ufb01ciently from whole image inputs and\nwhole image ground thruths."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fe8-6db4-11ee-817d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-8", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "learn simply and ef\ufb01ciently from whole image inputs and\nwhole image ground thruths.\nHariharan et al. [16] and Gupta et al. [14] likewise adapt\ndeep classi\ufb01cation nets to semantic segmentation, but do\nso in hybrid proposal-classi\ufb01er models. These approaches\n\ufb01ne-tune an R-CNN system [12] by sampling bounding\nboxes and/or region proposals for detection, semantic segmentation, and instance segmentation. Neither method is\nlearned end-to-end.\nThey achieve state-of-the-art results on PASCAL VOC\nsegmentation and NYUDv2 segmentation respectively, so\nwe directly compare our standalone, end-to-end FCN to\ntheir semantic segmentation results in Section 5.\n3. Fully convolutional networks\nEach layer of data in a convnet is a three-dimensional\narray of size h\u0002w\u0002d, wherehandware spatial dimensions, anddis the feature or channel dimension. The \ufb01rst\nlayer is the image, with pixel size h\u0002w, anddcolor channels. Locations in higher layers correspond to the locations\nin the image they are path-connected to, which are called\ntheir receptive \ufb01elds ."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fe9-6db4-11ee-939b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-9", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "in the image they are path-connected to, which are called\ntheir receptive \ufb01elds .\nConvnets are built on translation invariance. Their basic components (convolution, pooling, and activation functions) operate on local input regions, and depend only on\nrelative spatial coordinates. Writing xijfor the data vector\nat location (i;j)in a particular layer, and yijfor the following layer, these functions compute outputs yijby\nyij=fks(fxsi+\u000ei;sj +\u000ejg0\u0014\u000ei;\u000ej\u0014k)\nwherekis called the kernel size, sis the stride or subsampling factor, and fksdetermines the layer type: a matrix\nmultiplication for convolution or average pooling, a spatial\nmax for max pooling, or an elementwise nonlinearity for an\nactivation function, and so on for other types of layers.\nThis functional form is maintained under composition,\nwith kernel size and stride obeying the transformation rule\nfks\u000egk0s0= (f\u000eg)k0+(k\u00001)s0;ss0:\nWhile a general deep net computes a general nonlinear\nfunction, a net with only layers of this form computes a"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fea-6db4-11ee-81c9-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-10", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "While a general deep net computes a general nonlinear\nfunction, a net with only layers of this form computes a\nnonlinear \ufb01lter , which we call a deep \ufb01lter orfully convolutional network . An FCN naturally operates on an input of\nany size, and produces an output of corresponding (possibly\nresampled) spatial dimensions.\nA real-valued loss function composed with an FCN de\ufb01nes a task. If the loss function is a sum over the spatial\ndimensions of the \ufb01nal layer, `(x;\u0012) =P\nij`0(xij;\u0012), its\ngradient will be a sum over the gradients of each of its spatial components. Thus stochastic gradient descent on `computed on whole images will be the same as stochastic gradient descent on `0, taking all of the \ufb01nal layer receptive \ufb01elds\nas a minibatch.\nWhen these receptive \ufb01elds overlap signi\ufb01cantly, both\nfeedforward computation and backpropagation are much\nmore ef\ufb01cient when computed layer-by-layer over an entire\nimage instead of independently patch-by-patch.\nWe next explain how to convert classi\ufb01cation nets into\nfully convolutional nets that produce coarse output maps."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3feb-6db4-11ee-9491-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-11", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "image instead of independently patch-by-patch.\nWe next explain how to convert classi\ufb01cation nets into\nfully convolutional nets that produce coarse output maps.\nFor pixelwise prediction, we need to connect these coarse\noutputs back to the pixels. Section 3.2 describes a trick that\nOverFeat [29] introduced for this purpose. We gain insight\ninto this trick by reinterpreting it as an equivalent network\nmodi\ufb01cation. As an ef\ufb01cient, effective alternative, we introduce deconvolution layers for upsampling in Section 3.3.\nIn Section 3.4 we consider training by patchwise sampling,\nand give evidence in Section 4.3 that our whole image training is faster and equally effective.\n3.1. Adapting classi\ufb01ers for dense prediction\nTypical recognition nets, including LeNet [21], AlexNet\n[19], and its deeper successors [31, 32], ostensibly take\n\ufb01xed-sized inputs and produce nonspatial outputs. The fully\nconnected layers of these nets have \ufb01xed dimensions and\nthrow away spatial coordinates. However, these fully connected layers can also be viewed as convolutions with kernels that cover their entire input regions. Doing so casts\nthem into fully convolutional networks that take input of\nany size and output classi\ufb01cation maps. This transformation"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fec-6db4-11ee-bbdb-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-12", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "them into fully convolutional networks that take input of\nany size and output classi\ufb01cation maps. This transformation\n`tabby cat\" `\n96256384384256409640961000\n96384256409640961000\n256384tabby cat heatmapconvolutionalizationFigure 2. Transforming fully connected layers into convolution\nlayers enables a classi\ufb01cation net to output a heatmap. Adding\nlayers and a spatial loss (as in Figure 1) produces an ef\ufb01cient machine for end-to-end dense learning.\nis illustrated in Figure 2. (By contrast, nonconvolutional\nnets, such as the one by Le et al. [20], lack this capability.)\nFurthermore, while the resulting maps are equivalent to\nthe evaluation of the original net on particular input patches,\nthe computation is highly amortized over the overlapping\nregions of those patches. For example, while AlexNet takes\n1:2ms (on a typical GPU) to produce the classi\ufb01cation\nscores of a 227\u0002227image, the fully convolutional version takes 22ms to produce a 10\u000210grid of outputs from\na500\u0002500image, which is more than 5times faster than\nthe na \u00a8\u0131ve approach1."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fed-6db4-11ee-a6fe-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-13", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "a500\u0002500image, which is more than 5times faster than\nthe na \u00a8\u0131ve approach1.\nThe spatial output maps of these convolutionalized models make them a natural choice for dense problems like semantic segmentation. With ground truth available at every output cell, both the forward and backward passes are\nstraightforward, and both take advantage of the inherent\ncomputational ef\ufb01ciency (and aggressive optimization) of\nconvolution.\nThe corresponding backward times for the AlexNet example are 2:4ms for a single image and 37ms for a fully\nconvolutional 10\u000210output map, resulting in a speedup\nsimilar to that of the forward pass. This dense backpropagation is illustrated in Figure 1.\nWhile our reinterpretation of classi\ufb01cation nets as fully\nconvolutional yields output maps for inputs of any size, the\noutput dimensions are typically reduced by subsampling.\nThe classi\ufb01cation nets subsample to keep \ufb01lters small and\ncomputational requirements reasonable. This coarsens the\noutput of a fully convolutional version of these nets, reducing it from the size of the input by a factor equal to the pixel\nstride of the receptive \ufb01elds of the output units.\n1Assuming ef\ufb01cient batching of single image inputs. The classi\ufb01cation"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fee-6db4-11ee-9664-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-14", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "stride of the receptive \ufb01elds of the output units.\n1Assuming ef\ufb01cient batching of single image inputs. The classi\ufb01cation\nscores for a single image by itself take 5.4 ms to produce, which is nearly\n25times slower than the fully convolutional version.\n3.2. Shift-and-stitch is \ufb01lter rarefaction\nInput shifting and output interlacing is a trick that yields\ndense predictions from coarse outputs without interpolation, introduced by OverFeat [29]. If the outputs are downsampled by a factor of f, the input is shifted (by left and top\npadding)xpixels to the right and ypixels down, once for\nevery value of (x;y)2f0;:::;f\u00001g\u0002f 0;:::;f\u00001g.\nThesef2inputs are each run through the convnet, and the\noutputs are interlaced so that the predictions correspond to\nthe pixels at the centers of their receptive \ufb01elds.\nChanging only the \ufb01lters and layer strides of a convnet\ncan produce the same output as this shift-and-stitch trick.\nConsider a layer (convolution or pooling) with input stride"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fef-6db4-11ee-b247-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-15", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "can produce the same output as this shift-and-stitch trick.\nConsider a layer (convolution or pooling) with input stride\ns, and a following convolution layer with \ufb01lter weights fij\n(eliding the feature dimensions, irrelevant here). Setting the\nlower layer\u2019s input stride to 1upsamples its output by a factor ofs, just like shift-and-stitch. However, convolving the\noriginal \ufb01lter with the upsampled output does not produce\nthe same result as the trick, because the original \ufb01lter only\nsees a reduced portion of its (now upsampled) input. To\nreproduce the trick, rarefy the \ufb01lter by enlarging it as\nf0\nij=\u001afi=s;j=s ifsdivides both iandj;\n0 otherwise,\n(withiandjzero-based). Reproducing the full net output\nof the trick involves repeating this \ufb01lter enlargement layerby-layer until all subsampling is removed.\nSimply decreasing subsampling within a net is a tradeoff:\nthe \ufb01lters see \ufb01ner information, but have smaller receptive\n\ufb01elds and take longer to compute. We have seen that the\nshift-and-stitch trick is another kind of tradeoff: the output"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3ff0-6db4-11ee-ae0e-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-16", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "\ufb01elds and take longer to compute. We have seen that the\nshift-and-stitch trick is another kind of tradeoff: the output\nis made denser without decreasing the receptive \ufb01eld sizes\nof the \ufb01lters, but the \ufb01lters are prohibited from accessing\ninformation at a \ufb01ner scale than their original design.\nAlthough we have done preliminary experiments with\nshift-and-stitch, we do not use it in our model. We \ufb01nd\nlearning through upsampling, as described in the next section, to be more effective and ef\ufb01cient, especially when\ncombined with the skip layer fusion described later on.\n3.3. Upsampling is backwards strided convolution\nAnother way to connect coarse outputs to dense pixels\nis interpolation. For instance, simple bilinear interpolation\ncomputes each output yijfrom the nearest four inputs by a\nlinear map that depends only on the relative positions of the\ninput and output cells.\nIn a sense, upsampling with factor fis convolution with\nafractional input stride of 1=f. So long as fis integral, a\nnatural way to upsample is therefore backwards convolution\n(sometimes called deconvolution ) with an output stride of\nf. Such an operation is trivial to implement, since it simply"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3ff1-6db4-11ee-a9d4-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-17", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "natural way to upsample is therefore backwards convolution\n(sometimes called deconvolution ) with an output stride of\nf. Such an operation is trivial to implement, since it simply\nreverses the forward and backward passes of convolution.Thus upsampling is performed in-network for end-to-end\nlearning by backpropagation from the pixelwise loss.\nNote that the deconvolution \ufb01lter in such a layer need not\nbe \ufb01xed (e.g., to bilinear upsampling), but can be learned.\nA stack of deconvolution layers and activation functions can\neven learn a nonlinear upsampling.\nIn our experiments, we \ufb01nd that in-network upsampling\nis fast and effective for learning dense prediction. Our best\nsegmentation architecture uses these layers to learn to upsample for re\ufb01ned prediction in Section 4.2.\n3.4. Patchwise training is loss sampling\nIn stochastic optimization, gradient computation is\ndriven by the training distribution. Both patchwise training and fully-convolutional training can be made to produce any distribution, although their relative computational\nef\ufb01ciency depends on overlap and minibatch size. Whole\nimage fully convolutional training is identical to patchwise\ntraining where each batch consists of all the receptive \ufb01elds"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3ff2-6db4-11ee-b5ac-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-18", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "image fully convolutional training is identical to patchwise\ntraining where each batch consists of all the receptive \ufb01elds\nof the units below the loss for an image (or collection of\nimages). While this is more ef\ufb01cient than uniform sampling\nof patches, it reduces the number of possible batches. However, random selection of patches within an image may be\nrecovered simply. Restricting the loss to a randomly sampled subset of its spatial terms (or, equivalently applying a\nDropConnect mask [36] between the output and the loss)\nexcludes patches from the gradient computation.\nIf the kept patches still have signi\ufb01cant overlap, fully\nconvolutional computation will still speed up training. If\ngradients are accumulated over multiple backward passes,\nbatches can include patches from several images.2\nSampling in patchwise training can correct class imbalance [27, 8, 2] and mitigate the spatial correlation of dense\npatches [28, 16]. In fully convolutional training, class balance can also be achieved by weighting the loss, and loss\nsampling can be used to address spatial correlation.\nWe explore training with sampling in Section 4.3, and do\nnot \ufb01nd that it yields faster or better convergence for dense\nprediction. Whole image training is effective and ef\ufb01cient.\n4. Segmentation Architecture"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3ff3-6db4-11ee-bb99-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-19", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "not \ufb01nd that it yields faster or better convergence for dense\nprediction. Whole image training is effective and ef\ufb01cient.\n4. Segmentation Architecture\nWe cast ILSVRC classi\ufb01ers into FCNs and augment\nthem for dense prediction with in-network upsampling and\na pixelwise loss. We train for segmentation by \ufb01ne-tuning.\nNext, we build a novel skip architecture that combines\ncoarse, semantic and local, appearance information to re\ufb01ne prediction.\nFor this investigation, we train and validate on the PASCAL VOC 2011 segmentation challenge [7]. We train with\n2Note that not every possible patch is included this way, since the receptive \ufb01elds of the \ufb01nal layer units lie on a \ufb01xed, strided grid. However,\nby shifting the image left and down by a random value up to the stride,\nrandom selection from all possible patches may be recovered.\na per-pixel multinomial logistic loss and validate with the\nstandard metric of mean pixel intersection over union, with\nthe mean taken over all classes, including background. The\ntraining ignores pixels that are masked out (as ambiguous\nor dif\ufb01cult) in the ground truth.\n4.1. From classi\ufb01er to dense FCN"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3ff4-6db4-11ee-8159-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-20", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "training ignores pixels that are masked out (as ambiguous\nor dif\ufb01cult) in the ground truth.\n4.1. From classi\ufb01er to dense FCN\nWe begin by convolutionalizing proven classi\ufb01cation architectures as in Section 3. We consider the AlexNet3architecture [19] that won ILSVRC12, as well as the VGG\nnets [31] and the GoogLeNet4[32] which did exceptionally well in ILSVRC14. We pick the VGG 16-layer net5,\nwhich we found to be equivalent to the 19-layer net on this\ntask. For GoogLeNet, we use only the \ufb01nal loss layer, and\nimprove performance by discarding the \ufb01nal average pooling layer. We decapitate each net by discarding the \ufb01nal\nclassi\ufb01er layer, and convert all fully connected layers to\nconvolutions. We append a 1\u00021convolution with channel dimension 21to predict scores for each of the PASCAL classes (including background) at each of the coarse\noutput locations, followed by a deconvolution layer to bilinearly upsample the coarse outputs to pixel-dense outputs\nas described in Section 3.3. Table 1 compares the preliminary validation results along with the basic characteristics"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3ff5-6db4-11ee-b391-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-21", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "as described in Section 3.3. Table 1 compares the preliminary validation results along with the basic characteristics\nof each net. We report the best results achieved after convergence at a \ufb01xed learning rate (at least 175 epochs).\nFine-tuning from classi\ufb01cation to segmentation gave reasonable predictions for each net. Even the worst model\nachieved\u001875% of state-of-the-art performance. The\nsegmentation-equippped VGG net (FCN-VGG16) already\nappears to be state-of-the-art at 56.0 mean IU on val, compared to 52.6 on test [16]. Training on extra data raises\nperformance to 59.4 mean IU on a subset of val7. Training\ndetails are given in Section 4.3.\nDespite similar classi\ufb01cation accuracy, our implementation of GoogLeNet did not match this segmentation result.\n4.2. Combining what and where\nWe de\ufb01ne a new fully convolutional net (FCN) for segmentation that combines layers of the feature hierarchy and\nre\ufb01nes the spatial precision of the output. See Figure 3.\nWhile fully convolutionalized classi\ufb01ers can be \ufb01netuned to segmentation as shown in 4.1, and even score"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3ff6-6db4-11ee-8ea4-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-22", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "While fully convolutionalized classi\ufb01ers can be \ufb01netuned to segmentation as shown in 4.1, and even score\nhighly on the standard metric, their output is dissatisfyingly\ncoarse (see Figure 4). The 32 pixel stride at the \ufb01nal prediction layer limits the scale of detail in the upsampled output.\nWe address this by adding links that combine the \ufb01nal\nprediction layer with lower layers with \ufb01ner strides. This\n3Using the publicly available CaffeNet reference model.\n4Since there is no publicly available version of GoogLeNet, we use\nour own reimplementation. Our version is trained with less extensive data\naugmentation, and gets 68.5% top-1 and 88.4% top-5 ILSVRC accuracy.\n5Using the publicly available version from the Caffe model zoo.Table 1. We adapt and extend three classi\ufb01cation convnets to segmentation. We compare performance by mean intersection over\nunion on the validation set of PASCAL VOC 2011 and by inference time (averaged over 20 trials for a 500\u0002500 input on an\nNVIDIA Tesla K40c). We detail the architecture of the adapted\nnets as regards dense prediction: number of parameter layers, receptive \ufb01eld size of output units, and the coarsest stride within the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3ff7-6db4-11ee-b55c-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-23", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "nets as regards dense prediction: number of parameter layers, receptive \ufb01eld size of output units, and the coarsest stride within the\nnet. (These numbers give the best performance obtained at a \ufb01xed\nlearning rate, not best performance possible.)\nFCNAlexNetFCNVGG16FCNGoogLeNet4\nmean IU 39.8 56.0 42.5\nforward time 50 ms 210 ms 59 ms\nconv. layers 8 16 22\nparameters 57M 134M 6M\nrf size 355 404 907\nmax stride 32 32 32\nturns a line topology into a DAG, with edges that skip ahead\nfrom lower layers to higher ones (Figure 3). As they see\nfewer pixels, the \ufb01ner scale predictions should need fewer\nlayers, so it makes sense to make them from shallower net\noutputs. Combining \ufb01ne layers and coarse layers lets the\nmodel make local predictions that respect global structure.\nBy analogy to the multiscale local jet of Florack et al. [10],\nwe call our nonlinear local feature hierarchy the deep jet .\nWe \ufb01rst divide the output stride in half by predicting\nfrom a 16 pixel stride layer. We add a 1\u00021convolution\nlayer on top of pool4 to produce additional class predictions. We fuse this output with the predictions computed"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3ff8-6db4-11ee-a509-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-24", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "from a 16 pixel stride layer. We add a 1\u00021convolution\nlayer on top of pool4 to produce additional class predictions. We fuse this output with the predictions computed\non top of conv7 (convolutionalized fc7) at stride 32 by\nadding a 2\u0002upsampling layer and summing6both predictions. (See Figure 3). We initialize the 2\u0002upsampling to\nbilinear interpolation, but allow the parameters to be learned\nas described in Section 3.3. Finally, the stride 16 predictions\nare upsampled back to the image. We call this net FCN-16s.\nFCN-16s is learned end-to-end, initialized with the parameters of the last, coarser net, which we now call FCN-32s.\nThe new parameters acting on pool4 are zero-initialized so\nthat the net starts with unmodi\ufb01ed predictions. The learning\nrate is decreased by a factor of 100.\nLearning this skip net improves performance on the validation set by 3.0 mean IU to 62.4. Figure 4 shows improvement in the \ufb01ne structure of the output. We compared\nthis fusion with learning only from the pool4 layer (which\nresulted in poor performance), and simply decreasing the\nlearning rate without adding the extra link (which results"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3ff9-6db4-11ee-86d0-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-25", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "this fusion with learning only from the pool4 layer (which\nresulted in poor performance), and simply decreasing the\nlearning rate without adding the extra link (which results\nin an insigni\ufb01cant performance improvement, without improving the quality of the output).\nWe continue in this fashion by fusing predictions from\npool3 with a 2\u0002upsampling of predictions fused from\npool4 andconv7 , building the net FCN-8s. We obtain\n6Max fusion made learning dif\ufb01cult due to gradient switching.\nFCN-32s FCN-16s FCN-8s Ground truth\nFigure 4. Re\ufb01ning fully convolutional nets by fusing information\nfrom layers with different strides improves segmentation detail.\nThe \ufb01rst three images show the output from our 32, 16, and 8\npixel stride nets (see Figure 3).\nTable 2. Comparison of skip FCNs on a subset of PASCAL\nVOC2011 validation7. Learning is end-to-end, except for FCN32s-\ufb01xed, where only the last layer is \ufb01ne-tuned. Note that FCN32s is FCN-VGG16, renamed to highlight stride.\npixel\nacc.mean\nacc.mean\nIUf.w.\nIU"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3ffa-6db4-11ee-b2c7-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-26", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "pixel\nacc.mean\nacc.mean\nIUf.w.\nIU\nFCN-32s-\ufb01xed 83.0 59.7 45.4 72.0\nFCN-32s 89.1 73.3 59.4 81.4\nFCN-16s 90.0 75.7 62.4 83.0\nFCN-8s 90.3 75.9 62.7 83.2\na minor additional improvement to 62.7 mean IU, and \ufb01nd\na slight improvement in the smoothness and detail of our\noutput. At this point our fusion improvements have met diminishing returns, both with respect to the IU metric which\nemphasizes large-scale correctness, and also in terms of the\nimprovement visible e.g. in Figure 4, so we do not continue\nfusing even lower layers.\nRe\ufb01nement by other means Decreasing the stride of\npooling layers is the most straightforward way to obtain\n\ufb01ner predictions. However, doing so is problematic for our\nVGG16-based net. Setting the pool5 layer to have stride 1\nrequires our convolutionalized fc6 to have a kernel size of14\u000214in order to maintain its receptive \ufb01eld size. In addition to their computational cost, we had dif\ufb01culty learning"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3ffb-6db4-11ee-9535-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-27", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "such large \ufb01lters. We made an attempt to re-architect the\nlayers above pool5 with smaller \ufb01lters, but were not successful in achieving comparable performance; one possible\nexplanation is that the initialization from ImageNet-trained\nweights in the upper layers is important.\nAnother way to obtain \ufb01ner predictions is to use the shiftand-stitch trick described in Section 3.2. In limited experiments, we found the cost to improvement ratio from this\nmethod to be worse than layer fusion.\n4.3. Experimental framework\nOptimization We train by SGD with momentum. We\nuse a minibatch size of 20 images and \ufb01xed learning rates of\n10\u00003,10\u00004, and 5\u00005for FCN-AlexNet, FCN-VGG16, and\nFCN-GoogLeNet, respectively, chosen by line search. We\nuse momentum 0:9, weight decay of 5\u00004or2\u00004, and doubled the learning rate for biases, although we found training\nto be insensitive to these parameters (but sensitive to the\nlearning rate). We zero-initialize the class scoring convolution layer, \ufb01nding random initialization to yield neither\nbetter performance nor faster convergence. Dropout was included where used in the original classi\ufb01er nets."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3ffc-6db4-11ee-8cb2-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-28", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "better performance nor faster convergence. Dropout was included where used in the original classi\ufb01er nets.\nFine-tuning We \ufb01ne-tune all layers by backpropagation through the whole net. Fine-tuning the\noutput classi\ufb01er alone yields only 70% of the full \ufb01netuning performance as compared in Table 2. Training from\nscratch is not feasible considering the time required to\nlearn the base classi\ufb01cation nets. (Note that the VGG net is\ntrained in stages, while we initialize from the full 16-layer\nversion.) Fine-tuning takes three days on a single GPU for\nthe coarse FCN-32s version, and about one day each to\nupgrade to the FCN-16s and FCN-8s versions.\nPatch Sampling As explained in Section 3.4, our full\nimage training effectively batches each image into a reguimage pool4 pool5 pool1 pool2 pool332x upsampled\nprediction (FCN-32s)2x upsampled\nprediction16x upsampled\nprediction (FCN-16s)8x upsampled\nprediction (FCN-8s)\npool4\nprediction2x upsampled\nprediction\npool3\npredictionP P"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3ffd-6db4-11ee-9418-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-29", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "prediction (FCN-8s)\npool4\nprediction2x upsampled\nprediction\npool3\npredictionP P\nFigure 3. Our DAG nets learn to combine coarse, high layer information with \ufb01ne, low layer information. Layers are shown as grids that\nreveal relative spatial coarseness. Only pooling and prediction layers are shown; intermediate convolution layers (including our converted\nfully connected layers) are omitted. Solid line (FCN-32s): Our single-stream net, described in Section 4.1, upsamples stride 32 predictions\nback to pixels in a single step. Dashed line (FCN-16s): Combining predictions from both the \ufb01nal layer and the pool4 layer, at stride\n16, lets our net predict \ufb01ner details, while retaining high-level semantic information. Dotted line (FCN-8s): Additional predictions from\npool3 , at stride 8, provide further precision.\n500 1000 1500\niteration number0.40.60.81.01.2lossfull images\n50% sampling\n25% sampling\n10000 20000 30000\nrelative time (num. images processed)0.40.60.81.01.2lossFigure 5. Training on whole images is just as effective as sampling\npatches, but results in faster (wall time) convergence by making"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3ffe-6db4-11ee-b6a7-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-30", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "patches, but results in faster (wall time) convergence by making\nmore ef\ufb01cient use of data. Left shows the effect of sampling on\nconvergence rate for a \ufb01xed expected batch size, while right plots\nthe same by relative wall time.\nlar grid of large, overlapping patches. By contrast, prior\nwork randomly samples patches over a full dataset [27, 2,\n8, 28, 11], potentially resulting in higher variance batches\nthat may accelerate convergence [22]. We study this tradeoff by spatially sampling the loss in the manner described\nearlier, making an independent choice to ignore each \ufb01nal\nlayer cell with some probability 1\u0000p. To avoid changing the\neffective batch size, we simultaneously increase the number\nof images per batch by a factor 1=p. Note that due to the ef\ufb01ciency of convolution, this form of rejection sampling is\nstill faster than patchwise training for large enough values\nofp(e.g., at least for p > 0:2according to the numbers\nin Section 3.1). Figure 5 shows the effect of this form of\nsampling on convergence. We \ufb01nd that sampling does not\nhave a signi\ufb01cant effect on convergence rate compared to\nwhole image training, but takes signi\ufb01cantly more time due"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a3fff-6db4-11ee-95b8-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-31", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "have a signi\ufb01cant effect on convergence rate compared to\nwhole image training, but takes signi\ufb01cantly more time due\nto the larger number of images that need to be considered\nper batch. We therefore choose unsampled, whole image\ntraining in our other experiments.\nClass Balancing Fully convolutional training can balance classes by weighting or sampling the loss. Although\nour labels are mildly unbalanced (about 3=4are background), we \ufb01nd class balancing unnecessary.\nDense Prediction The scores are upsampled to the input dimensions by deconvolution layers within the net. Final layer deconvolutional \ufb01lters are \ufb01xed to bilinear interpolation, while intermediate upsampling layers are initialized to bilinear upsampling, and then learned. Shift-andstitch (Section 3.2), or the \ufb01lter rarefaction equivalent, are\nnot used.\nAugmentation We tried augmenting the training data\nby randomly mirroring and \u201cjittering\u201d the images by translating them up to 32 pixels (the coarsest scale of prediction)\nin each direction. This yielded no noticeable improvement.\nMore Training Data The PASCAL VOC 2011 segmentation challenge training set, which we used for Table 1,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4000-6db4-11ee-9932-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-32", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "in each direction. This yielded no noticeable improvement.\nMore Training Data The PASCAL VOC 2011 segmentation challenge training set, which we used for Table 1,\nlabels 1112 images. Hariharan et al . [15] have collectedlabels for a much larger set of 8498 PASCAL training images, which was used to train the previous state-of-the-art\nsystem, SDS [16]. This training data improves the FCNVGG16 validation score7by 3.4 points to 59.4 mean IU.\nImplementation All models are trained and tested with\nCaffe [18] on a single NVIDIA Tesla K40c. The models\nand code will be released open-source on publication.\n5. Results\nWe test our FCN on semantic segmentation and scene\nparsing, exploring PASCAL VOC, NYUDv2, and SIFT\nFlow. Although these tasks have historically distinguished\nbetween objects and regions, we treat both uniformly as\npixel prediction. We evaluate our FCN skip architecture8\non each of these datasets, and then extend it to multi-modal\ninput for NYUDv2 and multi-task prediction for the semantic and geometric labels of SIFT Flow.\nMetrics We report four metrics from common semantic\nsegmentation and scene parsing evaluations that are variations on pixel accuracy and region intersection over union"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4001-6db4-11ee-be3d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-33", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "Metrics We report four metrics from common semantic\nsegmentation and scene parsing evaluations that are variations on pixel accuracy and region intersection over union\n(IU). Letnijbe the number of pixels of class ipredicted to\nbelong to class j, where there are ncldifferent classes, and\nletti=P\njnijbe the total number of pixels of class i. We\ncompute:\n\u000fpixel accuracy:P\ninii=P\niti\n\u000fmean accuraccy: (1=ncl)P\ninii=ti\n\u000fmean IU: (1=ncl)P\ninii=\u0010\nti+P\njnji\u0000nii\u0011\n\u000ffrequency weighted IU:\n(P\nktk)\u00001P\nitinii=\u0010\nti+P\njnji\u0000nii\u0011\nPASCAL VOC Table 3 gives the performance of our\nFCN-8s on the test sets of PASCAL VOC 2011 and 2012,\nand compares it to the previous state-of-the-art, SDS [16],\nand the well-known R-CNN [12]. We achieve the best results on mean IU9by a relative margin of 20%. Inference\ntime is reduced 114\u0002(convnet only, ignoring proposals and\nre\ufb01nement) or 286\u0002(overall)."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4002-6db4-11ee-9ad7-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-34", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "time is reduced 114\u0002(convnet only, ignoring proposals and\nre\ufb01nement) or 286\u0002(overall).\nTable 3. Our fully convolutional net gives a 20% relative improvement over the state-of-the-art on the PASCAL VOC 2011 and 2012\ntest sets, and reduces inference time.\nmean IU mean IU inference\nVOC2011 test VOC2012 test time\nR-CNN [12] 47.9 - SDS [16] 52.6 51.6 \u001850 s\nFCN-8s 62.7 62.2 \u0018175 ms\nNYUDv2 [30] is an RGB-D dataset collected using the\n7There are training images from [15] included in the PASCAL VOC\n2011 val set, so we validate on the non-intersecting set of 736 images. An\nearlier version of this paper mistakenly evaluated on the entire val set.\n8Our models and code are publicly available at\nhttps://github.com/BVLC/caffe/wiki/Model-Zoo#fcn .\n9This is the only metric provided by the test server.\nTable 4. Results on NYUDv2. RGBD is early-fusion of the\nRGB and depth channels at the input. HHA is the depth embedding of [14] as horizontal disparity, height above ground, and"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4003-6db4-11ee-aee9-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-35", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "RGB and depth channels at the input. HHA is the depth embedding of [14] as horizontal disparity, height above ground, and\nthe angle of the local surface normal with the inferred gravity\ndirection. RGB-HHA is the jointly trained late fusion model\nthat sums RGB and HHA predictions.\npixel\nacc.mean\nacc.mean\nIUf.w.\nIU\nGupta et al. [14] 60.3 - 28.6 47.0\nFCN-32s RGB 60.0 42.2 29.2 43.9\nFCN-32s RGBD 61.5 42.4 30.5 45.5\nFCN-32s HHA 57.1 35.2 24.2 40.4\nFCN-32s RGB-HHA 64.3 44.9 32.8 48.0\nFCN-16s RGB-HHA 65.4 46.1 34.0 49.5\nMicrosoft Kinect. It has 1449 RGB-D images, with pixelwise labels that have been coalesced into a 40 class semantic segmentation task by Gupta et al. [13]. We report results\non the standard split of 795 training images and 654 testing\nimages. (Note: all model selection is performed on PASCAL 2011 val.) Table 4 gives the performance of our model"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4004-6db4-11ee-a8b2-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-36", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "on the standard split of 795 training images and 654 testing\nimages. (Note: all model selection is performed on PASCAL 2011 val.) Table 4 gives the performance of our model\nin several variations. First we train our unmodi\ufb01ed coarse\nmodel (FCN-32s) on RGB images. To add depth information, we train on a model upgraded to take four-channel\nRGB-D input (early fusion). This provides little bene\ufb01t,\nperhaps due to the dif\ufb01cultly of propagating meaningful\ngradients all the way through the model. Following the success of Gupta et al. [14], we try the three-dimensional HHA\nencoding of depth, training nets on just this information, as\nwell as a \u201clate fusion\u201d of RGB and HHA where the predictions from both nets are summed at the \ufb01nal layer, and the\nresulting two-stream net is learned end-to-end. Finally we\nupgrade this late fusion net to a 16-stride version.\nSIFT Flow is a dataset of 2,688 images with pixel labels\nfor 33 semantic categories (\u201cbridge\u201d, \u201cmountain\u201d, \u201csun\u201d),"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4005-6db4-11ee-8390-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-37", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "SIFT Flow is a dataset of 2,688 images with pixel labels\nfor 33 semantic categories (\u201cbridge\u201d, \u201cmountain\u201d, \u201csun\u201d),\nas well as three geometric categories (\u201chorizontal\u201d, \u201cvertical\u201d, and \u201csky\u201d). An FCN can naturally learn a joint representation that simultaneously predicts both types of labels.\nWe learn a two-headed version of FCN-16s with semantic and geometric prediction layers and losses. The learned\nmodel performs as well on both tasks as two independently\ntrained models, while learning and inference are essentially\nas fast as each independent model by itself. The results in\nTable 5, computed on the standard split into 2,488 training\nand 200 test images,10show state-of-the-art performance on\nboth tasks.\n10Three of the SIFT Flow categories are not present in the test set. We\nmade predictions across all 33 categories, but only included categories actually present in the test set in our evaluation. (An earlier version of this paper reported a lower mean IU, which included all categories either present\nor predicted in the evaluation.)Table 5. Results on SIFT Flow10with class segmentation\n(center) and geometric segmentation (right). Tighe [33] is\na non-parametric transfer method. Tighe 1 is an exemplar"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4006-6db4-11ee-b282-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-38", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "(center) and geometric segmentation (right). Tighe [33] is\na non-parametric transfer method. Tighe 1 is an exemplar\nSVM while 2 is SVM + MRF. Farabet is a multi-scale convnet trained on class-balanced samples (1) or natural frequency\nsamples (2). Pinheiro is a multi-scale, recurrent convnet, denoted RCNN 3(\u000e3). The metric for geometry is pixel accuracy.\npixel\nacc.mean\nacc.mean\nIUf.w.\nIUgeom.\nacc.\nLiuet al. [23] 76.7 - - - Tighe et al. [33] - - - - 90.8\nTighe et al. [34] 1 75.6 41.1 - - Tighe et al. [34] 2 78.6 39.2 - - Farabet et al. [8] 1 72.3 50.8 - - Farabet et al. [8] 2 78.5 29.6 - - Pinheiro et al. [28] 77.7 29.8 - - FCN-16s 85.2 51.7 39.5 76.1 94.3\nFCN-8s SDS [16] Ground Truth Image"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4007-6db4-11ee-a3b0-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-39", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "FCN-8s SDS [16] Ground Truth Image\nFigure 6. Fully convolutional segmentation nets produce stateof-the-art performance on PASCAL. The left column shows the\noutput of our highest performing net, FCN-8s. The second shows\nthe segmentations produced by the previous state-of-the-art system\nby Hariharan et al. [16]. Notice the \ufb01ne structures recovered (\ufb01rst\nrow), ability to separate closely interacting objects (second row),\nand robustness to occluders (third row). The fourth row shows a\nfailure case: the net sees lifejackets in a boat as people.\n6. Conclusion\nFully convolutional networks are a rich class of models, of which modern classi\ufb01cation convnets are a special case. Recognizing this, extending these classi\ufb01cation\nnets to segmentation, and improving the architecture with\nmulti-resolution layer combinations dramatically improves\nthe state-of-the-art, while simultaneously simplifying and\nspeeding up learning and inference.\nAcknowledgements This work was supported in part\nby DARPA\u2019s MSEE and SMISC programs, NSF awards IIS1427425, IIS-1212798, IIS-1116411, and the NSF GRFP,"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4008-6db4-11ee-9839-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-40", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "Toyota, and the Berkeley Vision and Learning Center. We\ngratefully acknowledge NVIDIA for GPU donation. We\nthank Bharath Hariharan and Saurabh Gupta for their advice and dataset tools. We thank Sergio Guadarrama for\nreproducing GoogLeNet in Caffe. We thank Jitendra Malik\nfor his helpful comments. Thanks to Wei Liu for pointing\nout an issue wth our SIFT Flow mean IU computation and\nan error in our frequency weighted mean IU formula.\nA. Upper Bounds on IU\nIn this paper, we have achieved good performance on\nthe mean IU segmentation metric even with coarse semantic\nprediction. To better understand this metric and the limits\nof this approach with respect to it, we compute approximate\nupper bounds on performance with prediction at various\nscales. We do this by downsampling ground truth images\nand then upsampling them again to simulate the best results\nobtainable with a particular downsampling factor. The following table gives the mean IU on a subset of PASCAL\n2011 val for various downsampling factors.\nfactor mean IU\n128 50.9\n64 73.3\n32 86.1\n16 92.8\n896.4\n498.5\nPixel-perfect prediction is clearly not necessary to"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4009-6db4-11ee-9d19-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-41", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "128 50.9\n64 73.3\n32 86.1\n16 92.8\n896.4\n498.5\nPixel-perfect prediction is clearly not necessary to\nachieve mean IU well above state-of-the-art, and, conversely, mean IU is a not a good measure of \ufb01ne-scale accuracy.\nB. More Results\nWe further evaluate our FCN for semantic segmentation.\nPASCAL-Context [26] provides whole scene annotations of PASCAL VOC 2010. While there are over 400 distinct classes, we follow the 59 class task de\ufb01ned by [26] that\npicks the most frequent classes. We train and evaluate on\nthe training and val sets respectively. In Table 6, we compare to the joint object + stuff variation of Convolutional\nFeature Masking [3] which is the previous state-of-the-art\non this task. FCN-8s scores 35.1 mean IU for an 11% relative improvement.\nChangelog\nThe arXiv version of this paper is kept up-to-date with\ncorrections and additional relevant material. The following\ngives a brief history of changes.Table 6. Results on PASCAL-Context. CFM is the best result of\n[3] by convolutional feature masking and segment pursuit with the"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a400a-6db4-11ee-bb50-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-42", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "[3] by convolutional feature masking and segment pursuit with the\nVGG net. O2Pis the second order pooling method [1] as reported\nin the errata of [26]. The 59 class task includes the 59 most frequent classes while the 33 class task consists of an easier subset\nidenti\ufb01ed by [26].\n59 classpixel\nacc.mean\nacc.mean\nIUf.w.\nIU\nO2P - - 18.1 CFM - - 31.5 FCN-32s 63.8 42.7 31.8 48.3\nFCN-16s 65.7 46.2 34.8 50.7\nFCN-8s 65.9 46.5 35.1 51.0\n33 class\nO2P - - 29.2 CFM - - 46.1 FCN-32s 69.8 65.1 50.4 54.9\nFCN-16s 71.8 68.0 53.4 57.5\nFCN-8s 71.8 67.6 53.5 57.7\nv2Add Appendix A giving upper bounds on mean IU and\nAppendix B with PASCAL-Context results. Correct PASCAL validation numbers (previously, some val images were\nincluded in train), SIFT Flow mean IU (which used an inappropriately strict metric), and an error in the frequency"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a400b-6db4-11ee-9871-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-43", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "included in train), SIFT Flow mean IU (which used an inappropriately strict metric), and an error in the frequency\nweighted mean IU formula. Add link to models and update\ntiming numbers to re\ufb02ect improved implementation (which\nis publicly available).\nReferences\n[1] J. Carreira, R. Caseiro, J. Batista, and C. Sminchisescu. Semantic segmentation with second-order pooling. In ECCV ,\n2012. 9\n[2] D. C. Ciresan, A. Giusti, L. M. Gambardella, and J. Schmidhuber. Deep neural networks segment neuronal membranes\nin electron microscopy images. In NIPS , pages 2852\u20132860,\n2012. 1, 2, 4, 7\n[3] J. Dai, K. He, and J. Sun. Convolutional feature masking for joint object and stuff segmentation. arXiv preprint\narXiv:1412.1283 , 2014. 9\n[4] J. Donahue, Y . Jia, O. Vinyals, J. Hoffman, N. Zhang,\nE. Tzeng, and T. Darrell. DeCAF: A deep convolutional activation feature for generic visual recognition. In ICML , 2014.\n1, 2"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a400c-6db4-11ee-a1d4-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-44", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "E. Tzeng, and T. Darrell. DeCAF: A deep convolutional activation feature for generic visual recognition. In ICML , 2014.\n1, 2\n[5] D. Eigen, D. Krishnan, and R. Fergus. Restoring an image\ntaken through a window covered with dirt or rain. In Computer Vision (ICCV), 2013 IEEE International Conference\non, pages 633\u2013640. IEEE, 2013. 2\n[6] D. Eigen, C. Puhrsch, and R. Fergus. Depth map prediction\nfrom a single image using a multi-scale deep network. arXiv\npreprint arXiv:1406.2283 , 2014. 2\n[7] M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn,\nand A. Zisserman. The PASCAL Visual Object Classes\nChallenge 2011 (VOC2011) Results. http://www.pascalnetwork.org/challenges/VOC/voc2011/workshop/index.html.\n4\n[8] C. Farabet, C. Couprie, L. Najman, and Y . LeCun. Learning\nhierarchical features for scene labeling. Pattern Analysis and\nMachine Intelligence, IEEE Transactions on , 2013. 1, 2, 4,\n7, 8"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a400d-6db4-11ee-9318-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-45", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "hierarchical features for scene labeling. Pattern Analysis and\nMachine Intelligence, IEEE Transactions on , 2013. 1, 2, 4,\n7, 8\n[9] P. Fischer, A. Dosovitskiy, and T. Brox. Descriptor matching\nwith convolutional neural networks: a comparison to SIFT.\nCoRR , abs/1405.5769, 2014. 1\n[10] L. Florack, B. T. H. Romeny, M. Viergever, and J. Koenderink. The gaussian scale-space paradigm and the multiscale local jet. International Journal of Computer Vision ,\n18(1):61\u201375, 1996. 5\n[11] Y . Ganin and V . Lempitsky. N4-\ufb01elds: Neural network nearest neighbor \ufb01elds for image transforms. In ACCV , 2014. 1,\n2, 7\n[12] R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature hierarchies for accurate object detection and semantic\nsegmentation. In Computer Vision and Pattern Recognition ,\n2014. 1, 2, 7\n[13] S. Gupta, P. Arbelaez, and J. Malik. Perceptual organization\nand recognition of indoor scenes from RGB-D images. In"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a400e-6db4-11ee-9ac3-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-46", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "[13] S. Gupta, P. Arbelaez, and J. Malik. Perceptual organization\nand recognition of indoor scenes from RGB-D images. In\nCVPR , 2013. 8\n[14] S. Gupta, R. Girshick, P. Arbelaez, and J. Malik. Learning\nrich features from RGB-D images for object detection and\nsegmentation. In ECCV . Springer, 2014. 1, 2, 8\n[15] B. Hariharan, P. Arbelaez, L. Bourdev, S. Maji, and J. Malik.\nSemantic contours from inverse detectors. In International\nConference on Computer Vision (ICCV) , 2011. 7\n[16] B. Hariharan, P. Arbel \u00b4aez, R. Girshick, and J. Malik. Simultaneous detection and segmentation. In European Conference on Computer Vision (ECCV) , 2014. 1, 2, 4, 5, 7, 8\n[17] K. He, X. Zhang, S. Ren, and J. Sun. Spatial pyramid pooling\nin deep convolutional networks for visual recognition. In\nECCV , 2014. 1, 2"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a400f-6db4-11ee-8f36-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-47", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "in deep convolutional networks for visual recognition. In\nECCV , 2014. 1, 2\n[18] Y . Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. arXiv preprint\narXiv:1408.5093 , 2014. 7\n[19] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet\nclassi\ufb01cation with deep convolutional neural networks. In\nNIPS , 2012. 1, 2, 3, 5\n[20] Q. V . Le, R. Monga, M. Devin, K. Chen, G. S. Corrado,\nJ. Dean, and A. Y . Ng. Building high-level features using\nlarge scale unsupervised learning. In ICML , 2012. 3\n[21] Y . LeCun, B. Boser, J. Denker, D. Henderson, R. E. Howard,\nW. Hubbard, and L. D. Jackel. Backpropagation applied to\nhand-written zip code recognition. In Neural Computation ,\n1989. 2, 3"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4010-6db4-11ee-a369-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-48", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "W. Hubbard, and L. D. Jackel. Backpropagation applied to\nhand-written zip code recognition. In Neural Computation ,\n1989. 2, 3\n[22] Y . A. LeCun, L. Bottou, G. B. Orr, and K.-R. M \u00a8uller. Ef\ufb01cient backprop. In Neural networks: Tricks of the trade ,\npages 9\u201348. Springer, 1998. 7\n[23] C. Liu, J. Yuen, and A. Torralba. Sift \ufb02ow: Dense correspondence across scenes and its applications. Pattern Analysis\nand Machine Intelligence, IEEE Transactions on , 33(5):978\u2013\n994, 2011. 8[24] J. Long, N. Zhang, and T. Darrell. Do convnets learn correspondence? In NIPS , 2014. 1\n[25] O. Matan, C. J. Burges, Y . LeCun, and J. S. Denker. Multidigit recognition using a space displacement neural network.\nInNIPS , pages 488\u2013495. Citeseer, 1991. 2"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4011-6db4-11ee-95eb-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-49", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "InNIPS , pages 488\u2013495. Citeseer, 1991. 2\n[26] R. Mottaghi, X. Chen, X. Liu, N.-G. Cho, S.-W. Lee, S. Fidler, R. Urtasun, and A. Yuille. The role of context for object\ndetection and semantic segmentation in the wild. In Computer Vision and Pattern Recognition (CVPR), 2014 IEEE\nConference on , pages 891\u2013898. IEEE, 2014. 9\n[27] F. Ning, D. Delhomme, Y . LeCun, F. Piano, L. Bottou, and\nP. E. Barbano. Toward automatic phenotyping of developing\nembryos from videos. Image Processing, IEEE Transactions\non, 14(9):1360\u20131371, 2005. 1, 2, 4, 7\n[28] P. H. Pinheiro and R. Collobert. Recurrent convolutional\nneural networks for scene labeling. In ICML , 2014. 1, 2,\n4, 7, 8\n[29] P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus,\nand Y . LeCun. Overfeat: Integrated recognition, localization"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4012-6db4-11ee-b11b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-50", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "and Y . LeCun. Overfeat: Integrated recognition, localization\nand detection using convolutional networks. In ICLR , 2014.\n1, 2, 3, 4\n[30] N. Silberman, D. Hoiem, P. Kohli, and R. Fergus. Indoor\nsegmentation and support inference from rgbd images. In\nECCV , 2012. 7\n[31] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. CoRR ,\nabs/1409.1556, 2014. 1, 2, 3, 5\n[32] C. Szegedy, W. Liu, Y . Jia, P. Sermanet, S. Reed,\nD. Anguelov, D. Erhan, V . Vanhoucke, and A. Rabinovich.\nGoing deeper with convolutions. CoRR , abs/1409.4842,\n2014. 1, 2, 3, 5\n[33] J. Tighe and S. Lazebnik. Superparsing: scalable nonparametric image parsing with superpixels. In ECCV , pages 352\u2013\n365. Springer, 2010. 8\n[34] J. Tighe and S. Lazebnik. Finding things: Image parsing with"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4013-6db4-11ee-8af9-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-51", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "365. Springer, 2010. 8\n[34] J. Tighe and S. Lazebnik. Finding things: Image parsing with\nregions and per-exemplar detectors. In CVPR , 2013. 8\n[35] J. Tompson, A. Jain, Y . LeCun, and C. Bregler. Joint training\nof a convolutional network and a graphical model for human\npose estimation. CoRR , abs/1406.2984, 2014. 2\n[36] L. Wan, M. Zeiler, S. Zhang, Y . L. Cun, and R. Fergus. Regularization of neural networks using dropconnect. In Proceedings of the 30th International Conference on Machine\nLearning (ICML-13) , pages 1058\u20131066, 2013. 4\n[37] R. Wolf and J. C. Platt. Postal address block location using\na convolutional locator network. Advances in Neural Information Processing Systems , pages 745\u2013745, 1994. 2\n[38] M. D. Zeiler and R. Fergus. Visualizing and understanding\nconvolutional networks. In Computer Vision\u2013ECCV 2014 ,\npages 818\u2013833. Springer, 2014. 2"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4014-6db4-11ee-874d-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.4038-52", "title": "Fully Convolutional Networks for Semantic Segmentation", "source": "http://arxiv.org/pdf/1411.4038", "start_index": 0, "chroma:document": "convolutional networks. In Computer Vision\u2013ECCV 2014 ,\npages 818\u2013833. Springer, 2014. 2\n[39] N. Zhang, J. Donahue, R. Girshick, and T. Darrell. Partbased r-cnns for \ufb01ne-grained category detection. In Computer Vision\u2013ECCV 2014 , pages 834\u2013849. Springer, 2014.\n1"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4015-6db4-11ee-a279-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-0", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "Deep Filter Banks for Texture Recognition and Segmentation\nMircea Cimpoi\nUniversity of Oxford\nmircea@robots.ox.ac.ukSubhransu Maji\nUniversity of Massachusetts, Amherst\nsmaji@cs.umass.eduAndrea Vedaldi\nUniversity of Oxford\nvedaldi@robots.ox.ac.uk\nAbstract\nResearch in texture recognition often concentrates on the\nproblem of material recognition in uncluttered conditions,\nan assumption rarely met by applications. In this work we\nconduct a \ufb01rst study of material and describable texture attributes recognition in clutter, using a new dataset derived\nfrom the OpenSurface texture repository. Motivated by the\nchallenge posed by this problem, we propose a new texture\ndescriptor, FV-CNN, obtained by Fisher Vector pooling of a\nConvolutional Neural Network (CNN) \ufb01lter bank. FV-CNN\nsubstantially improves the state-of-the-art in texture, material and scene recognition. Our approach achieves 79.8%\naccuracy on Flickr material dataset and 81% accuracy on\nMIT indoor scenes, providing absolute gains of more than\n10% over existing approaches. FV-CNN easily transfers\nacross domains without requiring feature adaptation as for\nmethods that build on the fully-connected layers of CNNs."}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4016-6db4-11ee-835b-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-1", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "10% over existing approaches. FV-CNN easily transfers\nacross domains without requiring feature adaptation as for\nmethods that build on the fully-connected layers of CNNs.\nFurthermore, FV-CNN can seamlessly incorporate multiscale information and describe regions of arbitrary shapes\nand sizes. Our approach is particularly suited at localizing \u201cstuff\u201d categories and obtains state-of-the-art results\non MSRC segmentation dataset, as well as promising results\non recognizing materials and surface attributes in clutter on\nthe OpenSurfaces dataset.\n1. Introduction\nTexture is ubiquitous and provides useful cues of material properties of objects and their identity, especially when\nshape is not useful. Hence, a signi\ufb01cant amount of effort in\nthe computer vision community has gone into recognizing\ntexture via tasks such as texture perception [1, 2, 12, 13]\nand description [7, 11], material recognition [26, 36, 37],\nsegmentation [20, 29], and even synthesis [9, 43].\nPerhaps the most studied task in texture understanding\nis the one of material recognition, as captured in benchmarks such as CuRET [8], KTH-TIPS [5], and, more recently, FMD [38]. However, while at least the FMD dataset\ncontains images collected from the Internet, vividly dubbed"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4017-6db4-11ee-89e4-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-2", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "contains images collected from the Internet, vividly dubbed\n\u201cimages in the wild\u201d, all these datasets make the simplifying\nbanded\n blotchy\n chequered\n grid\nmarbled\n paisley\n paisley\n wrinkled\nbrick\n ceramic\n carpet\n fabric\nFigure 1. Texture recognition in clutter . Example of top retrieved texture segments by attributes (top two rows) and materials\n(bottom) in the OpenSurfaces dataset.\nassumption that textures \ufb01ll images. Thus, they are not necessarily representative of the signi\ufb01cantly harder problem\nof recognising materials in natural images, where textures\nappear in clutter. Building on a recent dataset collected by\nthe computer graphics community, the \ufb01rst contribution of\nthis paper is a large-scale analysis of material and perceptual texture attribute recognition and segmentation in clutter(Fig. 1 and Sect. 2).\nMotivated by the challenge posed by recognising texture\nin clutter, we develop a new texture descriptor. In the simplest terms a texture is characterized by the arrangement of\nlocal patterns, as captured in early works [26, 41] by the\ndistribution of local \u201c\ufb01lter bank\u201d responses. These \ufb01lter\nbanks were designed to capture edges, spots and bars at\ndifferent scales and orientations. Typical combinations of"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4018-6db4-11ee-9c03-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-3", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "banks were designed to capture edges, spots and bars at\ndifferent scales and orientations. Typical combinations of\nthe \ufb01lter responses, identi\ufb01ed by vector quantisation, were\nused as the computational basis of the \u201ctextons\u201d proposed\nby Julesz [22]. Texton distributions were the early versions\nof \u201cbag-of-words\u201d representations, a dominant approach in\nrecognition in the early 2000s, since then improved by new\npooling schemes such as soft-assignment [27, 42, 48] and\nFisher Vectors (FVs) [32]. Until recently, FV with SIFT\nfeatures [28] as a local representation was the state-of-theart method for recognition, not only for textures, but for\nobjects and scenes too.\nLater, however, Convolutional Neural Networks (CNNs)\n1arXiv:1411.6836v2  [cs.CV]  9 Jul 2015\nhave emerged as the new state-of-the-art for recognition, exempli\ufb01ed by remarkable results in image classi\ufb01cation [23],\ndetection [14] and segmentation [16] on a number of widely\nused benchmarks. Key to their success is the ability to leverage large labelled datasets to learn increasingly complex"}


created_at: 2023-10-18 12:45:45
topic: persistent://default/default/052498be-6b77-431d-a531-3048a38fd9e3
id: 2d1a4019-6db4-11ee-9cc5-00155d3fea2c
encoding: FLOAT32
metadata:
{"uid": "1411.6836-4", "title": "Deep convolutional filter banks for texture recognition and segmentation", "source": "http://arxiv.org/pdf/1411.6836", "start_index": 0, "chroma:document": "detection [14] and segmentation [16] on a number of widely\nused benchmarks. Key to their success is the ability to leverage large labelled datasets to learn increasingly complex\ntransformations of the input to capture invariances. Importantly, CNNs pre-trained on such large datasets have been\nshown [6, 14, 30] to contain general-purpose feature extractors, transferrable to many other domains.\nDomain transfer in CNNs is usually achieved by using\nas features the output of a deep, fully-connected layer of\nthe network. From the perspective of textures, however,\nthis choice has three drawbacks. The \ufb01rst one (I) is that,\nwhile the convolutional layers are akin to non-linear \ufb01lter\nbanks, the fully connected layers capture their spatial layout. While this may be useful for representing the shape of\nan object, it may not be as useful for representing texture. A\nsecond drawback (II) is that the input to the CNN has to be\nof \ufb01xed size to be compatible with the fully connected layers, which requires an expensive resizing of the input image,\nparticularly when features are computed for many different\nregions [14, 15]. A third and more subtle drawback (III) is\nthat deeper layers may be more domain-speci\ufb01c and therefore potentially less transferrable than shallower layers."}


