query: Visual Language Modeling on CNN Image Representations

positive: Dynamic visual attention: searching for coding length increments A visual attention system should respond placidly when comm n stimuli are presented, while at the same time keep alert to anomalous visual inputs. In this paper, a dynamic visual attention model based on the rarity of featu r s is proposed. We introduce the Incremental Coding Length (ICL) to measure th e perspective entropy gain of each feature. The objective of our model is to ma xi ize the entropy of the sampled visual features. In order to optimize energy c onsumption, the limit amount of energy of the system is re-distributed among st features according to their Incremental Coding Length. By selecting featur es with large coding length increments, the computational system can achieve at tention selectivity in both static and dynamic scenes. We demonstrate that the prop osed model achieves superior accuracy in comparison to mainstream approaches i n stat c saliency map generation. Moreover, we also show that our model captures s everal less-reported dynamic visual search behaviors, such as attentional swing and inhibition of return.

positive: Deep Boltzmann Machines We present a new learning algorithm for Boltzmann machines that contain many layers of hidden variables. Data-dependent expectations are estimated using a variational approximation that tends to focus on a single mode, and dataindependent expectations are approximated using persistent Markov chains. The use of two quite different techniques for estimating the two types of expectation that enter into the gradient of the log-likelihood makes it practical to learn Boltzmann machines with multiple hidden layers and millions of parameters. The learning can be made more efficient by using a layer-by-layer “pre-training” phase that allows variational inference to be initialized with a single bottomup pass. We present results on the MNIST and NORB datasets showing that deep Boltzmann machines learn good generative models and perform well on handwritten digit and visual object recognition tasks.

positive: CAT2000: A Large Scale Fixation Dataset for Boosting Saliency Research Saliency modeling has been an active research area in computer vision for about two decades. Existing state of the art models perform very well in predicting where people look in natural scenes. There is, however, the risk that these models may have been overfitting themselves to available small scale biased datasets, thus trapping the progress in a local minimum. To gain a deeper insight regarding current issues in saliency modeling and to better gauge progress, we recorded eye movements of 120 observers while they freely viewed a large number of naturalistic and artificial images. Our stimuli includes 4000 images; 200 from each of 20 categories covering different types of scenes such as Cartoons, Art, Objects, Low resolution images, Indoor, Outdoor, Jumbled, Random, and Line drawings. We analyze some basic properties of this dataset and compare some successful models. We believe that our dataset opens new challenges for the next generation of saliency models and helps conduct behavioral studies on bottom-up visual attention.

positive:
Caffe: Convolutional Architecture for Fast Feature Embedding Caffe provides multimedia scientists and practitioners with a clean and modifiable framework for state-of-the-art deep learning algorithms and a collection of reference models. The framework is a BSD-licensed C++ library with Python and MATLAB bindings for training and deploying general-purpose convolutional neural networks and other deep models efficiently on commodity architectures. Caffe fits industry and internet-scale media needs by CUDA GPU computation, processing over 40 million images a day on a single K40 or Titan GPU (approx 2 ms per image). By separating model representation from actual implementation, Caffe allows experimentation and seamless switching among platforms for ease of development and deployment from prototyping machines to cloud environments.
 Caffe is maintained and developed by the Berkeley Vision and Learning Center (BVLC) with the help of an active community of contributors on GitHub. It powers ongoing research projects, large-scale industrial applications, and startup prototypes in vision, speech, and multimedia.

positive: On the importance of initialization and momentum in deep learning Deep and recurrent neural networks (DNNs and RNNs respectively) are powerful models that were considered to be almost impossible to train using stochastic gradient descent with momentum. In this paper, we show that when stochastic gradient descent with momentum uses a well-designed random initialization and a particular type of slowly increasing schedule for the momentum parameter, it can train both DNNs and RNNs (on datasets with long-term dependencies) to levels of performance that were previously achievable only with Hessian-Free optimization. We find that both the initialization and the momentum are crucial since poorly initialized networks cannot be trained with momentum and well-initialized networks perform markedly worse when the momentum is absent or poorly tuned. Our success training these models suggests that previous attempts to train deep and recurrent neural networks from random initializations have likely failed due to poor initialization schemes. Furthermore, carefully tuned momentum methods su ce for dealing with the curvature issues in deep and recurrent network training objectives without the need for sophisticated second-order methods.

negative: Dynamic queries for visual information seeking Considers how dynamic queries allow users to "fly through" databases by adjusting widgets and viewing the animated results. In studies, users reacted to this approach with an enthusiasm more commonly associated with video games. Adoption requires research into retrieval and display algorithms and user-interface design. The author discusses how experts may benefit from visual interfaces because they will be able to formulate more complex queries and interpret intricate results.<<ETX>>

negative: Calculating Completeness of Agile Scope in Scaled Agile Development Flexible nature of scope definition in agile makes it difficult or impossible to measure its completeness and quality. The aim of this paper is to highlight the important ingredients of scope definition for agile projects and to present a method for agile projects in order to measure the quality and completeness of their scope definitions. The proposed method considers the elements that are retrieved as a result of the systematic literature review. An industrial survey is conducted to validate and prioritize these elements. Elements are then assigned weights according to their importance in scope definition to build a scorecard for calculating the score of user stories present in the product backlog. The proposed method is able to identify the clear and complete user stories that can better be implemented in the coming iteration. Formal experiments are performed for the evaluation of the proposed method, and it suggests that the method is useful for experts in order to quantify the completeness and quality of scope definition of an agile software project.

negative: User Profile Based Research Paper Recommendation Recommender Systems (RSs) are software tools and techniques providing suggestions for items to be of use to a user. [1] For a recommender system to be effective, it has to have a comprehensive and appropriately catalogued repository of resources. Also, it has to be accurate in understanding the need of the user, and basically has to correctly profile the user. Ideally, one needs to take into consideration the fact that a user’s preferences are constantly changing and that is often not done by recommender systems. To meet this need, we propose an interactive research paper recommender system that observes the various themes that occur in a researcher’s work over time and keeps learning the user’s preferences constantly based on the user’s feedback of the papers recommended to them. We use a topic model to take into account the themes occurring in the document. The query is modeled as a bag of topics, where topics represent latent generative clusters of relative words. The entire collection or research papers is also modeled this way. We then estimate the similarity between the query and each research paper using a bag-of-topics based similarity measure and find the best ones. To take into account the users preferences, we keep track of the papers which the user likes and augment the query with the topics which recur in a lot of the users preferred papers the next time recommendations are required. We also truncate the topics which the user seems to ignore.

negative: Software Design and Architecture The once and future focus of software engineering The design of software has been a focus of software engineering research since the field's beginning. This paper explores key aspects of this research focus and shows why design will remain a principal focus. The intrinsic elements of software design, both process and product, are discussed: concept formation, use of experience, and means for representation, reasoning, and directing the design activity. Design is presented as being an activity engaged by a wide range of stakeholders, acting throughout most of a system's lifecycle, making a set of key choices which constitute the application's architecture. Directions for design research are outlined, including: (a) drawing lessons, inspiration, and techniques from design fields outside of computer science, (b) emphasizing the design of application "character" (functionality and style) as well as the application's structure, and (c) expanding the notion of software to encompass the design of additional kinds of intangible complex artifacts.

negative: Detecting Targeted Smartphone Malware with Behavior-Triggering Stochastic Models Malware for current smartphone platforms is becoming increasingly sophisticated. The presence of advanced networking and sensing functions in the device is giving rise to a new generation of targeted malware characterized by a more situational awareness, in which decisions are made on the basis of factors such as the device location, the user profile, or the presence of other apps. This complicates behavioral detection, as the analyst must reproduce very specific activation conditions in order to trigger malicious payloads. In this paper, we propose a system that addresses this problem by relying on stochastic models of usage and context events derived from real user traces. By incorporating the behavioral particularities of a given user, our scheme provides a solution for detecting malware targeting such a specific user. Our results show that the properties of these models follow a power-law distribution: a fact that facilitates an efficient generation of automatic testing patterns tailored for individual users, when done in conjunction with a cloud infrastructure supporting device cloning and parallel testing. We report empirical results with various representative case studies, demonstrating the effectiveness of this approach to detect complex activation patterns.

negative: Anaphoric Annotation in the ARRAU Corpus Arrau is a new corpus annotated for anaphoric relations, with information about agreement and explicit representation of multiple antecedents for ambiguous anaphoric expressions and discourse antecedents for expressions which refer to abstract entities such as events, actions and plans. The corpus contains texts from different genres: task-oriented dialogues from the Trains-91 and Trains-93 corpus, narratives from the English Pear Stories corpus, newspaper articles from the Wall Street Journal portion of the Penn Treebank, and mixed text from the Gnome corpus.

negative: Index Selection for OLAP On-line analytical processing (OLAP) is a recent and important application of database systems. Typically, OLAP data is presented as a multidimensional \data cube." OLAP queries are complex and can take many hours or even days to run, if executed directly on the raw data. The most common method of reducing execution time is to precompute some of the queries into summary tables (subcubes of the data cube) and then to build indexes on these summary tables. In most commercial OLAP systems today, the summary tables that are to be precomputed are picked rst, followed by the selection of the appropriate indexes on them. A trial-and-error approach is used to divide the space available between the summary tables and the indexes. This two-step process can perform very poorly. Since both summary tables and indexes consume the same resource |space | their selection should be done together for the most e cient use of space. In this paper, we give algorithms that automate the selection of summary tables and indexes. In particular, we present a family of algorithms of increasing time complexities, and prove strong performance bounds for them. The algorithms with higher complexities have better performance bounds. However, the increase in the performance bound is diminishing, and we show that an algorithm of moderate complexity can perform fairly close to the optimal. This work was supported by NSF grant IRI{92{23405, ARO grant DAAH04{95{1{0192, and Air Force Contract F33615{93{ 1{1339. Present address of V. Harinarayan and A. Rajaraman: Junglee Corp., Palo Alto, CA.

negative: My First Deep Learning System of 1991 + Deep Learning Timeline 1962-2013 Deep Learning has attracted significant attention in recent years. Here I present a brief overview of my first Deep Learner of 1991, and its historic context, with a timeline of Deep Learning highlights. Note: As a machine learning researcher I am obsessed with proper credit assignment. This draft is the result of an experiment in rapid massive open online peer review. Since 20 September 2013, subsequent revisions published under www.deeplearning.me have absorbed many suggestions for improvements by experts. The abbreviation “TL” is used to refer to subsections of the timeline section. Figure 1: My first Deep Learning system of 1991 used a deep stack of recurrent neural networks (a Neural Hierarchical Temporal Memory) pre-trained in unsupervised fashion to accelerate subsequent supervised learning [79, 81, 82]. 1 ar X iv :1 31 2. 55 48 v1 [ cs .N E ] 1 9 D ec 2 01 3 In 2009, our Deep Learning Artificial Neural Networks became the first Deep Learners to win official international pattern recognition competitions [40, 83] (with secret test sets known only to the organisers); by 2012 they had won eight of them (TL 1.13), including the first contests on object detection in large images (ICPR 2012) [2, 16] and image segmentation (ISBI 2012) [3, 15]. In 2011, they achieved the world’s first superhuman visual pattern recognition results [20, 19]. Others have implemented very similar techniques, e.g., [51], and won additional contests or set benchmark records since 2012, e.g., (TL 1.13, TL 1.14). The field of Deep Learning research is far older though—compare the timeline (TL) further down. My first Deep Learner dates back to 1991 [79, 81, 82] (TL 1.7). It can perform credit assignment across hundreds of nonlinear operators or neural layers, by using unsupervised pre-training for a stack of recurrent neural networks (RNN) (deep by nature) as in Figure 1. Such RNN are general computers more powerful than normal feedforward NN, and can encode entire sequences of input vectors. The basic idea is still relevant today. Each RNN is trained for a while in unsupervised fashion to predict its next input. From then on, only unexpected inputs (errors) convey new information and get fed to the next higher RNN which thus ticks on a slower, self-organising time scale. It can easily be shown that no information gets lost. It just gets compressed (much of machine learning is essentially about compression). We get less and less redundant input sequence encodings in deeper and deeper levels of this hierarchical temporal memory, which compresses data in both space (like feedforward NN) and time. There also is a continuous variant [84]. One ancient illustrative Deep Learning experiment of 1993 [82] required credit assignment across 1200 time steps, or through 1200 subsequent nonlinear virtual layers. The top level code of the initially unsupervised RNN stack, however, got so compact that (previously infeasible) sequence classification through additional supervised learning became possible. There is a way of compressing higher levels down into lower levels, thus partially collapsing the hierarchical temporal memory. The trick is to retrain lower-level RNN to continually imitate (predict) the hidden units of already trained, slower, higher-level RNN, through additional predictive output neurons [81, 79, 82]. This helps the lower RNN to develop appropriate, rarely changing memories that may bridge very long time lags. The Deep Learner of 1991 was a first way of overcoming the Fundamental Deep Learning Problem identified and analysed in 1991 by my very first student (now professor) Sepp Hochreiter (TL 1.6): the problem of vanishing or exploding gradients [46, 47, 10]. The latter motivated all our subsequent Deep Learning research of the 1990s and 2000s. Through supervised LSTM RNN (1997) (e.g., [48, 32, 39, 36, 37, 40, 38, 83], TL 1.8) we could eventually perform similar feats as with the 1991 system [81, 82], overcoming the Fundamental Deep Learning Problem without any unsupervised pre-training. Moreover, LSTM could also learn tasks unlearnable by the partially unsupervised 1991 chunker [81, 82]. Particularly successful are stacks of LSTM RNN [40] trained by Connectionist Temporal Classification (CTC) [36]. On faster computers of 2009, this became the first RNN system ever to win an official international pattern recognition competition [40, 83], through the work of my PhD student and postdoc Alex Graves, e.g., [40]. To my knowledge, this also was the first Deep Learning system ever (recurrent or not) to win such a contest (TL 1.10). (In fact, it won three different ICDAR 2009 contests on connected handwriting in three different languages, e.g., [83, 40], TL 1.10.) A while ago, Alex moved on to Geoffrey Hinton’s lab (Univ. Toronto), where a stack [40] of our bidirectional LSTM RNN [39] also broke a famous TIMIT speech recognition record [38] (TL 1.14), despite thousands of man years previously spent on HMM-based speech recognition research. CTC-LSTM also helped to score first at NIST’s OpenHaRT2013 evaluation [11]. Recently, well-known entrepreneurs also got interested [43, 52] in such hierarchical temporal memories [81, 82] (TL 1.7). The expression Deep Learning actually got coined relatively late, around 2006, in the context of unsupervised pre-training for less general feedforward networks [44] (TL 1.9). Such a system reached 1.2% error rate [44] on the MNIST handwritten digits [54, 55], perhaps the most famous benchmark of Machine Learning. Our team first showed that good old backpropagation (TL 1.2) on GPUs (with training pattern distortions [6, 86] but without any unsupervised pre-training) can actually achieve a three times better result of 0.35% [17] back then, a world record (a previous standard net achieved 0.7% [86]; a backprop-trained [54, 55] Convolutional NN (CNN or convnet) [29, 30, 54, 55] got 0.39% [70](TL 1.9);

negative: Event-Centric Clustering of News Articles Entertainity AB plans to build a news service to provide news to end-users in an innovative way. The service must include a way to automatically group series of news from different sources and publications, based on the stories they are covering. This thesis include three contributions: a survey of known clustering methods, an evaluation of human versus human results when grouping news articles in an event-centric manner, and last an evaluation of an incremental clustering algorithm to see if it is possible to consider a reduced input size and still get a sufficient result. The conclusions are that the result of the human evaluation indicates that users are different enough to warrant a need to take that into account when evaluating algorithms. It is also important that this difference is considered when conducting cluster analysis to avoid overfitting. The evaluation of an incremental event-centric algorithm shows it is desirable to adjust the similarity threshold, depending on what result one want. When running tests with different input sizes, the result implies that a short summary of a news article is a natural feature selection when performing cluster analysis.

negative: Can Invisible Watermarks Resolve Rightful Ownerships? Digital watermarks have been proposed in recent literature as the means for copyright protection of multimedia data. In this paper we address the capability of invisible watermarking schemes to resolve copyright ownerships. We will show that rightful ownerships cannot be resolved by current watermarking schemes alone. In addition, in the absence of standardization of watermarking procedures, anyone can claim ownership of any watermarked image. Specifically, we provide counterfeit watermarking schemes that can be performed on a watermarked image to allow multiple claims of rightful ownerships. We also propose non-invertible watermarking schemes in this paper and discuss in general the usefulness of digital watermarks in identifying the rightful copyright owners. The results, coupled with the recent attacks on some image watermarks, further imply that we have to carefully re-think our approaches to invisible watermarking of images, and re-evaluate the promises, applications and limitations of such digital means of copyright protection.

negative: Reflection from layered surfaces due to subsurface scattering The reflection of light from most materials consists of two major terms: the specular and the diffuse. Specular reflection may be modeled from first principles by considering a rough surface consisting of perfect reflectors, or micro-facets. Diffuse reflection is generally considered to result from multiple scattering either from a rough surface or from within a layer near the surface. Accounting for diffuse reflection by Lambert’s Cosine Law, as is universally done in computer graphics, is not a physical theory based on first principles. This paper presents a model for subsurface scattering in layered surfaces in terms of one-dimensional linear transport theory. We derive explicit formulas for backscattering and transmission that can be directly incorporated in most rendering systems, and a general Monte Carlo method that is easily added to a ray tracer. This model is particularly appropriate for common layered materials appearing in nature, such as biological tissues (e.g. skin, leaves, etc.) or inorganic materials (e.g. snow, sand, paint, varnished or dusty surfaces). As an application of the model, we simulate the appearance of a face and a cluster of leaves from experimental data describing their layer properties. CR

negative: Semantic mapping for mobile robotics tasks: A survey The evolution of contemporary mobile robotics has given thrust to a series of additional conjunct technologies. Of such is the semantic mapping, which provides an abstraction of space and a means for human-robot communication. The recent introduction and evolution of semantic mapping motivated this survey, in which an explicit analysis of the existing methods is sought. The several algorithms are categorized according to their primary characteristics, namely scalability, inference model, temporal coherence and topological map usage. The applications involving semantic maps are also outlined in the work at hand, emphasizing on human interaction, knowledge representation and planning. The existence of public available validation datasets and benchmarking, suitable for the evaluation of semantic mapping techniques is also discussed in detail. Last, an attempt to address open issues and questions is also made.

negative: Power Electronics Technologies for Railway Vehicles This paper describes the current range of advanced IGBT propulsion inverters used in railways from light rail vehicles to high speed trains & locomotives. The second part of the paper describes future trends in IGBT railway propulsion drives. One such trend is the concept of power integration which leads to weight, volume and cost reduction . Finally, for systems with an AC input supply, a concept for reducing the weight of the main input transformer is also described. This uses a configuration of resonant converter and a medium frequency transformer , the so called e-transformer, and could be particularly targeted for 15 kV, 16.7 Hz supplied systems.

negative: Real time face recognition engine using compact features for electronics key In this research, a design and implementation of real-time face recognition engine using compact features vector extracted by linear binary pattern (LBP) and zoned discrete cosine transforms (DCT) analysis is proposed for electronics key. The function of LBP is to normalize the lighting variations of the input face image and the function of zoned DCT is to define the local descriptors of the face image. In order to get more compact features vector, the predictive LDA is employed for dimensional reduction. The aims of this research is to develop fast and strong face recognition that can be implemented for electronic key which will be implemented for substituting current security system (PIN and password). In addition, the recognition engine is also designed for real time face recognition which can work on hardware having limited resources such as Intel atom computer, raspberry pi, and android smart phone. The experimental results show that the proposed engine provide high enough recognition rate, and small false rejection rate (FRR) and false acceptance rate (FAR). In addition, the engine needs short processing time.

negative: "It's alive, it's magic, it's in love with you": opportunities, challenges and open questions for actuated interfaces Actuated Interfaces are receiving a great deal of interest from the research community. The field can now present a range of point designs, illustrating the potential design space of Actuated Interfaces. However, despite the increasing interest in Actuated Interfaces, the research carried out is nevertheless primarily preoccupied with the technical challenges and potential application areas, rather than how users actually approach, experience, interpret and understand Actuated Interfaces. Based on three case studies, investigating how people experience Actuated Interfaces, we point to; magic, movement and ambiguity as fruitful perspectives for understanding users' experiences with Actuated Interfaces. The three perspectives are employed to reflect upon opportunities and challenges, as well as point to open questions and relevant areas for future research for Actuated Interfaces.

negative: Modeling ITIL Business Motivation Model in ArchiMate According to Enterprise Architecture (EA) approaches, organizations have motivational concepts that are used to model the motivations that underlie its design or change, which represent the organization’s Business Motivation Model (BMM). Likewise, this BMM is also present in the organizations who provide IT services. ITIL has become a reference for IT service providers, but is commonly modeled as a process-oriented approach to IT Service Management, often disregarding the remaining EA domains or its motivational elements. Conversely, we believe that like EA, ITIL has an important motivation model that should be formally represented.

negative: The use of unlabeled data to improve supervised learning for text summarization With the huge amount of information available electronically, there is an increasing demand for automatic text summarization systems. The use of machine learning techniques for this task allows one to adapt summaries to the user needs and to the corpus characteristics. These desirable properties have motivated an increasing amount of work in this field over the last few years. Most approaches attempt to generate summaries by extracting sentence segments and adopt the supervised learning paradigm which requires to label documents at the text span level. This is a costly process, which puts strong limitations on the applicability of these methods. We investigate here the use of semi-supervised algorithms for summarization. These techniques make use of few labeled data together with a larger amount of unlabeled data. We propose new semi-supervised algorithms for training classification models for text summarization. We analyze their performances on two data sets - the Reuters news-wire corpus and the Computation and Language (cmp_lg) collection of TIPSTER SUMMAC. We perform comparisons with a baseline - non learning - system, and a reference trainable summarizer system.

negative: Kinematics analysis of a novel 4-dof stabilized platform This paper presents a novel stabilized platform with three UPS-type and one RPU-type active limbs based on the shipboard helicopter's landing process and its kinematics are studied systematically. First, a prototype of this stabilized platform is constructed and its dof is analyzed, Second, analytic formulas for solving the inverse displacement kinematics are derived Third, analytic formulas for solving the forward displacement kinematics are derived and verified. Theoretical formulas and results provide a basis for the further optimization, control, and manufacturing.

negative: Neither Quick Nor Proper - Evaluation of QuickProp for Learning Deep Neural Networks Neural networks and especially convolutional neural networks are of great interest in current computer vision research. However, many techniques, extensions, and modifications have been published in the past, which are not yet used by current approaches. In this paper, we study the application of a method called QuickProp for training of deep neural networks. In particular, we apply QuickProp during learning and testing of fully convolutional networks for the task of semantic segmentation. We compare QuickProp empirically with gradient descent, which is the current standard method. Experiments suggest that QuickProp can not compete with standard gradient descent techniques for complex computer vision tasks like semantic segmentation.

negative: Triple-Wideband Open-Slot Antenna for the LTE Metal-Framed Tablet device An open-slot antenna with a low profile of 7 mm to provide a triple-wideband LTE operation in the metal-framed tablet device is presented. With the low profile, the antenna can fit in the narrow region between the metal frame and the display panel of the metal-framed tablet device. The triple-wideband operation covers 698-960 MHz (low band), 1710-2690 MHz (middle band), and 3400-3800 MHz (high band). The antenna mainly has a rectangular open slot and an L-shaped metal strip embedded therein to provide two open slots of different resonant paths. Using a microstrip feedline having a step-shaped section across the two open slots, good excitation of multiple slot resonant modes is obtained. Details of the proposed antenna are presented.

negative: Static analysis tools as early indicators of pre-release defect density During software development it is helpful to obtain early estimates of the defect density of software components. Such estimates identify fault-prone areas of code requiring further testing. We present an empirical approach for the early prediction of pre-release defect density based on the defects found using static analysis tools. The defects identified by two different static analysis tools are used to fit and predict the actual pre-release defect density for Windows Server 2003. We show that there exists a strong positive correlation between the static analysis defect density and the pre-release defect density determined by testing. Further, the predicted pre-release defect density and the actual pre-release defect density are strongly correlated at a high degree of statistical significance. Discriminant analysis shows that the results of static analysis tools can be used to separate high and low quality components with an overall classification rate of 82.91%.

negative: LTSG: Latent Topical Skip-Gram for Mutually Improving Topic Model and Vector Representations Topic models have been widely used in discovering latent topics which are shared across documents in text mining. Vector representations, word embeddings and topic embeddings, map words and topics into a low-dimensional and dense real-value vector space, which have obtained high performance in NLP tasks. However, most of the existing models assume the results trained by one of them are perfect correct and used as prior knowledge for improving the other model. Some other models use the information trained from external large corpus to help improving smaller corpus. In this paper, we aim to build such an algorithm framework that makes topic models and vector representations mutually improve each other within the same corpus. An EM-style algorithm framework is employed to iteratively optimize both topic model and vector representations. Experimental results show that our model outperforms state-of-the-art methods on various NLP tasks.

negative: Hierarchical Classification of Ten Skin Lesion Classes This paper presents a hierarchical classification system based on the kNearest Neighbors (kNN) classifier for classification of ten different classes of Malignant and Benign skin lesions from color image data. Our key contribution is to focus on the ten most common classes of skin lesions. There are five malignant: Actinic Keratosis (AK), Basal Cell Carcinoma (BCC), Squamous Cell Carcinoma (SCC), Melanoma (MEL), Intraepithelial Carcinoma (IEC) and five benign: Melanocytic Nevus / Mole (ML), Seborrhoeic Keratosis (SK), Dermatofibroma (DF), Haemangioma (VASC), Pyogenic Granuloma (PYO). Moreover, we use only high resolution color images acquired using a standard camera (nondermoscopy). Our image dataset contains 1300 lesions belonging to ten classes (45 AK, 239 BCC, 331 ML, 88 SCC, 257 SK, 76 MEL, 65 DF, 97 VASC, 24 PYO and 78 IEC).

negative: Visual saliency detection based on Bayesian model Image saliency detection is very useful in many computer vision tasks while it still remains a challenging problem. In this paper, we propose a new computational saliency detection model which is implemented with a coarse to fine strategy under the Bayesian framework. First, saliency points are applied to get a coarse location of the saliency region. And then, based on the rough region, we compute a prior map for the Bayesian model to achieve the final saliency map. Experimental results on a public available dataset show the effectiveness of the proposed prior map and the strength of our saliency map compared with several previous method.

negative: Tangible Query Interfaces: Physically Constrained Tokens for Manipulating Database Queries We present a new approach for using physically constrained tokens to express, manipulate, and visualize parameterized database queries. This method extends tangible interfaces to enable interaction with large aggregates of information. We describe two interface prototypes that use physical tokens to represent database parameters. These tokens are manipulated upon physical constraints, which map compositions of tokens onto interpretations including database queries, views, and Boolean operations. We propose a framework for “ token + constraint” interfaces, and compare one of our prototypes with a comparable graphical interface in a preliminary user study. † Current affiliation: Visualization Dept., Zuse Institute Berlin (ZIB), ullmer@zib.de †† Current affiliation: Computer Science Dept., Tufts University, jacob@cs.tufts.edu

negative: Semantics in Visual Information Retrieval V isual information retrieval systems have entered a new era. First-generation systems allowed access to images and videos through textual data. 1,2 Typical searches for these systems include, for example, " all images of paintings of the Florentine school of the 15th century " or " all images by Cezanne with landscapes. " Such systems expressed information through alphanumeric keywords or scripts. They employed representation schemes like relational models, frame models, and object-oriented models. On the other hand, current-generation retrieval systems support full retrieval by visual content. 3,4 Access to visual information is not only performed at a conceptual level, using keywords as in the textu-al domain, but also at a perceptual level, using objective measurements of visual content. In these systems, image processing, pattern recognition , and computer vision constitute an integral part of the system's architecture and operation. They objectively analyze pixel distribution and extract the content descriptors automatically from raw sensory data. Image content descriptors are commonly represented as feature vectors, whose elements correspond to significant parameters that model image attributes. Therefore, visual attributes are regarded as points in a multidimen-sional feature space, where point closeness reflects feature similarity. These advances (for comprehensive reviews of the field, see the " Further Reading " sidebar) have paved the way for third-generation systems, featuring full multimedia data management and networking support. Forthcoming standards such as MPEG-4 and MPEG-7 (see the Nack and Lindsay article in this issue) provide the framework for efficient representation, processing, and retrieval of visual information. Yet many problems must still be addressed and solved before these technologies can emerge. An important issue is the design of indexing structures for efficient retrieval from large, possibly distributed , multimedia data repositories. To achieve this goal, image and video content descriptors can be internally organized and accessed through multidimensional index structures. 5 A second key problem is to bridge the semantic gap between the system and users. That is, devise representations capturing visual content at high semantic levels especially relevant for retrieval tasks. Specifically, automatically obtaining a representation of high-level visual content remains an open issue. Virtually all the systems based on automatic storage and retrieval of visual information proposed so far use low-level perceptual representations of pictorial data, which have limited semantics. Building up a representation proves tantamount to defining a model of the world, possibly through a formal description language, whose semantics capture only a few …

negative: Tutorial on Answering Questions about Images with Deep Learning Together with the development of more accurate methods in Computer Vision and Natural Language Understanding, holistic architectures that answer on questions about the content of real-world images have emerged. In this tutorial, we build a neural-based approach to answer questions about images. We base our tutorial on two datasets: (mostly on) DAQUAR, and (a bit on) VQA. With small tweaks the models that we present here can achieve a competitive performance on both datasets, in fact, they are among the best methods that use a combination of LSTM with a global, full frame CNN representation of an image. We hope that after reading this tutorial, the reader will be able to use Deep Learning frameworks, such as Keras and introduced Kraino, to build various architectures that will lead to a further performance improvement on this challenging task. 1 Preface In this tutorial1 we build a few architectures that can answer questions about images. The architectures are based on our two papers on this topic: Malinowski et al. [2015] and Malinowski et al. [2016]; and more broadly, on our project towards a Visual Turing Test2. In particular, an encoder-decoder perspective of Malinowski et al. [2016] allows us to effectively experiment with various design choices. For the sake of simplicity, we only consider a classification-based approach to answer questions about images, although an approach that generate answers word-by-word is also studied in the community [Malinowski et al., 2015]. In the tutorial, we mainly focus on the DAQUAR dataset [Malinowski and Fritz, 2014a], but a few possible directions to apply learnt techniques to VQA [Antol et al., 2015] are also pointed. First, we will get familiar with the task of answering questions about images, and a dataset that implements the task (due to a small size, we mainly use DAQUAR as it better serves an educational purpose that we aim at this tutorial). Next, we build a few blind models that answer questions about images without actually seeing such images. Such models already exhibit a reasonable performance as they can effectively learn various biases that exist in a dataset, which we also interpret as learning a common sense knowledge [Malinowski et al., 2015, 2016]. Subsequently, we build a few language+vision models that answer questions based on both a textual and a visual inputs. Finally, we leave the tutorial with a few possible research directions. Technical aspects The tutorial is originally written using Python Notebook, which the reader is welcome to download and use through the tutorial. Instructions necessary to run the Notebook version of this tutorial are provided in the following: https://github.com/mateuszmalinowski/visual_turing_test-tutorial. In this tutorial, we heavily use a Python code, and therefore it is expected the reader either already knows this language, or can quickly learn it. However, we made an effort to make this tutorial approachable to a wider audience. We use Kraino that is a framework prepared for this tutorial in order to simplify the development of the question answering architectures. Under the hood, it uses Theano4 [Bastien et al., 2012] and Keras5 [Chollet, 2015] – two frameworks to build Deep Learning models. We also use various CNNs representations extracted from images that can be downloaded as explained at the beginning of our Notebook tutorial. We also highlight exercises that a curious reader may attempt to solve. This tutorial was presented for the first time during the 2nd Summer School on Integrating Vision and Language: Deep Learning. http://mpii.de/visual_turing_test https://github.com/mateuszmalinowski/visual_turing_test-tutorial/blob/master/visual_ turing_test.ipynb http://deeplearning.net/software/theano/ https://keras.io 1 ar X iv :1 61 0. 01 07 6v 1 [ cs .C V ] 4 O ct 2 01 6

negative: Texture Segmentation Based Video Compression Using Convolutional Neural Networks There has been a growing interest in using different approaches to improve the coding efficiency of modern video codec in recent years as demand for web-based video consumption increases. In this paper, we propose a model-based approach that uses texture analysis/synthesis to reconstruct blocks in texture regions of a video to achieve potential coding gains using the AV1 codec developed by the Alliance for Open Media (AOM). The proposed method uses convolutional neural networks to extract texture regions in a frame, which are then reconstructed using a global motion model. Our preliminary results show an increase in coding efficiency while maintaining satisfactory visual quality. Introduction With the increasing amount of videos being created and consumed, better video compression tools are needed to provide fast transmission and high visual quality. Modern video coding standards utilize spatial and temporal redundancy in the videos to achieve high coding efficiency and high visual quality with motion compensation techniques and 2-D orthogonal transforms. However, efficient exploitation of statistical dependencies measured by a mean squared error (MSE) does not always produce the best psychovisual result, and may require higher data rate to preserve detail information in the video. Recent advancement in GPU computing has enabled the analysis of large scale data using deep learning method. Deep learning techniques have shown promising performance in many applications such as object detection, natural language process, and synthetic images generation [1, 2, 3, 4]. Several methods have been developed for video applications to improve coding efficiency using deep learning. In [5], sample adaptive offset (SAO) is replaced by a CNN-based in-loop filter (IFCNN) to improve the coding efficiency in HEVC. By learning the predicted residue between the quantized reconstructed frames obtained after deblocking filter (DF) and the original frames, IFCNN is able to reconstruct video frames with higher quality without requiring any bit transmission during coding process. Similar to [5], [6] proposes a Variable-filter-size Residue-learning CNN (VRCNN) to improving coding efficiency by replacing DF and SAO in HEVC. VRCNN is based on the concept of ARCNN [7] which is originally designed for JPEG applications. Instead of only using spatial information to train a CNN to reduce the coding artifacts in HEVC, [8] proposed a spatial temporal residue network (STResNet) as an additional in-loop filter after SAO. A rate-distortion optimization strategy is used to control the on/off switch of the proposed in-loop filter. There are also some works that have been done in the decoder of HEVC to improve the coding efficiency. In [9], a deep CNN-based auto decoder (DCAD) is implemented in the decoder of HEVC to improve the video quality of decoded video. DCAD is trained to learn the predict residue between decoded video frames and original video frames. By adding the predicted residual generated from DCAD to the compressed video frames, this method enhances the compressed video frames to higher quality. In summary, the above methods improve the coding efficiency by enhancing the quality of reconstructed video frames. However, they require different trained models for video reconstruction at different quantization levels. We are interested in developing deep learning approaches to only encode visually relevant information and use a different coding method for “perceptually insignificant” regions in a frame, which can lead to substantial data rate reductions while maintaining visual quality. In particular, we have developed a model based approach that can be used to improve the coding efficiency by identifying texture areas in a video frame that contain detail irrelevant information, which the viewer does not perceive specific details and can be skipped or encoded at a much lower data rate. The task is then to divide a frame into “perceptually insignificant” texture region and then use a texture model for the pixels in that region. In 1959, Schreiber and colleagues proposed a coding method that divides an image into textures and edges and used it in image coding [10]. This work was later extended by using the human visual system and statistical model to determine the texture region [11, 12, 13]. More recently, several groups have focused on adapting perceptual based approaches to the video coding framework [14]. In our previous work [15], we introduced a texture analyzer before encoding the input sequences to identify detail irrelevant regions in the frame which are classified into different texture classes. At the encoder, no inter-frame prediction is performed for these regions. Instead, displacement of the entire texture region is modeled by just one set of motion parameters. Therefore, only the model parameters are transmitted to the decoder for reconstructing the texture regions using a texture synthesizer. Non-texture regions in the frame are coded conventionally. Since this method uses feature extraction based on texture segmentation technique, a proper set of parameters are required to achieve accurate texture segmentation for different videos. Deep learning methods usually do not require such parameter tuning for inference. As a result, deep learning techniques can be developed to perform texture segmentation and classification for the proposed model-based video coding. A Fisher vector convolution neural networks (FVCNN) that can produce segmentation labels for different texture classes was proposed in [16]. One of the advantage of FV-CNN is that the image input size is flexible and is not limited by the network architecture. Instead of doing pixel-wise classification on texture regions, a texture classification CNN network was described in [17]. To reduce computational expenses, [17] uses a small classification network to classify image patches with size of 227 × 227. A smaller network is needed to classify smaller image patches in our case. In this paper, we propose a block-based texture segmentation method to extract texture region in a video frame using convolutional neural networks. The block-based segmentation network classifies each 16 × 16 block in a frame as texture or non-texture. The identified texture region is then synthesized using the temporal correlations among the frames. Our method was implemented using the AOM/AV1 codec. Preliminary results show significant bitrate savings while maintaining a satisfactory visual quality.

negative: Learning to Read Chest X-Ray Images from 16000+ Examples Using CNN Chest radiography (chest X-ray) is a low-cost yet effective and widely used medical imaging procedures. The lacking of qualified radiologist seriously limits the applicability of the technique. We explore the possibility of designing a computer-aided diagnosis for chest X-rays using deep convolutional neural networks. Using a real-world dataset of 16,000 chest X-rays with natural language diagnosis reports, we can train a multi-class classification model from images and preform accurate diagnosis, without any prior domain knowledge.

negative: Unsupervised learning of invariant representations with low sample complexity: the magic of sensory cortex or a new framework for machine learning? The present phase of Machine Learning is characterized by supervised learning algorithms relying on large sets of labeled examples (n→∞). The next phase is likely to focus on algorithms capable of learning from very few labeled examples (n → 1), like humans seem able to do. We propose an approach to this problem and describe the underlying theory, based on the unsupervised, automatic learning of a “good” representation for supervised learning, characterized by small sample complexity (n). We consider the case of visual object recognition though the theory applies to other domains. The starting point is the conjecture, proved in specific cases, that image representations which are invariant to translations, scaling and other transformations can considerably reduce the sample complexity of learning. We prove that an invariant and unique (discriminative) signature can be computed for each image patch, I, in terms of empirical distributions of the dot-products between I and a set of templates stored during unsupervised learning. A module performing filtering and pooling, like the simple and complex cells described by Hubel and Wiesel, can compute such estimates. Hierarchical architectures consisting of this basic Hubel-Wiesel moduli inherit its properties of invariance, stability, and discriminability while capturing the compositional organization of the visual world in terms of wholes and parts. The theory extends existing deep learning convolutional architectures for image and speech recognition. It also suggests that the main computational goal of the ventral stream of visual cortex is to provide a hierarchical representation of new objects/images which is invariant to transformations, stable, and discriminative for recognition—and that this representation may be continuously learned in an unsupervised way during development and visual experience. This work was supported by the Center for Brains, Minds and Machines (CBMM), funded by NSF STC award CCF 1231216. http://cbmm.mit.edu CBMM Paper March, 2014 1–24 ar X iv :1 31 1. 41 58 v5 [ cs .C V ] 1 1 M ar 2 01 4 Unsupervised learning of invariant representations with low sample complexity: the magic of sensory cortex or a new framework for machine learning? Fabio Anselmi ∗ †, Joel Z. Leibo ∗ , Lorenzo Rosasco ∗ † , Jim Mutch ∗ , Andrea Tacchetti ∗ † , and Tomaso Poggio ∗ † ∗Center for Brains, Minds and Machines, Massachusetts Institute of Technology, Cambridge, MA 02139, and †Istituto Italiano di Tecnologia, Genova, 16163 The present phase of Machine Learning is characterized by supervised learning algorithms relying on large sets of labeled examples (n→∞). The next phase is likely to focus on algorithms capable of learning from very few labeled examples (n → 1), like humans seem able to do. We propose an approach to this problem and describe the underlying theory, based on the unsupervised, automatic learning of a “good” representation for supervised learning, characterized by small sample complexity (n). We consider the case of visual object recognition though the theory applies to other domains. The starting point is the conjecture, proved in specific cases, that image representations which are invariant to translations, scaling and other transformations can considerably reduce the sample complexity of learning. We prove that an invariant and unique (discriminative) signature can be computed for each image patch, I, in terms of empirical distributions of the dot-products between I and a set of templates stored during unsupervised learning. A module performing filtering and pooling, like the simple and complex cells described by Hubel and Wiesel, can compute such estimates. Hierarchical architectures consisting of this basic Hubel-Wiesel moduli inherit its properties of invariance, stability, and discriminability while capturing the compositional organization of the visual world in terms of wholes and parts. The theory extends existing deep learning convolutional architectures for image and speech recognition. It also suggests that the main computational goal of the ventral stream of visual cortex is to provide a hierarchical representation of new objects/images which is invariant to transformations, stable, and discriminative for recognition—and that this representation may be continuously learned in an unsupervised way during development and visual experience.1 Invariance | Hierarchy | Convolutional networks | Visual cortex It is known that Hubel and Wiesel’s original proposal [1] for visual area V1—of a module consisting of complex cells (C-units) combining the outputs of sets of simple cells (S-units) with identical orientation preferences but differing retinal positions—can be used to construct translationinvariant detectors. This is the insight underlying many networks for visual recognition, including HMAX [2] and convolutional neural nets [3, 4]. We show here how the original idea can be expanded into a comprehensive theory of visual recognition relevant for computer vision and possibly for visual cortex. The first step in the theory is the conjecture that a representation of images and image patches, with a feature vector that is invariant to a broad range of transformations—such as translation, scale, expression of a face, pose of a body, and viewpoint—makes it possible to recognize objects from only a few labeled examples, as humans do. The second step is proving that hierarchical architectures of Hubel-Wiesel (‘HW’) modules (indicated by ∧ in Fig. 1) can provide such invariant representations while maintaining discriminative information about the original image. Each ∧ -module provides a feature vector, which we call a signature, for the part of the visual field that is inside its “receptive field”; the signature is invariant to (R) affine transformations within the receptive field. The hierarchical architecture, since it computes a set of signatures for l=4

negative:
A deep learning method for classifying mammographic breast density categories. PURPOSE
Mammographic breast density is an established risk marker for breast cancer and is visually assessed by radiologists in routine mammogram image reading, using four qualitative Breast Imaging and Reporting Data System (BI-RADS) breast density categories. It is particularly difficult for radiologists to consistently distinguish the two most common and most variably assigned BI-RADS categories, i.e., "scattered density" and "heterogeneously dense". The aim of this work was to investigate a deep learning-based breast density classifier to consistently distinguish these two categories, aiming at providing a potential computerized tool to assist radiologists in assigning a BI-RADS category in current clinical workflow.

METHODS
In this study, we constructed a convolutional neural network (CNN)-based model coupled with a large (i.e., 22,000 images) digital mammogram imaging dataset to evaluate the classification performance between the two aforementioned breast density categories. All images were collected from a cohort of 1,427 women who underwent standard digital mammography screening from 2005 to 2016 at our institution. The truths of the density categories were based on standard clinical assessment made by board-certified breast imaging radiologists. Effects of direct training from scratch solely using digital mammogram images and transfer learning of a pretrained model on a large nonmedical imaging dataset were evaluated for the specific task of breast density classification. In order to measure the classification performance, the CNN classifier was also tested on a refined version of the mammogram image dataset by removing some potentially inaccurately labeled images. Receiver operating characteristic (ROC) curves and the area under the curve (AUC) were used to measure the accuracy of the classifier.

RESULTS
The AUC was 0.9421 when the CNN-model was trained from scratch on our own mammogram images, and the accuracy increased gradually along with an increased size of training samples. Using the pretrained model followed by a fine-tuning process with as few as 500 mammogram images led to an AUC of 0.9265. After removing the potentially inaccurately labeled images, AUC was increased to 0.9882 and 0.9857 for without and with the pretrained model, respectively, both significantly higher (P < 0.001) than when using the full imaging dataset.

CONCLUSIONS
Our study demonstrated high classification accuracies between two difficult to distinguish breast density categories that are routinely assessed by radiologists. We anticipate that our approach will help enhance current clinical assessment of breast density and better support consistent density notification to patients in breast cancer screening.