query: Supervised Learning of Universal Sentence Representations from Natural Language Inference Data

positive: Multimodal Convolutional Neural Networks for Matching Image and Sentence In this paper, we propose multimodal convolutional neural networks (m-CNNs) for matching image and sentence. Our m-CNN provides an end-to-end framework with convolutional architectures to exploit image representation, word composition, and the matching relations between the two modalities. More specifically, it consists of one image CNN encoding the image content and one matching CNN modeling the joint representation of image and sentence. The matching CNN composes different semantic fragments from words and learns the inter-modal relations between image and the composed fragments at different levels, thus fully exploit the matching relations between image and sentence. Experimental results demonstrate that the proposed m-CNNs can effectively capture the information necessary for image and sentence matching. More specifically, our proposed m-CNNs significantly outperform the state-of-the-art approaches for bidirectional image and sentence retrieval on the Flickr8K and Flickr30K datasets.

positive: A SICK cure for the evaluation of compositional distributional semantic models Shared and internationally recognized benchmarks are fundamental for the development of any computational system. We aim to help the research community working on compositional distributional semantic models (CDSMs) by providing SICK (Sentences Involving Compositional Knowldedge), a large size English benchmark tailored for them. SICK consists of about 10,000 English sentence pairs that include many examples of the lexical, syntactic and semantic phenomena that CDSMs are expected to account for, but do not require dealing with other aspects of existing sentential data sets (idiomatic multiword expressions, named entities, telegraphic language) that are not within the scope of CDSMs. By means of crowdsourcing techniques, each pair was annotated for two crucial semantic tasks: relatedness in meaning (with a 5-point rating scale as gold score) and entailment relation between the two elements (with three possible gold labels: entailment, contradiction, and neutral). The SICK data set was used in SemEval-2014 Task 1, and it freely available for research purposes.

positive: Natural Language Processing (almost) from Scratch We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including: part-of-speech tagging, chunking, named entity recognition, and semantic role labeling. This versatility is achieved by trying to avoid task-specific engineering and therefore disregarding a lot of prior knowledge. Instead of exploiting man-made input features carefully optimized for each task, our system learns internal representations on the basis of vast amounts of mostly unlabeled training data. This work is then used as a basis for building a freely available tagging system with good performance and minimal computational requirements.

positive: Adam: A Method for Stochastic Optimization We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. The method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or parameters. The method is also appropriate for non-stationary objectives and problems with very noisy and/or sparse gradients. The hyper-parameters have intuitive interpretations and typically require little tuning. Some connections to related algorithms, on which Adam was inspired, are discussed. We also analyze the theoretical convergence properties of the algorithm and provide a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework. Empirical results demonstrate that Adam works well in practice and compares favorably to other stochastic optimization methods. Finally, we discuss AdaMax, a variant of Adam based on the infinity norm.

positive: Deep Residual Learning for Image Recognition Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers - 8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.

negative: IJ CS Differences in Individualistic and Collectivistic Tendencies among College Students in Japan and the United States Emiko It is a worldwide stereotype that Japanese, compared to Americans, are oriented more toward collectivism. But this stereotypical notion of more collectivism among Japanese, which typically stems from a view that individualism and collectivism stand at opposite ends of a continuum, has been filled with dashed empirical findings, especially in a sample of college students. In the current study, following the view that individualism and collectivism are two separate concepts rather than one with two extremes, we test and compare both individualistic and collectivistic tendencies among college students in Japan and the United States. A review of theories and research on this dimension of cultural variability across the two diverse cultures and the literature on societal pressure of collectivity and on parents as primary socialization agents of culturally expected values lead to two hypotheses: 1) Japanese college students tend less toward individualism than do Americans, and 2) Japanese college students tend less toward collectivism than do Americans. Analysis of identical survey data from college students in Japan and in the United States provides strong support for both hypotheses.

negative: Autonomous robotic system for bridge deck data collection and analysis Bridge deck inspection is conducted to identify bridge condition deterioration and, thus, to facilitate implementation of appropriate maintenance or rehabilitation procedures. In this paper, we report the development of a robotic system for bridge deck data collection and analysis. The robotic system accurately localizes itself and autonomously maneuvers on the bridge deck to collect visual images and conduct nondestructive evaluation (NDE) measurements. The developed robotic system can reduce the cost and time of the bridge deck data collection. Crack detection and mapping algorithm to build the deck crack maps is presented in detail. The electrical resistivity (ER), impact-echo (IE) and ultrasonic surface waves (USW) data collected by the robot are analyzed to generate the corrosion, delamination and concrete elastic modulus maps of the deck. The presented robotic system has been successfully deployed to inspect numerous bridges.

negative: Effects of Harsh and Unpredictable Environments in Adolescence on Development of Life History Strategies: A Longitudinal Test of an Evolutionary Model. The National Longitudinal Study of Adolescent Health data were used to test predictions from life history theory. We hypothesized that (1) in young adulthood an emerging life history strategy would exist as a common factor underlying many life history traits (e.g., health, relationship stability, economic success), (2) both environmental harshness and unpredictability would account for unique variance in expression of adolescent and young adult life history strategies, and (3) adolescent life history traits would predict young adult life history strategy. These predictions were supported. The current findings suggest that the environmental parameters of harshness and unpredictability have concurrent effects on life history development in adolescence, as well as longitudinal effects into young adulthood. In addition, life history traits appear to be stable across developmental time from adolescence into young adulthood.

negative: Pattern Recognition Systems under Attack: Design Issues and Research Challenges We analyze the problem of designing pattern recognition systems in adversarial settings, under an engineering viewpoint, motivated by their increasing exploitation in security-sensitive applications like spam and malware detection, despite their vulnerability to potential attacks has not yet been deeply understood. We first review previous work and report examples of how a complex system may be evaded either by leveraging on trivial vulnerabilities of its untrained components, e.g., parsing errors in the pre-processing steps, or by exploiting more subtle vulnerabilities of learning algorithms. We then discuss the need of exploiting both reactive and proactive security paradigms complementarily to improve the security by design. Our ultimate goal is to provide some useful guidelines for improving the security of pattern recognition in adversarial settings, and to suggest related open issues to foster research in this area.

negative: Security, privacy and safety evaluation of dynamic and static fleets of drones Interconnected everyday objects, either via public or private networks, are gradually becoming reality in modern life — often referred to as the Internet of Things (IoT) or Cyber-Physical Systems (CPS). One stand-out example are those systems based on Unmanned Aerial Vehicles (UAVs). Fleets of such vehicles (drones) are prophesied to assume multiple roles from mundane to high-sensitive applications, such as prompt pizza or shopping deliveries to the home, or to deployment on battlefields for battlefield and combat missions. Drones, which we refer to as UAVs in this paper, can operate either individually (solo missions) or as part of a fleet (group missions), with and without constant connection with a base station. The base station acts as the command centre to manage the drones' activities; however, an independent, localised and effective fleet control is necessary, potentially based on swarm intelligence, for several reasons: 1) an increase in the number of drone fleets; 2) fleet size might reach tens of UAVs; 3) making time-critical decisions by such fleets in the wild; 4) potential communication congestion and latency; and 5) in some cases, working in challenging terrains that hinders or mandates limited communication with a control centre, e.g. operations spanning long period of times or military usage of fleets in enemy territory. This self-aware, mission-focused and independent fleet of drones may utilise swarm intelligence for a), air-traffic or flight control management, b) obstacle avoidance, c) self-preservation (while maintaining the mission criteria), d) autonomous collaboration with other fleets in the wild, and e) assuring the security, privacy and safety of physical (drones itself) and virtual (data, software) assets. In this paper, we investigate the challenges faced by fleet of drones and propose a potential course of action on how to overcome them.

negative: Pharmaceutical cold chain management: Platform based on a distributed ledger In the process of distributing pharmaceutical products multiple stages are involved until products are delivered to patient. A reliable way for validating and verifying that products had been maintained within a licensed range is required. The paper presents a solution for pharmaceutical cold chain management using distributed ledger technologies. An application framework is proposed for shipment tracking which will deliver information to all stakeholders during the distribution phase of pharmaceutical products. The solution is based on Hyperledger Sawtooth distributed ledger framework, which has been extended with a custom transactions family and a sensors gateway for automatically collecting data from temperature tracking devices. The focus of the paper is to describe the data model and how entities of the system communicate.

negative: A retrospective of knowledge graphs Information on the Internet is fragmented and presented in different data sources, which makes automatic knowledge harvesting and understanding formidable for machines, and even for humans. Knowledge graphs have become prevalent in both of industry and academic circles these years, to be one of the most efficient and effective knowledge integration approaches. Techniques for knowledge graph construction can mine information from either structured, semi-structured, or even unstructured data sources, and finally integrate the information into knowledge, represented in a graph. Furthermore, knowledge graph is able to organize information in an easy-to-maintain, easy-to-understand and easy-to-use manner. In this paper, we give a summarization of techniques for constructing knowledge graphs. We review the existing knowledge graph systems developed by both academia and industry. We discuss in detail about the process of building knowledge graphs, and survey state-of-the-art techniques for automatic knowledge graph checking and expansion via logical inferring and reasoning. We also review the issues of graph data management by introducing the knowledge data models and graph databases, especially from a NoSQL point of view. Finally, we overview current knowledge graph systems and discuss the future research directions.

negative: Sentiment analysis using Telugu SentiWordNet In recent times, sentiment analysis in low resourced languages and regional languages has become emerging areas in natural language processing. Researchers have shown greater interest towards analyzing sentiment in Indian languages such as Hindi, Telugu, Tamil, Bengali, Malayalam, etc. In best of our knowledge, microscopic work has been reported till date towards Indian languages due to lack of annotated data set. In this paper, we proposed a two-phase sentiment analysis for Telugu news sentences using Telugu SentiWordNet. Initially, it identifies subjectivity classification where sentences are classified as subjective or objective. Objective sentences are treated as neutral sentiment as they don't carry any sentiment value. Next, Sentiment Classification has been done where the subjective sentences are further classified into positive and negative sentences. With the existing Telugu SentiWordNet, our proposed system attains an accuracy of 74% and 81% for subjectivity and sentiment classification respectively.

negative: Combating VR sickness through subtle dynamic field-of-view modification Virtual Reality (VR) sickness can cause intense discomfort, shorten the duration of a VR experience, and create an aversion to further use of VR. High-quality tracking systems can minimize the mismatch between a user's visual perception of the virtual environment (VE) and the response of their vestibular system, diminishing VR sickness for moving users. However, this does not help users who do not or cannot move physically the way they move virtually, because of preference or physical limitations such as a disability. It has been noted that decreasing field of view (FOV) tends to decrease VR sickness, though at the expense of sense of presence. To address this tradeoff, we explore the effect of dynamically, yet subtly, changing a physically stationary person's FOV in response to visually perceived motion as they virtually traverse a VE. We report the results of a two-session, multi-day study with 30 participants. Each participant was seated in a stationary chair, wearing a stereoscopic head-worn display, and used control and FOV-modifying conditions in the same VE. Our data suggests that by strategically and automatically manipulating FOV during a VR session, we can reduce the degree of VR sickness perceived by participants and help them adapt to VR, without decreasing their subjective level of presence, and minimizing their awareness of the intervention.

negative: Advances in rapid detection methods for foodborne pathogens. Food safety is increasingly becoming an important public health issue, as foodborne diseases present a widespread and growing public health problem in both developed and developing countries. The rapid and precise monitoring and detection of foodborne pathogens are some of the most effective ways to control and prevent human foodborne infections. Traditional microbiological detection and identification methods for foodborne pathogens are well known to be time consuming and laborious as they are increasingly being perceived as insufficient to meet the demands of rapid food testing. Recently, various kinds of rapid detection, identification, and monitoring methods have been developed for foodborne pathogens, including nucleic-acid-based methods, immunological methods, and biosensor-based methods, etc. This article reviews the principles, characteristics, and applications of recent rapid detection methods for foodborne pathogens.

negative: Regression testing in the presence of non-code changes Regression testing is an important activity performed to validate modified software, and one of its key tasks is regression test selection (RTS) -- selecting a subset of existing test cases to run on the modified software. Most existing RTS techniques focus on changes made to code components and completely ignore non-code elements, such as configuration files and databases, which can also change and affect the system behavior. To address this issue, we present a new RTS technique that performs accurate test selection in the presence of changes to non-code components. To do this, our technique computes traceability between test cases and the external data accessed by an application, and uses this information to perform RTS in the presence of changes to non-code elements. We present our technique, a prototype implementation of our technique, and a set of preliminary empirical results that illustrate the feasibility, effectiveness, and potential usefulness of our approach.

negative: Evidence for van der Waals adhesion in gecko setae. Geckos have evolved one of the most versatile and effective adhesives known. The mechanism of dry adhesion in the millions of setae on the toes of geckos has been the focus of scientific study for over a century. We provide the first direct experimental evidence for dry adhesion of gecko setae by van der Waals forces, and reject the use of mechanisms relying on high surface polarity, including capillary adhesion. The toes of live Tokay geckos were highly hydrophobic, and adhered equally well to strongly hydrophobic and strongly hydrophilic, polarizable surfaces. Adhesion of a single isolated gecko seta was equally effective on the hydrophobic and hydrophilic surfaces of a microelectro-mechanical systems force sensor. A van der Waals mechanism implies that the remarkable adhesive properties of gecko setae are merely a result of the size and shape of the tips, and are not strongly affected by surface chemistry. Theory predicts greater adhesive forces simply from subdividing setae to increase surface density, and suggests a possible design principle underlying the repeated, convergent evolution of dry adhesive microstructures in gecko, anoles, skinks, and insects. Estimates using a standard adhesion model and our measured forces come remarkably close to predicting the tip size of Tokay gecko seta. We verified the dependence on size and not surface type by using physical models of setal tips nanofabricated from two different materials. Both artificial setal tips stuck as predicted and provide a path to manufacturing the first dry, adhesive microstructures.

negative: Evaluation of nutritional assessment techniques in elderly people newly admitted to municipal care Objectives: To evaluate the Subjective Global Assessment (SGA) and the Mini Nutritional Assessment (MNA) with regard to validity using a combination of anthropometric and serum-protein measurements as standard criteria to assess protein-energy malnutrition (PEM).Design: Cross-sectional study with consecutive selection of residents aged ≥65 y.Setting: A municipality in the south of Sweden.Subjects: During a year, starting in October 1996, 148 females and 113 males, aged ≥65–104 y of age, newly admitted to special types of housing for the elderly, were included in the study.Results: According to SGA, 53% were assessed as malnourished or moderately malnourished on admission. The corresponding figure from MNA was 79% malnourished or at risk of malnutrition. Both tools indicated that anthropometric values and serum proteins were significantly lower in residents classified as being malnourished (P<0.05). Sensitivity in detecting PEM was in SGA 0.93 and in MNA 0.96 and specificity was 0.61 and 0.26, respectively. Using regression analysis, weight index and serum albumin were the best objective nutritional parameters in predicting the SGA- and MNA classifications. Item ‘muscle wasting’ in SGA and ‘self-experienced health status’ in MNA showed most predictive power concerning the odds of being assessed as malnourished.Conclusions: SGA was shown to be the more useful tool in detecting residents with established malnutrition and MNA in detecting residents who need preventive nutritional measures.

negative: Youth texting: Help or hindrance to literacy? An extensive amount of research has been performed in recent years into the widespread practice of text messaging in youth. As part of this broad area of research, the associations between youth texting and literacy have been investigated in a variety of contexts. A comprehensive, semi-systematic review of the literature into texting and literacy was conducted, with a particular focus on quantitative empirical studies. Media reports, teacher surveys, and qualitative studies were also taken into consideration as part of this wide-ranging examination of previous research into this phenomenon. There were no clear positive or negative links discovered between adolescent texting practices and literacy, with the research findings in this study area best summarized as mixed and inconclusive. More studies in this research area are required, especially of an experimental or longitudinal kind. In addressing the debate on the impact of texting on literacy, research analyses as well as media reports would benefit from a more balanced presentation of existing findings.

negative: Deep Neural Networks for i-Vector Language Identification of Short Utterances in Cars This paper is focused on the application of the Language Identification (LID) technology for intelligent vehicles. We cope with short sentences or words spoken in moving cars in four languages: English, Spanish, German, and Finnish. As the response time of the LID system is crucial for user acceptance in this particular task, speech signals of different durations with total average of 3.8s are analyzed. In this paper, the authors propose the use of Deep Neural Networks (DNN) to model effectively the i-vector space of languages. Both raw i-vectors and session variability compensated i-vectors are evaluated as input vectors to DNNs. The performance of the proposed DNN architecture is compared with both conventional GMM-UBM and i-vector/LDA systems considering the effect of durations of signals. It is shown that the signals with durations between 2 and 3s meet the requirements of this application, i.e., high accuracy and fast decision, in which the proposed DNN architecture outperforms GMM-UBM and i-vector/LDA systems by 37% and 28%, respectively.

negative: Cloud RAN challenges and solutions In this paper we take an overall look at key technical challenges in the evolution of the Radio Access Network (RAN) architecture towards Cloud RAN, and solutions to overcome them. To address fronthaul limitations, we examine the implications and tradeoffs enabled by functional splits on fronthaul needs, system performance, and centralization scale. We examine the architecture of algorithms for multi-cell coordination and implications in a Cloud RAN environment. To maximize the use of General-Purpose Processors (GPP) and operating systems such as Linux for Cloud RAN, we propose methods of achieving real-time performance suitable for RAN functions. To enable right-sizing the amount of compute used for various RAN functions based on the workload, we propose methods of pooling and elastic scaling for RAN functions that exploit the fact that certain RAN functions perform per-user operations while others perform per-cell operations. Cloud RAN also aims to use cloud management technologies such as virtualized infrastructure management (VIM) and orchestration for automating the instantiation and scaling of RAN functions. We identify special needs for RAN arising from real-time constraints and a mix of GPP and non-GPP hardware. Keywords—Cloud RAN, fronthaul, real-time, pooling, elastic scaling, multi-cell coordination, orchestration, NFV, 5G.

negative: Cyberbullying classification using text mining Cyberbully is a misuse of technology advantage to bully a person. Cyberbully and its impact have occurred around the world and now the number of cases are increasing. Cyberbullying detection is very important because the online information is too large so it is not possible to be tracked by humans. The purpose of this research is to construct a classification model with optimal accuracy in identifying cyberbully conversation using Naive Bayes method and Support Vector Machine (SVM) then applying n-gram 1 to 5 for the number of class 2, 4, and 11 for each method. Naive Bayes yields an average accuracy of 92.81%, SVM with a poly kernel yields an average accuracy of 97.11% It can be concluded that SVM with poly kernel yields higher accuracy than SVM with other kernels, Naive Bayes, and Kelly Reynolds research method of decision tree (J48) and k-NN.

negative: Compliance by design for artifact-centric business processes Compliance to legal regulations, internal policies, or best practices is becoming a more and more important aspect in business processes management. Compliance requirements are usually formulated in a set of rules that can be checked during or after the execution of the business process, called compliance by detection. If noncompliant behavior is detected, the business process needs to be redesigned. Alternatively, the rules can be already taken into account while modeling the business process to result in a business process that is compliant by design. This technique has the advantage that a subsequent verification of compliance is not required. This paper focuses on compliance by design and employs an artifactcentric approach. In this school of thought, business processes are not described as a sequence of tasks to be performed (i. e., imperatively), but from the point of view of the artifacts that are manipulated during the process (i. e., declaratively). We extend the artifact-centric approach to model compliance rules and show how compliant business processes can be synthesized automatically.

negative: Low Cost Dual Polarized Base Station Element for Long Term Evolution A broadband low profile dual polarized patch antenna for fourth generation long term evolution (4G LTE) is presented in this communication. By employing two wideband feeding mechanisms for the two input ports, a dual polarized patch antenna element with the characteristics of high input port isolation and wide impedance bandwidth is successfully designed. A wide meandering probe (M-probe) and a pair of twin-L-probes are proposed to feed a patch for dual polarization. The proposed design with a wideband balun has impedance bandwidths of over 47% at the two input ports as well as over 37 dB isolation within the entire operating band. Over 8 dBi gain can be achieved at the two ports.

negative: Mitigating DoS attacks against signature-based authentication in VANETs Vehicular ad hoc networks (VANETs) are supposed to improve traffic safety and drivers' experiences. In a typical VANET, a vehicle broadcasts safety messages to its neighbors. Since behaviors based on safety messages could be life-critical, authentication of these messages must be guaranteed. Many signature-based schemes have been proposed for authentication in VANETs, but few of them have addressed the problem of denial of service (DoS) attacks against signature-based authentication. In such a DoS attack, attackers can broadcast forged messages with invalid signatures to force the receiving vehicles to perform lots of unnecessary signature verifications, and thus the benign vehicles cannot verify the messages from other legitimate vehicles. Our scheme features a pre-authentication process before signature verifying process to deal with this kind of DoS attack. The pre-authentication process takes advantage of the one-way hash chain and a group rekeying scheme. Evaluations show that the proposed scheme mitigates such DoS attacks effectively.

negative: Dynamically pre-trained deep recurrent neural networks using environmental monitoring data for predicting PM2.5 Fine particulate matter ( $$\hbox {PM}_{2.5}$$ PM 2.5 ) has a considerable impact on human health, the environment and climate change. It is estimated that with better predictions, US$9 billion can be saved over a 10-year period in the USA (State of the science fact sheet air quality. http://www.noaa.gov/factsheets/new , 2012). Therefore, it is crucial to keep developing models and systems that can accurately predict the concentration of major air pollutants. In this paper, our target is to predict $$\hbox {PM}_{2.5}$$ PM 2.5 concentration in Japan using environmental monitoring data obtained from physical sensors with improved accuracy over the currently employed prediction models. To do so, we propose a deep recurrent neural network (DRNN) that is enhanced with a novel pre-training method using auto-encoder especially designed for time series prediction. Additionally, sensors selection is performed within DRNN without harming the accuracy of the predictions by taking advantage of the sparsity found in the network. The numerical experiments show that DRNN with our proposed pre-training method is superior than when using a canonical and a state-of-the-art auto-encoder training method when applied to time series prediction. The experiments confirm that when compared against the $$\hbox {PM}_{2.5}$$ PM 2.5 prediction system VENUS (National Institute for Environmental Studies. Visual Atmospheric Environment Utility System. http://envgis5.nies.go.jp/osenyosoku/ , 2014), our technique improves the accuracy of $$\hbox {PM}_{2.5}$$ PM 2.5 concentration level predictions that are being reported in Japan.

negative: Pareto Optimal Security Resource Allocation for Internet of Things In many Internet of Thing (IoT) application domains security is a critical requirement, because malicious parties can undermine the effectiveness of IoT-based systems by compromising single components and/or communication channels. Thus, a security infrastructure is needed to ensure the proper functioning of such systems even under attack. However, it is also critical that security be at a reasonable resource and energy cost. In this article, we focus on the problem of efficiently and effectively securing IoT networks by carefully allocating security resources in the network area. In particular, given a set of security resources R and a set of attacks to be faced A, our method chooses the subset of R that best addresses the attacks in A, and the set of locations where to place them, that ensure the security coverage of all IoT devices at minimum cost and energy consumption. We model our problem according to game theory and provide a Pareto-optimal solution in which the cost of the security infrastructure, its energy consumption, and the probability of a successful attack are minimized. Our experimental evaluation shows that our technique improves the system robustness in terms of packet delivery rate for different network topologies. Furthermore, we also provide a method for handling the computation of the resource allocation plan for large-scale networks scenarios, where the optimization problem may require an unreasonable amount of time to be solved. We show how our proposed method drastically reduces the computing time, while providing a reasonable approximation of the optimal solution.

negative: Deep Metric and Hash-Code Learning for Content-Based Retrieval of Remote Sensing Images The growing volume of Remote Sensing (RS) image archives demands for feature learning techniques and hashing functions which can: (1) accurately represent the semantics in the RS images; and (2) have quasi real-time performance during retrieval. This paper aims to address both challenges at the same time, by learning a semantic-based metric space for content based RS image retrieval while simultaneously producing binary hash codes for an efficient archive search. This double goal is achieved by training a deep network using a combination of different loss functions which, on the one hand, aim at clustering semantically similar samples (i.e., images), and, on the other hand, encourage the network to produce final activation values (i.e., descriptors) that can be easily binarized. Moreover, since RS annotated training images are too few to train a deep network from scratch, we propose to split the image representation problem in two different phases. In the first we use a general-purpose, pre-trained network to produce an intermediate representation, and in the second we train our hashing network using a relatively small set of training images. Experiments on two aerial benchmark archives show that the proposed method outperforms previous state-of-the-art hashing approaches by up to 5.4% using the same number of hash bits per image.

negative: Control Knowledge to Improve Plan Quality Generating production-quality plans is an essential element in transforming planners from research tools into real-world applications. However most of the work to date on learning planning control knowledge has been aimed at improving the efficiency of planning; this work has been termed “speed-up learning”. This paper focuses on learning control knowledge to guide a planner towards better solutions, i.e. to improve the quality of the plans produced by the planner, as its problem solving experience increases. We motivate the use of quality-enhancing search control knowledge and its automated acquisition from problem solving experience. We introduce an implemented mechanism for learning such control knowledge and some of our preliminary results in a process planning domain.

negative: Multi-Scale Convolutional Neural Networks for Time Series Classification Time series classification (TSC), the problem of predicting class labels of time series, has been around for decades within the community of data mining and machine learning, and found many important applications such as biomedical engineering and clinical prediction. However, it still remains challenging and falls short of classification accuracy and efficiency. Traditional approaches typically involve extracting discriminative features from the original time series using dynamic time warping (DTW) or shapelet transformation, based on which an off-the-shelf classifier can be applied. These methods are ad-hoc and separate the feature extraction part with the classification part, which limits their accuracy performance. Plus, most existing methods fail to take into account the fact that time series often have features at different time scales. To address these problems, we propose a novel end-to-end neural network model, Multi-scale Convolutional Neural Network (MCNN), which incorporates feature extraction and classification in a single framework. Leveraging a novel multi-branch layer and learnable convolutional layers, MCNN automatically extracts features at different scales and frequencies, leading to superior feature representation. MCNN is also computationally efficient, as it naturally leverages GPU computing. We conduct comprehensive empirical evaluation with various existing methods on a large number of benchmark datasets, and show that MCNN advances the state-of-the-art by achieving superior accuracy performance than other leading methods.

negative: The Deep Regression Bayesian Network and Its Applications: Probabilistic Deep Learning for Computer Vision Deep directed generative models have attracted much attention recently due to their generative modeling nature and powerful data representation ability. In this article, we review different structures of deep directed generative models and the learning and inference algorithms associated with the structures. We focus on a specific structure that consists of layers of Bayesian networks (BNs) due to the property of capturing inherent and rich dependencies among latent variables. The major difficulty of learning and inference with deep directed models with many latent variables is the intractable inference due to the dependencies among the latent variables and the exponential number of latent variable configurations. Current solutions use variational methods, often through an auxiliary network, to approximate the posterior probability inference. In contrast, inference can also be performed directly without using any auxiliary network to maximally preserve the dependencies among the latent variables. Specifically, by exploiting the sparse representation with the latent space, max-max instead of maxsum operation can be used to overcome the exponential number of latent configurations. Furthermore, the max-max operation and augmented coordinate ascent (AugCA) are applied to both supervised and unsupervised learning as well as to various inference. Quantitative evaluations on benchmark data sets of different models are given for both data representation and feature-learning tasks.

negative: Modeling textual entailment with role semantic information In this thesis, we present a novel approach for modeling textual entailment using lexicalsemantic information on the level of predicate-argument structure. To this end, we adopt information provided by the Berkeley FrameNet repository and embed it into an implemented end-to-end system. The two main goals of this thesis are the following: (i) to provide an analysis of the potential contribution of frame semantic information to the recognition textual entailment and (ii) to present a robust system architecture that can serve as basis for future experiments, research, and improvement. Our work was carried out in the context of the textual entailment initiative, which since 2005 has set the stage for the broad investigation of inference in natural-language processing tasks, including empirical evaluation of its coverage and reliability. In short, textual entailment describes inferential relations between (entailing) texts and (entailed) hypotheses as interpreted by typical language users. This pre-theoretic notion captures a natural range of inferences as compared to logical entailment, which has traditionally been used within theoretical approaches to natural language semantics. Various methods for modeling textual entailment have been proposed in the literature, ranging from shallow techniques like lexical overlap to shallow syntactic parsing and the exploitation of WordNet relations. Recently, there has been a move towards more structured meaning representations. In particular, the level of predicate-argument structure has gained much attention, which seems to be a natural and straightforward choice. Predicate-argument structure allows annotating sentences or texts with nuclear meaning representations (“who did what to whom”), which are of obvious relevance for this task. For example, it can account for paraphrases like “Ghosts scare John” vs. “John is scared by ghosts”. In this thesis, we present an approach to textual entailment that is centered around the analysis of predicate-argument structure. It combines LFG grammatical analysis, predicate-argument structure in the FrameNet paradigm, and taxonomic information from WordNet into tripartite graph structures. By way of a declarative graph matching algorithm, the “structural and semantic” similarity of hypotheses and texts is computed and the result is represented as feature vectors. A supervised machine learning architecture trained on entailment corpora is used to check textual entailment for new text/hypothesis pairs. The approach is implemented in the SALSA RTE system, which successfully participated in the second and third RTE challenge. While system performance is on a par with that of comparable systems, the intuitively expected strong positive effect of using FrameNet information has not yet been confirmed. In order to evaluate different system components and to assess the potential contribution of FrameNet information for checking textual entailment, we conducted a number of experiments. For example, with the help of a gold-standard corpus, we

negative: Any domain parsing: automatic domain adaptation for natural language parsing of “Any Domain Parsing: Automatic Domain Adaptation for Natural Language Parsing” by David McClosky , Ph.D., Brown University, May, 2010. Current efforts in syntactic parsing are largely data-driv en. These methods require labeled examples of syntactic structures to learn statistical patterns governing these structures. Labeled data typically requires expert annotators which makes it both time consuming and costly to p roduce. Furthermore, once training data has been created for one textual domain, portability to similar domains is limited. This domain-dependence has inspired a large body of work since syntactic parsing aims to capture syntactic patterns across an entire language rather than just a specific domain. The simplest approach to this task is to assume that the targe t domain is essentially the same as the source domain. No additional knowledge about the target domain is i cluded. A more realistic approach assumes that only raw text from the target domain is available. This a s umption lends itself well to semi-supervised learning methods since these utilize both labeled and unlab eled examples. This dissertation focuses on a family of semi-supervised me thods called self-training. Self-training creates semi-supervised learners from existing supervised learne rs with minimal effort. We first show results on self-training for constituency parsing within a single dom ain. While self-training has failed here in the past, we present a simple modification which allows it to succeed, p roducing state-of-the-art results for English constituency parsing. Next, we show how self-training is be neficial when parsing across domains and helps further when raw text is available from the target domain. On e of the remaining issues is that one must choose a training corpus appropriate for the target domain or perfo rmance may be severely impaired. Humans can do this in some situations, but this strategy becomes less prac tical as we approach larger data sets. We present a technique, Any Domain Parsing, which automatically detect s useful source domains and mixes them together to produce a customized parsing model. The resulting models perform almost as well as the best seen parsing models (oracle) for each target domain. As a result, we have a fully automatic syntactic constituency parser which can produce high-quality parses for all types of text, regardless of domain. Abstract of “Any Domain Parsing: Automatic Domain Adaptation for Natural Language Parsing” by David McClosky , Ph.D., Brown University, May, 2010.of “Any Domain Parsing: Automatic Domain Adaptation for Natural Language Parsing” by David McClosky , Ph.D., Brown University, May, 2010. Current efforts in syntactic parsing are largely data-driv en. These methods require labeled examples of syntactic structures to learn statistical patterns governing these structures. Labeled data typically requires expert annotators which makes it both time consuming and costly to p roduce. Furthermore, once training data has been created for one textual domain, portability to similar domains is limited. This domain-dependence has inspired a large body of work since syntactic parsing aims to capture syntactic patterns across an entire language rather than just a specific domain. The simplest approach to this task is to assume that the targe t domain is essentially the same as the source domain. No additional knowledge about the target domain is i cluded. A more realistic approach assumes that only raw text from the target domain is available. This a s umption lends itself well to semi-supervised learning methods since these utilize both labeled and unlab eled examples. This dissertation focuses on a family of semi-supervised me thods called self-training. Self-training creates semi-supervised learners from existing supervised learne rs with minimal effort. We first show results on self-training for constituency parsing within a single dom ain. While self-training has failed here in the past, we present a simple modification which allows it to succeed, p roducing state-of-the-art results for English constituency parsing. Next, we show how self-training is be neficial when parsing across domains and helps further when raw text is available from the target domain. On e of the remaining issues is that one must choose a training corpus appropriate for the target domain or perfo rmance may be severely impaired. Humans can do this in some situations, but this strategy becomes less prac tical as we approach larger data sets. We present a technique, Any Domain Parsing, which automatically detect s useful source domains and mixes them together to produce a customized parsing model. The resulting models perform almost as well as the best seen parsing models (oracle) for each target domain. As a result, we have a fully automatic syntactic constituency parser which can produce high-quality parses for all types of text, regardless of domain. Any Domain Parsing: Automatic Domain Adaptation for Natural Language Parsing by David McClosky B. S., University of Rochester, 2004 Sc. M., Brown University, 2006 A dissertation submitted in partial fulfillment of the requirements for the Degree of Doctor of Philosophy in the Department of Computer Science at Brown University. Providence, Rhode Island May, 2010 c © Copyright 2010 by David McClosky This dissertation by David McClosky is accepted in its prese nt form by the Department of Computer Science as satisfying the disser tation requirement for the degree of Doctor of Philosophy. Date Eugene Charniak, Director Recommended to the Graduate Council Date Mark Johnson, Reader Cognitive and Linguistic Sciences Date Dan Klein, Reader University of California at Berkeley Approved by the Graduate Council Date Sheila Bond Dean of the Graduate School

negative: Adding Gradient Noise Improves Learning for Very Deep Networks { Adding Gradient Noise Improves Learning for Very Deep Networks. Arvind Neelakantan, Luke Vilnis, Quoc V. Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach, James Martens. International Conference on Learning Representations Workshop (ICLR WS), 2016 { Generating Sentences from a Continuous Space. Samuel Bowman, Luke Vilnis, Oriol Vinyals, Andrew Dai, Rafal Jozefowicz, Samy Bengio. International Conference on Learning Representations Workshop (ICLR WS), 2016 { Bethe Projections for Non-Local Inference. Luke Vilnis, David Belanger, Daniel Sheldon, Andrew McCallum. Uncertainty in Artificial Intelligence (UAI), 2015 { Learning Dynamic Feature Selection for Fast Sequential Prediction. Emma Strubell, Luke Vilnis, Kate Silverstein, Andrew McCallum. Annual Meeting of the Association for Computational Linguistics (ACL), 2015 { Word Representations via Gaussian Embedding. Luke Vilnis, Andrew McCallum. International Conference on Learning Representations (ICLR), 2015 { Generalized Eigenvectors for Large Multiclass Problems. Luke Vilnis, Nikos Karampatziakis, Paul Mineiro. Neural Information Processing Systems Workshop on Representation and Learning Methods for Complex Outputs (NIPS WS), 2014 { Training for Fast Sequential Prediction Using Dynamic Feature Selection. Emma Strubell, Luke Vilnis, Andrew McCallum. Neural Information Processing Systems Workshop on Modern Machine Learning and Natural Language Processing (NIPS WS), 2014

negative: Predicate learning in neural systems: Discovering latent generative structures Humans learn complex latent structures from their environments (e.g., natural language, mathematics, music, social hierarchies). In cognitive science and cognitive neuroscience, models that infer higher-order structures from sensory or first-order representations have been proposed to account for the complexity and flexibility of human behavior. But how do the structures that these models invoke arise in neural systems in the first place? To answer this question, we explain how a system can learn latent representational structures (i.e., predicates) from experience with wholly unstructured data. During the process of predicate learning, an artificial neural network exploits the naturally occurring dynamic properties of distributed computing across neuronal assemblies in order to learn predicates, but also to combine them compositionally, two computational aspects which appear to be necessary for human behavior as per formal theories in multiple domains. We describe how predicates can be combined generatively using neural oscillations to achieve human-like extrapolation and compositionality in an artificial neural network. The ability to learn predicates from experience, to represent structures compositionally, and to extrapolate to unseen data offers an inroads to understanding and modeling the most complex human behaviors.

negative: Probabilistic Semantics and Pragmatics: Uncertainty in Language and Thought Language is used to communicate ideas. Ideas are mental tools for coping with a complex and uncertain world. Thus human conceptual structures should be key to language meaning, and probability—the mathematics of uncertainty— should be indispensable for describing both language and thought. Indeed, probabilistic models are enormously useful in modeling human cognition (Tenenbaum et al., 2011) and aspects of natural language (Bod et al., 2003; Chater et al., 2006). With a few early exceptions (e.g. Adams, 1975; Cohen, 1999b), probabilistic tools have only recently been used in natural language semantics and pragmatics. In this chapter we synthesize several of these modeling advances, exploring a formal model of interpretation grounded, via lexical semantics and pragmatic inference, in conceptual structure. Flexible human cognition is derived in large part from our ability to imagine possibilities (or possible worlds). A rich set of concepts, intuitive theories, and other mental representations support imagining and reasoning about possible worlds—together we will call these the conceptual lexicon. We posit that this collection of concepts also forms the set of primitive elements available for lexical semantics: word meanings can be built from the pieces of conceptual structure. Larger semantic structures are then built from word meanings by composition, ultimately resulting in a sentence meaning which is a phrase in the “language of thought” provided by the conceptual lexicon. This expression is truth-functional in that it takes on a Boolean value for each imagined world, and it can thus be used as the basis for belief updating. However, the connection between cognition, semantics, and belief is not direct: because language must flexibly adapt to the context of communication, the connection between lexical representation and interpreted meaning is mediated by pragmatic inference.

negative: Why Nouns Are Learned before Verbs: Linguistic Relativity Versus Natural Partitioning. Technical Report No. 257. ABSTRACT There is overwhelming evidence that children's first words are primarily nouns even across languages. These data are interpreted as evidence of a "Natural Partitions Theory," one that holds that the concepts referred to by nouns are conceptually more basic than those referred to by verbs or prepositions. Analysis of data from cross-linguistic studies designed to eliminate Inonconceptual differences between words in a language that might account for the Larlier acquisition of nouns--such as the position of a word in the sentence, morphological transparency (the ease with which the root can be heard in various uses of the word), and patterns of language teaching strategies--leaves some version of the Natural Partitions hypothesis as the most reasonable view of early vocabulary acquisition. A cross-linguistic examination of a single sentence suggests that if objecthood is created by spatial relations among perceptual elements, then good concrete objects are particularly cohesive collections of percepts. These highly cohesive collections of perceptual information tend to be lexicalized as nouns by almost every language. Object concepts are given to language users by the world and can be learned one at a time, while predicate concepts form a system in each language., that, from a child's point of view, must be discovered. (HTE)