query: Sequence to Sequence Learning with Neural Networks

positive: Learning representations by back-propagating errors We describe a new learning procedure, back-propagation, for networks of neurone-like units. The procedure repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector. As a result of the weight adjustments, internal ‘hidden’ units which are not part of the input or output come to represent important features of the task domain, and the regularities in the task are captured by the interactions of these units. The ability to create useful new features distinguishes back-propagation from earlier, simpler methods such as the perceptron-convergence procedure1.

positive: Long Short-Term Memory Learning to store information over extended time intervals by recurrent backpropagation takes a very long time, mostly because of insufficient, decaying error backflow. We briefly review Hochreiter's (1991) analysis of this problem, then address it by introducing a novel, efficient, gradient based method called long short-term memory (LSTM). Truncating the gradient where this does not do harm, LSTM can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units. Multiplicative gate units learn to open and close access to the constant error flow. LSTM is local in space and time; its computational complexity per time step and weight is O. 1. Our experiments with artificial data involve local, distributed, real-valued, and noisy pattern representations. In comparisons with real-time recurrent learning, back propagation through time, recurrent cascade correlation, Elman nets, and neural sequence chunking, LSTM leads to many more successful runs, and learns much faster. LSTM also solves complex, artificial long-time-lag tasks that have never been solved by previous recurrent network algorithms.

positive: Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies D3EGF(FIH)J KMLONPEGQSRPETN UCV.WYX(Z R.[ V R6\M[ X N@]_^O\`JaNcb V RcQ W d EGKeL(^(QgfhKeLOE?i)^(QSj ETNPfPQkRl[ V R)m"[ X ^(KeLOEG^ npo qarpo m"[ X ^(KeLOEG^tsAu EGNPb V ^ v wyx zlwO{(|(}<~OC}((xp{ay.~A}_~ Cl3#|<Azw#|l6 (|  JpfhL XV EG^O QgJ  ETFOR] ^O\JNPb V RcQ X E)ETR 6EGKeLOETNcKMLOE F ETN V RcQgJp^(^OE ZgZ E i ^(Qkj EGNPfhQSRO E OE2m1Jp^ RcNY E VZ sO! ¡ q.n sCD X KGKa8¢EG^ RPNhE¤£ ¥¦Q ZgZ Es m§J^ RPNO E VZ s( ̈ X  EG©#EKas# V ^ V  V s(H a «a¬3 ®#|.Y ̄y} xa°OC}l{x  yxlY~3{|  ±2Pz   V J Z J U N V fhKTJp^(Q  ETFOR J\ D vYf3RPEGb ́f V ^(§JpbF X RPETN@D KTQEG^(KTE i ^(QSjpEGNPfhQSR4vμJ\ U¶Z JaNPEG^(K·E jYQ V (Q ̧D V ^ R V m V N3R V aOs#1 o ¡Ga r U QNhE^OoTE1⁄4»,] R VZ vC1⁄2 3⁄4  x ± x  #¿ }À 3t}lC}2P}<~ ¬t[ X NPE^§D KeL(b ́Qg(L X ©yETN ]  DY]_Á JNPfhJÃÂ Z j EToQ V a rpopo2Ä X  V ^(J(sCD Å)QSRPoTEGN ZgV ^( Æ #|{3 ̄|.(C}.C¿Y}p Pzw

positive: Edinburgh's Phrase-based Machine Translation Systems for WMT-14 This paper describes the University of Edinburgh’s (UEDIN) phrase-based submissions to the translation and medical translation shared tasks of the 2014 Workshop on Statistical Machine Translation (WMT). We participated in all language pairs. We have improved upon our 2013 system by i) using generalized representations, specifically automatic word clusters for translations out of English, ii) using unsupervised character-based models to translate unknown words in RussianEnglish and Hindi-English pairs, iii) synthesizing Hindi data from closely-related Urdu data, and iv) building huge language on the common crawl corpus.

positive: Gradient-based learning applied to document recognition

negative: Extended Bandwidth Instantaneous Current Sharing Scheme for Parallel UPS Systems This paper investigates the suitability of instantaneous average current sharing scheme (IACS) for the parallel operation of three-phase uninterruptible power supply systems. A discrete-time model is developed for the analysis and design of the control loops. Some key issues are discussed based on the model and it is found that there is a compromise between system stability and current sharing performance with the conventional IACS scheme when the lengths of interconnecting cables are not negligible. Subsequently, an improved IACS scheme is proposed that ensures proper current sharing by extending the close-loop bandwidth. Its performance is analytically predicted and subsequently validated by experimental results on a 15-kW laboratory prototype.

negative:
AccessMiner: using system-centric models for malware protection Models based on system calls are a popular and common approach to characterize the run-time behavior of programs. For example, system calls are used by intrusion detection systems to detect software exploits. As another example, policies based on system calls are used to sandbox applications or to enforce access control. Given that malware represents a significant security threat for today's computing infrastructure, it is not surprising that system calls were also proposed to distinguish between benign processes and malicious code.
 Most proposed malware detectors that use system calls follows program-centric analysis approach. That is, they build models based on specific behaviors of individual applications. Unfortunately, it is not clear how well these models generalize, especially when exposed to a diverse set of previously-unseen, real-world applications that operate on realistic inputs. This is particularly problematic as most previous work has used only a small set of programs to measure their technique's false positive rate. Moreover, these programs were run for a short time, often by the authors themselves.
 In this paper, we study the diversity of system calls by performing a large-scale collection (compared to previous efforts) of system calls on hosts that run applications for regular users on actual inputs. Our analysis of the data demonstrates that simple malware detectors, such as those based on system call sequences, face significant challenges in such environments. To address the limitations of program-centric approaches, we propose an alternative detection model that characterizes the general interactions between benign programs and the operating system (OS). More precisely, our system-centric approach models the way in which benign programs access OS resources (such as files and registry entries). Our experiments demonstrate that this approach captures well the behavior of benign programs and raises very few (even zero) false positives while being able to detect a significant fraction of today's malware.

negative: How do people solve the "weather prediction" task?: individual variability in strategies for probabilistic category learning. Probabilistic category learning is often assumed to be an incrementally learned cognitive skill, dependent on nondeclarative memory systems. One paradigm in particular, the weather prediction task, has been used in over half a dozen neuropsychological and neuroimaging studies to date. Because of the growing interest in using this task and others like it as behavioral tools for studying the cognitive neuroscience of cognitive skill learning, it becomes especially important to understand how subjects solve this kind of task and whether all subjects learn it in the same way. We present here new experimental and theoretical analyses of the weather prediction task that indicate that there are at least three different strategies that describe how subjects learn this task. (1) An optimal multi-cue strategy, in which they respond to each pattern on the basis of associations of all four cues with each outcome; (2) a one-cue strategy, in which they respond on the basis of presence or absence of a single cue, disregarding all other cues; or (3) a singleton strategy, in which they learn only about the four patterns that have only one cue present and all others absent. This variability in how subjects approach this task may have important implications for interpreting how different brain regions are involved in probabilistic category learning.

negative: RISK FACTORS FOR INJURY TO WOMEN FROM DOMESTIC VIOLENCE A BSTRACT Background Domestic violence is the most common cause of nonfatal injury to women in the United States. To identify risk factors for such injuries, we examined the socioeconomic and behavioral characteristics of women who were victims of domestic violence and the men who injured them. Methods We conducted a case–control study at eight large, university-affiliated emergency departments. The 256 intentionally injured women had acute injuries resulting from a physical assault by a male partner. The 659 controls were women treated for other conditions in the emergency department. Information was collected with a standardized questionnaire; no information was obtained directly from the male partners. Results The 256 intentionally injured women had a total of 434 contusions and abrasions, 89 lacerations, and 41 fractures and dislocations. In a multivariate analysis, the characteristics of the partners that were most closely associated with an increased risk of inflicting injury as a result of domestic violence were alcohol abuse (adjusted relative risk, 3.6; 95 percent confidence interval, 2.2 to 5.9); drug use (adjusted relative risk, 3.5; 95 percent confidence interval, 2.0 to 6.4); intermittent employment (adjusted relative risk, 3.1; 95 percent confidence interval, 1.1 to 8.8); recent unemployment (adjusted relative risk, 2.7; 95 percent confidence interval, 1.2 to 6.5); having less than a high-school education (adjusted relative risk, 2.5; 95 percent confidence interval, 1.4 to 4.4); and being a former husband, estranged husband, or former boyfriend (adjusted relative risk, 3.5; 95 percent confidence interval, 1.5 to 8.3). Conclusions Women at greatest risk for injury from domestic violence include those with male partners who abuse alcohol or use drugs, are unemployed or intermittently employed, have less than a high-school education, and are former husbands, estranged husbands, or former boyfriends of the women. (N Engl J Med 1999;341:1892-8.)

negative: Private and Accurate Data Aggregation against Dishonest Nodes Privacy-preserving data aggregation in ad hoc networks is a challenging problem, considering the distributed communication and control requirement, dynamic network topology, unreliable communication links, etc. The difficulty is exaggerated when there exist dishonest nodes, and how to ensure privacy, accuracy, and robustness against dishonest nodes remains a n open issue. Different from the widely used cryptographic approaches, in this paper, we address this challenging proble m by exploiting the distributed consensus technique. We first pr opose a secure consensus-based data aggregation (SCDA) algorith m that guarantees an accurate sum aggregation while preservi ng the privacy of sensitive data. Then, to mitigate the polluti on from dishonest nodes, we propose an Enhanced SCDA (E-SCDA) algorithm that allows neighbors to detect dishonest nodes, and derive the error bound when there are undetectable dishones t nodes. We prove the convergence of both SCDA and E-SCDA. We also prove that the proposed algorithms are(ǫ, σ)-dataprivacy, and obtain the mathematical relationship betweenǫ and σ. Extensive simulations have shown that the proposed algori thms have high accuracy and low complexity, and they are robust against network dynamics and dishonest nodes.

negative: Predicting student academic performance in an engineering dynamics course: A comparison of four types of predictive mathematical models Predicting student academic performance has long been an important research topic in many academic disciplines. The present study is the first study that develops and compares four types of mathematical models to predict student academic performance in engineering dynamics – a high-enrollment, highimpact, and core course that many engineering undergraduates are required to take. The four types of mathematical models include the multiple linear regression model, the multilayer perception network model, the radial basis function network model, and the support vector machine model. The inputs (i.e., predictor variables) of the models include student’s cumulative GPA, grades earned in four pre-requisite courses (statics, calculus I, calculus II, and physics), and scores on three dynamics mid-term exams (i.e., the exams given to students during the semester and before the final exam). The output of the models is students’ scores on the dynamics final comprehensive exam. A total of 2907 data points were collected from 323 undergraduates in four semesters. Based on the four types of mathematical models and six different combinations of predictor variables, a total of 24 predictive mathematical models were developed from the present study. The analysis reveals that the type of mathematical model has only a slight effect on the average prediction accuracy (APA, which indicates on average how well a model predicts the final exam scores of all students in the dynamics course) and on the percentage of accurate predictions (PAP, which is calculated as the number of accurate predictions divided by the total number of predictions). The combination of predictor variables has only a slight effect on the APA, but a profound effect on the PAP. In general, the support vector machine models have the highest PAP as compared to the other three types of mathematical models. The research findings from the present study imply that if the goal of the instructor is to predict the average academic performance of his/her dynamics class as a whole, the instructor should choose the simplest mathematical model, which is the multiple linear regression model, with student’s cumulative GPA as the only predictor variable. Adding more predictor variables does not help improve the average prediction accuracy of any mathematical model. However, if the goal of the instructor is to predict the academic performance of individual students, the instructor should use the support vector machine model with the first six predictor variables as the inputs of the model, because this particular predictor combination increases the percentage of accurate predictions, and most importantly, allows sufficient time for the instructor to implement subsequent educational interventions to improve student learning. 2012 Elsevier Ltd. All rights reserved.

negative: Elephant herding optimization algorithm for support vector machine parameters tuning Classification is part of various applications and it is an important problem that represents active research topic. Support vector machine is one of the widely used and very powerful classifier. The accuracy of support vector machine highly depends on learning parameters. Optimal parameters can be efficiently determined by using swarm intelligence algorithms. In this paper, we proposed recent elephant herding optimization algorithm for support vector machine parameter tuning. The proposed approach is tested on standard datasets and it was compared to other approaches from literature. The results of computational experiments show that our proposed algorithm outperformed genetic algorithms and grid search considering accuracy of classification.

negative: New generation waveform approaches for 5G and beyond For 5G and beyond cellular communication systems, new waveform development studies have been continuing in the whole world. Solutions for providing high throughput, high data rates, low latency, large number of terminals, supporting multiple antenna systems, and the related problems originated from these issues are playing an important role in the development of 5G and new waveform technologies. OFDM which has many important advantages continues to be included in various standards. However, because of its disadvantages, OFDM will not be sufficient for dynamic spectrum access and cognitive radio applications in new generation communication systems gradually. Additionally, OFDM is not suitable for asynchronous networks and radios which we expect to see commonly in the future. In this study, some potential candidate 5G waveforms are examined and various comparisons are carried out. Besides, assessments about future hybrid solutions are presented.

negative: Very-low-profile, stand-alone, tri-band WLAN antenna design for laptop-tablet computer with complete metal-backed cover A very-low-profile, tri-band, planar inverted-F antenna (PIFA) with a height of 3.7 mm only for wireless local area network (WlAN) applications in the 2.4 GHz (2400∼2484 MHz), 5.2 GHz (5150∼5350 MHz), and 5.8 GHz (5725∼5825 MHz) bands for tablet-laptop computer with a metal back cover is presented. The antenna was made of a flat plate of size 18.7 mm × 32.5 mm and bent into a compact structure with the dimensions 3.7 mm × 7 mm × 32.5 mm. The PIFA comprised a radiating top patch, a feeding/shorting portion, and an antenna ground plane. The top patch was placed horizontally above the ground with the feeding/shorting portion connecting perpendicularly therebetween. The antenna was designed in a way that the radiating patch consisted of three branches, in which each branch controlled the corresponding resonant mode in the 2.4/5.2/5.8 GHz bands.

negative: GRAVITY-ASSIST: A series elastic body weight support system with inertia compensation We present GRAVITY-ASSIST, a series elastic active body weight support and inertia compensation system for use in robot assisted gait rehabilitation. The device consists of a single degree of freedom series elastic actuator that connects to the trunk of a patient. The series elastic system is novel in that, it can provide the desired level of dynamic unloading such that the patient experiences only a percentage of his/her weight and inertia. Inertia compensation is important, since the inertial forces can cause significant deviations from the desired unloading force, specially at low support forces and fast walking speeds. Furthermore, this feature enables the inertia of the harness and force sensing unit attached to the patient to be compensated for, making sure that the device does not interfere with the natural gait cycle. We present a functional prototype of the device, its characterization and experimental verification of the approach.

negative: A Framework for Information Systems Architecture With increasing size and complexity of the implementations of information systems, it is necessary to use some logical construct (or architecture) for defining and controlling the interfaces and the integration of all of the components of the system. This paper defines information systems architecture by creating a d e scriptive framework from disciplines quite independent of information systems, then by analogy specifies information systems architecture based upon the neutral , objective framework. Also, some preliminary conclusions about the implications of the resultant descriptive framework are drawn. The discussion is limited to architecture and does not include a strategic planning methodology. T he subject of information systems architecture is beginning to receive considerable attention. The increased scope of design and levels of complexity of information systems implementations are forcing the use of some logical construct (or architecture) for defining and controlling the interfaces and the integration of all of the components of the system. Thirty years ago this issue was not at all significant because the technology itself did not provide for either breadth in scope or depth in complexity in information systems. The inherent limitations of the then-available 4K machines, for example, constrained design and necessitated suboptimal approaches for automating a business. Current technology is rapidly removing both conceptual and financial constraints. It is not hard to speculate about, if not realize, very large, very complex systems implementations, extending in scope and complexity to encompass an entire enterprise. One can readily delineate the merits of the large, complex, enterprise-oriented approaches. Such systems allow flexibility in managing business changes and coher-ency in the management of business resources. However , there also is merit in the more traditional, smaller, suboptimal systems design approach. Such systems are relatively economical, quickly implemented , and easier to design and manage. In either case, since the technology permits " distributing " large amounts of computing facilities in small packages to remote locations, some kind of structure (or architecture) is imperative because decentralization without structure is chaos. Therefore, to keep the business from disintegrating, the concept of information systems architecture is becoming less an option and more a necessity for establishing some order and control in the investment of information systems resources. The cost involved and the success of the business depending increasingly on its information systems require a disciplined approach to the management of those systems. On the assumption that an understanding of information systems architecture …

negative: Networks versus markets in international trade I propose a network/search view of international trade in differentiated products. I present evidence that supports the view that proximity and common language/colonial ties are more important for differentiated products than for products traded on organized exchanges in matching international buyers and sellers, and that search barriers to trade are higher for differentiated than for homogeneous products. I also discuss alternative explanations for the findings.

negative: Reading Comprehension with Deep Learning We train a model that combines attention with multi-perspective matching to perform question answering. For each question and context pair in SQuAD, we perform an attention calculation over each context before extracting features of the question and context, matching them from multiple perspectives. Whilst we did not have time to perform a hyper-parameter search or incorporate other features into our model, this joint model obtained an F1 score of 22.5 and an EM score of 13.0, outperforming our baseline model without attention, which attained an F1 score of 19.4 and an EM score of 10.0. In the future, we would like to implement an early stopping feature and a co-dependent representation of each question and context pair.

negative: Mapping moods: Geo-mapped sentiment analysis during hurricane sandy Sentiment analysis has been widely researched in the domain of online review sites with the aim of generating summarized opinions of product users about different aspects of the products. However, there has been little work focusing on identifying the polarity of sentiments expressed by users during disaster events. Identifying sentiments expressed by users in an online social networking site can help understand the dynamics of the network, e.g., the main users’ concerns, panics, and the emotional impacts of interactions among members. Data produced through social networking sites is seen as ubiquitous, rapid and accessible, and it is believed to empower average citizens to become more situationally aware during disasters and coordinate to help themselves. In this work, we perform sentiment classification of user posts in Twitter during the Hurricane Sandy and visualize these sentiments on a geographical map centered around the hurricane. We show how users' sentiments change according not only to users’ locations, but also based on the distance from the disaster.

negative: AUTOMATIC BLOOD VESSEL SEGMENTATION IN COLOR IMAGES OF RETINA Automated image processing techniques have the ability to assist in the early detection of diabetic retinopathy disease which can be regarded as a manifestation of diabetes on the retina. Blood vessel segmentation is the basic foundation while developing retinal screening systems, since vessels serve as one of the main retinal landmark features. This paper proposes an automated method for identification of blood vessels in color images of the retina. For every image pixel, a feature vector is computed that utilizes properties of scale and orientation selective Gabor filters. The extracted features are then classified using generative Gaussian mixture model and discriminative support vector machines classifiers. Experimental results demonstrate that the area under the receiver operating characteristic (ROC) curve reached a value 0.974, which is highly comparable and, to some extent, higher than the previously reported ROCs that range from 0.787 to 0.961. Moreover, this method gives a sensitivity of 96.50% with a specificity of 97.10% for identification of blood vessels. Keywords– Retinal blood vessels, Gabor filters, support vector machines, vessel segmentation

negative: Cubical Type Theory: A Constructive Interpretation of the Univalence Axiom This paper presents a type theory in which it is possible to directly manipulate n-dimensional cubes (points, lines, squares, cubes, etc.) based on an interpretation of dependent type theory in a cubical set model. This enables new ways to reason about identity types, for instance, function extensionality is directly provable in the system. Further, Voevodsky’s univalence axiom is provable in this system. We also explain an extension with some higher inductive types like the circle and propositional truncation. Finally we provide semantics for this cubical type theory in a constructive meta-theory. 1998 ACM Subject Classification F.3.2 Logics and Meanings of Programs: Semantics of Programming Languages, F.4.1 Mathematical Logic and Formal Languages: Mathematical Logic

negative: A review of wind power and wind speed forecasting methods with different time horizons In recent years, environmental considerations have prompted the use of wind power as a renewable energy resource. However, the biggest challenge in integrating wind power into the electric grid is its intermittency. One approach to deal with wind intermittency is forecasting future values of wind power production. Thus, several wind power or wind speed forecasting methods have been reported in the literature over the past few years. This paper provides insight on the foremost forecasting techniques, associated with wind power and speed, based on numeric weather prediction (NWP), statistical approaches, artificial neural network (ANN) and hybrid techniques over different time-scales. An overview of comparative analysis of various available forecasting techniques is discussed as well. In addition, this paper further gives emphasis on the major challenges and problems associated with wind power prediction.

negative: What does it take to generate natural textures Natural image generation is currently one of the most actively explored fields in Deep Learning. Many approaches, e.g. for state-of-the-art artistic style transfer or natural texture synthesis, rely on the statistics of hierarchical representations in supervisedly trained deep neural networks. It is, however, unclear what aspects of this feature representation are crucial for natural image generation: is it the depth, the pooling or the training of the features on natural images? We here address this question for the task of natural texture synthesis and show that none of the above aspects are indispensable. Instead, we demonstrate that natural textures of high perceptual quality can be generated from networks with only a single layer, no pooling and random filters.

negative:
Inside help: An integrative review of champions in healthcare-related implementation Background/aims
The idea that champions are crucial to effective healthcare-related implementation has gained broad acceptance; yet the champion construct has been hampered by inconsistent use across the published literature. This integrative review sought to establish the current state of the literature on champions in healthcare settings and bring greater clarity to this important construct.

Methods
This integrative review was limited to research articles in peer-reviewed, English-language journals published from 1980 to 2016. Searches were conducted on the online MEDLINE database via OVID and PubMed using the keyword "champion." Several additional terms often describe champions and were also included as keywords: implementation leader, opinion leader, facilitator, and change agent. Bibliographies of full-text articles that met inclusion criteria were reviewed for additional references not yet identified via the main strategy of conducting keyword searches in MEDLINE. A five-member team abstracted all full-text articles meeting inclusion criteria.

Results
The final dataset for the integrative review consisted of 199 unique articles. Use of the term champion varied widely across the articles with respect to topic, specific job positions, or broader organizational roles. The most common method for operationalizing champion for purposes of analysis was the use of a dichotomous variable designating champion presence or absence. Four studies randomly allocated of the presence or absence of champions.

Conclusions
The number of published champion-related articles has markedly increased: more articles were published during the last two years of this review (i.e. 2015-2016) than during its first 30 years (i.e. 1980-2009).The number of champion-related articles has continued to increase sharply since the year 2000. Individual studies consistently found that champions were important positive influences on implementation effectiveness. Although few in number, the randomized trials of champions that have been conducted demonstrate the feasibility of using experimental design to study the effects of champions in healthcare.

negative: Investigating Social Entrepreneurship in Developing Countries Social entrepreneurship has drawn interest from global policy makers and social entrepreneurs to target developing countries. Generally, not-forprofit organizations, funded by government and donor grants have played a significant role in poverty alleviation. We argue that, by applying entrepreneurial concepts, organizations can create social value, hence mitigate poverty. This is a theoretical paper that builds upon a multidimensional model in analysing how three social enterprises from India and Kenya create social value to address social problems. The findings suggest that whilst the social mission is central to all these organizations, they also create social value through innovation and pro-activeness. Additionally, the cultural and political environmental contexts hinder their attempt to create social value. Building networks and partnerships to achieve social value creation is vital for these organizations. Policy makers should devise policies that would assist social enterprises to achieve development goals.

negative: Crowd-Squared: Amplifying the Predictive Power of Search Trend Data Table A1. Search Terms Generated by the Crowd (a) Influenza – Crowd-Squared Search Terms and % of Turkers Mentioning Each Term Search Term % Mention Search Term % Mention Search Term % Mention Search Term % Mention sick 68% influenza 4% achy 2% fatigue 1% fever 39% coughing 4% weak 2% home 1% cough 21% germs 4% bad 2% hospital 1% cold 16% headache 4% shots 2% nyquil 1% shot 13% ache 4% body aches 2% season 1% vomit 11% runny nose 4% flu shot 2% tea 1%

negative: Language Modeling with Feedforward Neural Networks In this paper, we describe the use of feedforward neural networks to improve the term-distance term-occurrence (TDTO) language model, previously proposed in [1]−[3]. The main idea behind the TDTO model proposition is to model separately both position and occurrence information of words in the history-context to better estimate n-gram probabilities. Neural networks have been shown to offer a better generalization property than other conventional smoothing methods. We take advantage of such property for a better smoothing mechanism for the TDTO model, referred to as the continuous space TDTO (cTDTO). The newly proposed model has reported an improved perplexity over the baseline TDTO model of up to 9.2%, at history length of ten, as evaluated on the Wall Street Journal (WSJ) corpus. Also, in the Aurora-4 speech recognition N-best re-ranking task, the cTDTO outperformed the TDTO model by reducing the word error rate (WER) up to 12.9% relatively.

negative: Diagnosis, prevention, and treatment of catheter-associated urinary tract infection in adults: 2009 International Clinical Practice Guidelines from the Infectious Diseases Society of America. Guidelines for the diagnosis, prevention, and management of persons with catheter-associated urinary tract infection (CA-UTI), both symptomatic and asymptomatic, were prepared by an Expert Panel of the Infectious Diseases Society of America. The evidence-based guidelines encompass diagnostic criteria, strategies to reduce the risk of CA-UTIs, strategies that have not been found to reduce the incidence of urinary infections, and management strategies for patients with catheter-associated asymptomatic bacteriuria or symptomatic urinary tract infection. These guidelines are intended for use by physicians in all medical specialties who perform direct patient care, with an emphasis on the care of patients in hospitals and long-term care facilities.

negative: A New Clustering Method Based On Type-2 Fuzzy Similarity and Inclusion Measures Similarity and inclusion measures between type-2 fuzzy sets have a wide range of applications. New similarity and inclusion measures between type-2 fuzzy sets are respectively defined in this paper. The properties of the measures are discussed. Some examples are used to compare the proposed measures with the existing results. Numerical results show that the proposed measures are more reasonable. Similarity measures and Yang and Shih’s algorithm are combined as a clustering method for type-2 fuzzy data. Clustering results demonstrate that the proposed similarity measures are better than the existing measures.

negative: Temperature insensitive current reference circuit using standard CMOS devices In this paper, temperature insensitive current reference circuit is proposed. The reference current value is determined by using the threshold voltage controlled circuit. The main difference from the previous work (Georglou and Toumazou, 2002) is that the circuit can be fabricated by the standard CMOS process. The resistor and the transistor physical parameter temperature dependences are compensated with each other to determine the output reference current. The detail temperature performance is analyzed, and is evaluated by simulations.

negative: Tensor2Tensor for Neural Machine Translation Tensor2Tensor is a library for deep learning models that is very well-suited for neural machine translation and includes the reference implementation of the state-of-the-art Transformer model. 1 Neural Machine Translation Background Machine translation using deep neural networks achieved great success with sequence-tosequence models Sutskever et al. (2014); Bahdanau et al. (2014); Cho et al. (2014) that used recurrent neural networks (RNNs) with LSTM cells Hochreiter and Schmidhuber (1997). The basic sequence-to-sequence architecture is composed of an RNN encoder which reads the source sentence one token at a time and transforms it into a fixed-sized state vector. This is followed by an RNN decoder, which generates the target sentence, one token at a time, from the state vector. While a pure sequence-to-sequence recurrent neural network can already obtain good translation results Sutskever et al. (2014); Cho et al. (2014), it suffers from the fact that the whole input sentence needs to be encoded into a single fixed-size vector. This clearly manifests itself in the degradation of translation quality on longer sentences and was partially overcome in Bahdanau et al. (2014) by using a neural model of attention. Convolutional architectures have been used to obtain good results in word-level neural machine translation starting from Kalchbrenner and Blunsom (2013) and later in Meng et al. (2015). These early models used a standard RNN on top of the convolution to generate the output, which creates a bottleneck and hurts performance. Fully convolutional neural machine translation without this bottleneck was first achieved in Kaiser and Bengio (2016) and Kalchbrenner et al. (2016). The model in Kaiser and Bengio (2016) (Extended Neural GPU) used a recurrent stack of gated convolutional layers, while the model in Kalchbrenner et al. (2016) (ByteNet) did away with recursion and used left-padded convolutions in the decoder. This idea, introduced in WaveNet van den Oord et al. (2016), significantly improves efficiency of the model. The same technique was improved in a number of neural translation models recently, including Gehring et al. (2017) and Kaiser et al. (2017).

negative: A Hierarchical Recurrent Neural Network for Symbolic Melody Generation In recent years, neural networks have been used to generate symbolic melodies. However, the long-term structure in the melody has posed great difficulty for designing a good model. In this paper, we present a hierarchical recurrent neural network for melody generation, which consists of three Long-Short-Term-Memory (LSTM) subnetworks working in a coarse-to-fine manner along time. Specifically, the three subnetworks generate bar profiles, beat profiles and notes in turn, and the output of the high-level subnetworks are fed into the low-level subnetworks, serving as guidance for generating the finer time-scale melody components in low-level subnetworks. Two human behavior experiments demonstrate the advantage of this structure over the single-layer LSTM which attempts to learn all hidden structures in melodies. Compared with the state-of-the-art models MidiNet (Yang, Chou, and Yang 2017) and MusicVAE (Roberts et al. 2018), the hierarchical recurrent neural network produces better melodies evaluated by humans. Introduction Automatic music generation using neural networks has attracted much attention. There are two classes of music generation approaches, symbolic music generation (Hadjeres, Pachet, and Nielsen 2017)(Waite et al. 2016)(Yang, Chou, and Yang 2017) and audio music generation (van den Oord et al. )(Mehri et al. 2016). In this study, we focus on symbolic melody generation, which requires learning from sheet music. Many music genres such as pop music consist of melody and harmony. Since usually beautiful harmonies can be ensured by using legitimate chord progressions which have been summarized by musicians, we only focus on melody generation, similar to some recent studies (Waite et al. 2016)(Yang, Chou, and Yang 2017)(Colombo, Seeholzer, and Gerstner 2017)(Roberts et al. 2018). This greatly simplifies the melody generation problem. Melody is a linear succession of musical notes along time. It has both short time scale such as notes and long time scale such as phrases and movements, which makes the melody generation a challenging task. Existing methods generate pitches and rhythm simultaneously (Waite et al. 2016) or sequentially (Chu, Urtasun, and Fidler 2016) using Recurrent Neural Networks (RNNs), but they usually work on the note scale without explicitly modeling the larger time-scale components such as rhythmic patterns. It is difficult for them to learn long-term dependency or structure in melody. Theoretically, an RNN can learn the temporal structure of any length in the input sequence, but in reality, as the sequence gets longer it is very hard to learn long-term structure. Different RNNs have different learning capability, e.g., LSTM (Hochreiter and Schmidhuber 1997) performs much better than the simple Elman network. But any model has a limit for the length of learnable structure, and this limit depends on the complexity of the sequence to be learned. To enhance the learning capability of an RNN, one approach is to invent a new structure. In this work we take another approach: increase the granularity of the input. Since each symbol in the sequence corresponds to longer segment than the original representation, the same model would learn longer temporal structure. To implement this idea, we propose a Hierarchical Recurrent Neural Network (HRNN) for learning melody. It consists of three LSTM-based sequence generators — Bar Layer, Beat Layer and Note Layer. The Bar Layer and Beat Layer are trained to generate bar profiles and beat profiles, which are designed to represent the high-level temporal features of melody. The Note Layer is trained to generate melody conditioned on the bar profile sequence and beat profile sequence output by the Bar Layer and Beat Layer. By learning on different time scales, the HRNN can grasp the general regular patterns of human composed melodies in different granularities, and generate melody with realistic long-term structures. This method follows the general idea of granular computing (Bargiela and Pedrycz 2012), in which different resolutions of knowledge or information is extracted and represented for problem solving. With the shorter profile sequences to guide the generation of note sequence, the difficulty of generating note sequence with wellorganized structure is alleviated. Related Work Melody Generation with Neural Networks There is a long history of generating melody with RNNs. A recurrent autopredictive connectionist network called CONCERT is used to compose music (Mozer 1994). With a set of composition rules as constraints to evaluate ar X iv :1 71 2. 05 27 4v 2 [ cs .S D ] 5 S ep 2 01 8 melodies, an evolving neural network is employed to create melodies (Chen and Miikkulainen 2001). As an important form of RNN, LSTM (Hochreiter and Schmidhuber 1997) is used to capture the global music structure and improve the quality of the generated music (Eck and Schmidhuber 2002). Boulanger-Lewandowski, Bengio, and Vincent explore complex polyphonic music generation with an RNNRBM model (Boulanger-Lewandowski, Bengio, and Vincent 2012). Lookback RNN and Attention RNN are proposed to tackle the problem of creating melody’s long-term structure (Waite et al. 2016). The Lookback RNN introduces a handcrafted lookback feature that makes the model repeat sequences easier while the Attention RNN leverages an attention mechanism to learn longer-term structures. Inspired by convolution, two variants of RNN are employed to attain transposition invariance (Johnson 2017). To model the relation between rhythm and melody flow, a melody is divided into pitch sequence and duration sequence and these two sequences are processed in parallel (Colombo et al. 2016). This approach is further extended in (Colombo, Seeholzer, and Gerstner 2017). A hierarchical VAE is employed to learn the distribution of melody pieces in (Roberts et al. 2018), the decoder of which is similar to our model. The major difference is that the higher layer of its decoder uses the automatically learned representation of bars, while our higher layers use predefined representation of bars and beats which makes the learning problem easier. Generative Adversarial Networks (GANs) have also been used to generate melodies. For example, RNN-based GAN (Mogren 2016) and CNN-based GAN (Yang, Chou, and Yang 2017) are employed to generate melodies, respectively. However, the generated melodies also lack realistic long-term structures. Some models are proposed to generate multi-track music. A 4-layer LSTM is employed to produce the key, press, chord and drum of pop music seperately (Chu, Urtasun, and Fidler 2016). With pseudo-Gibbs sampling, a model can generate highly convincing chorales in the style of Bach (Colombo, Seeholzer, and Gerstner 2017). Three GANs for symbolic-domain multi-track music generation were proposed (Dong et al. 2018). An end-to-end melody and arrangement generation framework XiaoIce Band was proposed to generate a melody track with accompany tracks with RNN (Zhu et al. 2018). Hierarchical and Multiple Time Scales Networks The idea of hierarchical or multiple time scales has been used in neural network design, especially in the area of natural language processing. The Multiple Timescale Recurrent Neural Network (MTRNN) realizes the self-organization of a functional hierarchy with two types of neurons “fast” unit and “slow” unit (Yamashita and Tani 2008). Then it is shown that the MTRNN can acquire the capabilities to recognize, generate, and correct sentences in a hierarchical way: characters grouped into words, and words into sentences (Hinoshita et al. 2011). An LSTM auto-encoder is trained to preserve and reconstruct paragraphs by hierarchically building embeddings of words, sentences and paragraphs (Li, Luong, and Jurafsky 2015). To process inputs at multiple time Time signature Bar Key signature

negative: Facial Performance Capture with Deep Neural Networks We present a deep learning technique for facial performance capture, i.e., the transfer of video footage into a motion sequence of a 3D mesh representing an actor’s face. Specifically, we build on a conventional capture pipeline based on computer vision and multiview video, and use its results to train a deep neural network to produce similar output from a monocular video sequence. Once trained, our network produces high-quality results for unseen inputs with greatly reduced effort compared to the conventional system. In practice, we have found that approximately 10 minutes worth of high-quality data is sufficient for training a network that can then automatically process as much footage from video to 3D as needed. This yields major savings in the development of modern narrativedriven video games involving digital doubles of actors and potentially hours of animated dialogue per character.

negative: A Hybrid Approach to Grapheme to Phoneme Conversion in Assamese Assamese is one of the low resource Indian languages. This paper implements both rule-based and data-driven grapheme to phoneme (G2P) conversion systems for Assamese. The rule-based system is used as the baseline which yields a word error rate of 35.3%. The data-driven systems are implemented using state-of-the-art sequence learning techniques such as —i) Joint-Sequence Model (JSM), ii) Recurrent Neural Networks with LTSM cell (LSTM-RNN) and iii) bidirectional LSTM (BiLSTM). The BiLSTM yields the lowest WER i.e., 18.7%, which is an absolute 16.6% improvement on the baseline system. We additionally implement the rules of syllabification for Assamese. The surface output is generated in two forms namely i) phonemic sequence with syllable boundaries, and ii) only phonemic sequence. The output of BiLSTM is fed as an input to Hybrid system. The Hybrid system syllabifies the input phonemic sequences to apply the vowel harmony rules. It also applies the rules of schwa-deletion as well as some rules in which the consonants change their form in clusters. The accuracy of the Hybrid system is 17.3% which is an absolute 1.4% improvement over the BiLSTM based G2P.

negative: Learning to dress: synthesizing human dressing motion via deep reinforcement learning Creating animation of a character putting on clothing is challenging due to the complex interactions between the character and the simulated garment. We take a model-free deep reinforcement learning (deepRL) approach to automatically discovering robust dressing control policies represented by neural networks. While deepRL has demonstrated several successes in learning complex motor skills, the data-demanding nature of the learning algorithms is at odds with the computationally costly cloth simulation required by the dressing task. This paper is the first to demonstrate that, with an appropriately designed input state space and a reward function, it is possible to incorporate cloth simulation in the deepRL framework to learn a robust dressing control policy. We introduce a salient representation of haptic information to guide the dressing process and utilize it in the reward function to provide learning signals during training. In order to learn a prolonged sequence of motion involving a diverse set of manipulation skills, such as grasping the edge of the shirt or pulling on a sleeve, we find it necessary to separate the dressing task into several subtasks and learn a control policy for each subtask. We introduce a policy sequencing algorithm that matches the distribution of output states from one task to the input distribution for the next task in the sequence. We have used this approach to produce character controllers for several dressing tasks: putting on a t-shirt, putting on a jacket, and robot-assisted dressing of a sleeve.

negative: A SeqGAN for Polyphonic Music Generation We propose an application of SeqGAN, generative adversarial networks for discrete sequence generation, for creating polyphonic musical sequences. Instead of monophonic melody generation suggested in the original work, we present an efficient representation of polyphony MIDI file that captures chords and melodies with dynamic timings simultaneously. The network can create sequences that are musically coherent. We also report that careful tuning of reinforcement learning signals of the model are crucial for general application.

negative: Deep Learning with Sets and Point Clouds We study a simple notion of structural invariance that readily suggests a parameter-sharing scheme in deep neural networks. In particular, we define structure as a collection of relations, and derive graph convolution and recurrent neural networks as special cases. We study composition of basic structures in defining models that are invariant to more complex “product” structures such as graph of graphs, sets of images or sequence of sets. For demonstration, our experimental results are focused on the setting where the discrete structure of interest is a set. We present results on several novel and non-trivial problems on sets, including point-cloud classification, set outlier detection and semi-supervised learning using clustering information.