query: Posterior distribution analysis for Bayesian inference in neural networks

positive: Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks Large multilayer neural networks trained with backpropagation have recently achieved state-ofthe-art results in a wide range of problems. However, using backprop for neural net learning still has some disadvantages, e.g., having to tune a large number of hyperparameters to the data, lack of calibrated probabilistic predictions, and a tendency to overfit the training data. In principle, the Bayesian approach to learning neural networks does not have these problems. However, existing Bayesian techniques lack scalability to large dataset and network sizes. In this work we present a novel scalable method for learning Bayesian neural networks, called probabilistic backpropagation (PBP). Similar to classical backpropagation, PBP works by computing a forward propagation of probabilities through the network and then doing a backward computation of gradients. A series of experiments on ten real-world datasets show that PBP is significantly faster than other techniques, while offering competitive predictive abilities. Our experiments also show that PBP provides accurate estimates of the posterior variance on the network weights.

positive: A family of algorithms for approximate Bayesian inference One of the major obstacles to using Bayesian methods for pattern recognition has been its computational expense. This thesis presents an approximation technique that can perform Bayesian inference faster and more accurately than previously possible. This method, "Expectation Propagation," unifies and generalizes two previous techniques: assumeddensity filtering, an extension of the Kalman filter, and loopy belief propagation, an extension of belief propagation in Bayesian networks. The unification shows how both of these algorithms can be viewed as approximating the true posterior distribution with a simpler distribution, which is close in the sense of KL-divergence. Expectation Propagation exploits the best of both algorithms: the generality of assumed-density filtering and the accuracy of loopy belief propagation. Loopy belief propagation, because it propagates exact belief states, is useful for limited types of belief networks, such as purely discrete networks. Expectation Propagation approximates the belief states with expectations, such as means and variances, giving it much wider scope. Expectation Propagation also extends belief propagation in the opposite direction-propagating richer belief states which incorporate correlations between variables. This framework is demonstrated in a variety of statistical models using synthetic and real-world data. On Gaussian mixture problems, Expectation Propagation is found, for the same amount of computation, to be convincingly better than rival approximation techniques: Monte Carlo, Laplace's method, and variational Bayes. For pattern recognition, Expectation Propagation provides an algorithm for training Bayes Point Machine classifiers that is faster and more accurate than any previously known. The resulting classifiers outperform Support Vector Machines on several standard datasets, in addition to having a comparable training time. Expectation Propagation can also be used to choose an appropriate feature set for classification, via Bayesian model selection. Thesis Supervisor: Rosalind Picard Title: Associate Professor of Media Arts and Sciences

positive: Stochastic Backpropagation and Approximate Inference in Deep Generative Models We marry ideas from deep neural networks and approximate Bayesian inference to derive a generalised class of deep, directed generative models, endowed with a new algorithm for scalable inference and learning. Our algorithm introduces a recognition model to represent an approximate posterior distribution and uses this for optimisation of a variational lower bound. We develop stochastic backpropagation – rules for gradient backpropagation through stochastic variables – and derive an algorithm that allows for joint optimisation of the parameters of both the generative and recognition models. We demonstrate on several real-world data sets that by using stochastic backpropagation and variational inference, we obtain models that are able to generate realistic samples of data, allow for accurate imputations of missing data, and provide a useful tool for high-dimensional data visualisation.

positive: Stochastic Gradient Hamiltonian Monte Carlo Hamiltonian Monte Carlo (HMC) sampling methods provide a mechanism for defining distant proposals with high acceptance probabilities in a Metropolis-Hastings framework, enabling more efficient exploration of the state space than standard random-walk proposals. The popularity of such methods has grown significantly in recent years. However, a limitation of HMC methods is the required gradient computation for simulation of the Hamiltonian dynamical system—such computation is infeasible in problems involving a large sample size or streaming data. Instead, we must rely on a noisy gradient estimate computed from a subset of the data. In this paper, we explore the properties of such a stochastic gradient HMC approach. Surprisingly, the natural implementation of the stochastic approximation can be arbitrarily bad. To address this problem we introduce a variant that uses second-order Langevin dynamics with a friction term that counteracts the effects of the noisy gradient, maintaining the desired target distribution as the invariant distribution. Results on simulated data validate our theory. We also provide an application of our methods to a classification task using neural networks and to online Bayesian matrix factorization.

negative: Query Segmentation and Resource Disambiguation Leveraging Background Knowledge Accessing the wealth of structured data available on the Data Web is still a key challenge for lay users. Keyword search is the most convenient way for users to access information (e.g., from data repositories). In this paper we introduce a novel approach for determining the correct resources for user-supplied keyword queries based on a hidden Markov model. In our approach the user-supplied query is modeled as the observed data and the background knowledge is used for parameter estimation. Instead of learning parameter estimation from training data, we leverage the semantic relationships between data items for computing the parameter estimations. In order to maximize accuracy and usability, query segmentation and resource disambiguation are mutually tightly interwoven. First, an initial set of potential segmentations is obtained leveraging the underlying knowledge base; then the final correct set of segments is determined after the most likely resource mapping was computed using a scoring function. While linguistic methods like named entity, multi-word unit recognition and POS-tagging fail in the case of an incomplete sentences (e.g. for keyword-based queries), we will show that our statistical approach is robust with regard to query expression variance. Our experimental results when employing the hidden Markov model for resource identification in keyword queries reveal very promising results.

negative: Humanoid locomotion on uneven terrain using the time-varying divergent component of motion This paper presents a framework for dynamic humanoid locomotion on uneven terrain using a novel time-varying extension to the Divergent Component of Motion (DCM). By varying the natural frequency of the DCM, we are able to achieve generic CoM height trajectories during stepping. The proposed planning algorithm computes admissible DCM reference trajectories given desired ZMP plans for single and double support. This is accomplished using reverse-time integration of the discretized DCM dynamics over a finite time horizon. To account for discontinuities during replanning, linear Model Predictive Control (MPC) is implemented over a short preview window. DCM tracking control is achieved using a time-varying proportional-integral controller based on the Virtual Repellent Point (VRP). The effectiveness of the combined approach is verified in simulation using a 30-DoF model of THOR, a compliant torque-controlled humanoid.

negative: Expanded Hemodialysis: A New Therapy for a New Class of Membranes. A wide spectrum of molecules is retained in end-stage kidney disease, normally defined as uremic toxins. These solutes have different molecular weights and radii. Current dialysis membranes and techniques only remove solutes in the range of 50-15,000 Da, with limited or no capability to remove solutes in the middle to high molecular weight range (up to 50,000 Da). Improved removal has been obtained with high cut-off (HCO) membranes, with albumin loss representing a limitation to their practical application. Hemodiafiltration (HDF) at high volumes (>23 L/session) has produced some results on middle molecules and clinical outcomes, although complex hardware and high blood flows are required. A new class of membrane has been recently developed with a cut off (MWCO) close to the molecular weight of albumin. While presenting negligible albumin loss, these membranes have a very high retention onset (MWRO), allowing high clearances of solutes in a wide spectrum of molecular weights. These membranes originally defined (medium cut off) are probably better classified as high retention onset. The introduction of such membranes in the clinical routine has allowed the development of a new concept therapy called "expanded hemodialysis" (HDx). The new therapy is based on a special hollow fiber and dialyzer design. Its simple set-up and application offer the possibility to use it even in patients with suboptimal vascular access or even with an indwelling catheter. The system does not require a particular hardware or unusual nursing skill. The quality of dialysis fluid is, however, mandatory to ensure a safe conduction of the dialysis session. This new therapy is likely to modify the outcome of end-stage kidney disease patients, thanks to the enhanced removal of molecules traditionally retained by current dialysis techniques.

negative: Less-structured time in children's daily lives predicts self-directed executive functioning Executive functions (EFs) in childhood predict important life outcomes. Thus, there is great interest in attempts to improve EFs early in life. Many interventions are led by trained adults, including structured training activities in the lab, and less-structured activities implemented in schools. Such programs have yielded gains in children's externally-driven executive functioning, where they are instructed on what goal-directed actions to carry out and when. However, it is less clear how children's experiences relate to their development of self-directed executive functioning, where they must determine on their own what goal-directed actions to carry out and when. We hypothesized that time spent in less-structured activities would give children opportunities to practice self-directed executive functioning, and lead to benefits. To investigate this possibility, we collected information from parents about their 6-7 year-old children's daily, annual, and typical schedules. We categorized children's activities as "structured" or "less-structured" based on categorization schemes from prior studies on child leisure time use. We assessed children's self-directed executive functioning using a well-established verbal fluency task, in which children generate members of a category and can decide on their own when to switch from one subcategory to another. The more time that children spent in less-structured activities, the better their self-directed executive functioning. The opposite was true of structured activities, which predicted poorer self-directed executive functioning. These relationships were robust (holding across increasingly strict classifications of structured and less-structured time) and specific (time use did not predict externally-driven executive functioning). We discuss implications, caveats, and ways in which potential interpretations can be distinguished in future work, to advance an understanding of this fundamental aspect of growing up.

negative: Conditional High-Order Boltzmann Machines for Supervised Relation Learning Relation learning is a fundamental problem in many vision tasks. Recently, high-order Boltzmann machine and its variants have shown their great potentials in learning various types of data relation in a range of tasks. But most of these models are learned in an unsupervised way, i.e., without using relation class labels, which are not very discriminative for some challenging tasks, e.g., face verification. In this paper, with the goal to perform supervised relation learning, we introduce relation class labels into conventional high-order multiplicative interactions with pairwise input samples, and propose a conditional high-order Boltzmann Machine (CHBM), which can learn to classify the data relation in a binary classification way. To be able to deal with more complex data relation, we develop two improved variants of CHBM: 1) latent CHBM, which jointly performs relation feature learning and classification, by using a set of latent variables to block the pathway from pairwise input samples to output relation labels and 2) gated CHBM, which untangles factors of variation in data relation, by exploiting a set of latent variables to multiplicatively gate the classification of CHBM. To reduce the large number of model parameters generated by the multiplicative interactions, we approximately factorize high-order parameter tensors into multiple matrices. Then, we develop efficient supervised learning algorithms, by first pretraining the models using joint likelihood to provide good parameter initialization, and then finetuning them using conditional likelihood to enhance the discriminant ability. We apply the proposed models to a series of tasks including invariant recognition, face verification, and action similarity labeling. Experimental results demonstrate that by exploiting supervised relation labels, our models can greatly improve the performance.

negative: From Hunt the Wumpus to EverQuest: Introduction to Quest Theory The paper will explore how the landscape types and the quest types are used in various games, how they structure the gameplay, how they act as bones for the game-content (graphics, dialogue, sound) and how they sometimes form the base on which a story is imposed and related to the player. The question then becomes, how does the quest structure influence the story structure? How do the limitations of the quest combinations limit the kinds of story that are possible? How rich can the imposed story be, without breaking the gameplay? Are landscape and quest-structure the dominant factors in quest game design, to which the story-ambitions must defer? The main thesis of the paper is that if we understand the powerful but simple structure the grammar of quests (how they work, how they are used) we can understand both the limits and the potential of these kinds of games.

negative: Two practical man-in-the-middle attacks on Bluetooth secure simple pairing and countermeasures We propose two new Man-In-The-Middle (MITM) attacks on Bluetooth Secure Simple Pairing (SSP). The attacks are based on the falsification of information sent during the input/output capabilities exchange and also the fact that the security of the protocol is likely to be limited by the capabilities of the least powerful or the least secure device type. In addition, we devise countermeasures that render the attacks impractical, as well as improvements to the existing Bluetooth SSP in order to make it more secure. Moreover, we provide a comparative analysis of the existing MITM attacks on Bluetooth.

negative: Optimizing orthodontic treatment in patients taking bisphosphonates for osteoporosis. Bisphosphonates have unique pharmacological characteristics unlike those of any other drug group. Millions of adults take oral bisphosphonates for long-term treatment of osteoporosis and osteopenia; some of these people will most likely also seek orthodontic treatment. Adverse dental effects from bisphosphonates have been reported, including decreased tooth movement, impaired bone healing, and osteonecrosis in the mandible and the maxilla. Osteonecrosis has been rarely observed after bisphosphonate use for osteoporosis. However, adverse drug effects might occur more frequently in orthodontic patients, and they would probably be noted before the end-stage pathology of osteonecrosis. Adverse effects during orthodontic treatment, including decreased tooth movement, could last for years after the drug therapy is stopped. Successful orthodontic treatment requires optimal bone healing to prevent excessive tooth mobility. Bisphosphonates appear to have 2 bone elimination rates--a fast elimination of weeks from the bone surface and a slow elimination of years after incorporation into the bone structure. This article presents methods to clinically and radiographically monitor orthodontic patients who are taking oral bisphosphonates. Efforts to minimize adverse effects and optimize orthodontic procedures with physician-approved drug holidays are discussed. The orthodontic treatment results of 3 patients who received bisphosphonate therapy are reported.

negative: Use of e-books in an academic and research environment: A case study from the Indian Institute of Science Purpose: E-books are not as popular as other types of e-publications, such as e-journals and e-newspapers. The possible reasons for this could be because the technology for creating/accessing e-books (both hardware and software) is not yet matured, users’ perspectives about e-books need to be changed and the awareness of e-books needs to be raised. The purpose of this study is to investigate the use and usability of e-books from users’ perspectives in an academic and research environment.

negative: Robust Visual Tracking with Deep Convolutional Neural Network Based Object Proposals on PETS Tracking by detection based object tracking methods encounter numerous complications including object appearance changes, size and shape deformations, partial and full occlusions, which make online adaptation of classifiers and object models a substantial challenge. In this paper, we employ an object proposal network that generates a small yet refined set of bounding box candidates to mitigate the this object model refitting problem by concentrating on hard negatives when we update the classifier. This helps improving the discriminative power as hard negatives are likely to be due to background and other distractions. Another intuition is that, in each frame, applying the classifier only on the refined set of object-like candidates would be sufficient to eliminate most of the false positives. Incorporating an object proposal makes the tracker robust against shape deformations since they are handled naturally by the proposal stage. We demonstrate evaluations on the PETS 2016 dataset and compare with the state-of-theart trackers. Our method provides the superior results.

negative: Design and Sensitivity Analysis of Highly Compact Comparator for Ku-Band Monopulse Radar This paper gives the design of a highly compact comparator at a Ku-band frequency and presents analysis results of the comparator for the fabrication inaccuracies. First an unconventional magic-t using a nonstandard waveguide is designed at 15.50 GHz. To reduce the volume occupied by the magic-t, its E-arm (or difference port) is kept parallel to the plane of two inputs of the magic-t instead of perpendicular to them as is done in a convention magic-t. The sum and the difference ports of the above folded magic-t are then matched using inductive windows at 15.50 GHz. Keeping the required location of the outputs of the comparator in mind, four of these matched folded magic-ts are suitably interconnected to design a highly compact comparator. The effects of the fabrication errors in the waveguide and matching elements dimensions on the centre frequency, magnitude and phase response of the comparator are also analyzed and presented.

negative: 5G 3GPP-Like Channel Models for Outdoor Urban Microcellular and Macrocellular Environments For the development of new 5G systems to operate in bands up to 100 GHz, there is a need for accurate radio propagation models at these bands that currently are not addressed by existing channel models developed for bands below 6 GHz. This document presents a preliminary overview of 5G channel models for bands up to 100 GHz. These have been derived based on extensive measurement and ray tracing results across a multitude of frequencies from 6 GHz to 100 GHz, and this document describes an initial 3D channel model which includes: 1) typical deployment scenarios for urban microcells (UMi) and urban macrocells (UMa), and 2) a baseline model for incorporating path loss, shadow fading, line of sight probability, penetration and blockage models for the typical scenarios. Various processing methodologies such as clustering and antenna decoupling algorithms are also presented.

negative:
Breathing at a rate of 5.5 breaths per minute with equal inhalation-to-exhalation ratio increases heart rate variability. OBJECTIVES
Prior studies have found that a breathing pattern of 6 or 5.5 breaths per minute (bpm) was associated with greater heart rate variability (HRV) than that of spontaneous breathing rate. However, the effects of combining the breathing rate with the inhalation-to-exhalation ratio (I:E ratio) on HRV indices are inconsistent. This study aimed to examine the differences in HRV indices and subjective feelings of anxiety and relaxation among four different breathing patterns.

METHODS
Forty-seven healthy college students were recruited for the study, and a Latin square experimental design with a counterbalance in random sequences was applied. Participants were instructed to breathe at two different breathing rates (6 and 5.5 breaths) and two different I:E ratios (5:5 and 4:6). The HRV indices as well as anxiety and relaxation levels were measured at baseline (spontaneous breathing) and for the four different breathing patterns.

RESULTS
The results revealed that a pattern of 5.5 bpm with an I:E ratio of 5:5 produced a higher NN interval standard deviation and higher low frequency power than the other breathing patterns. Moreover, the four different breathing patterns were associated with significantly increased feeling of relaxation compared with baseline.

CONCLUSION
The study confirmed that a breathing pattern of 5.5 bpm with an I:E ratio of 5:5 achieved greater HRV than the other breathing patterns. This finding can be applied to HRV biofeedback or breathing training in the future.

negative: Data transmission via GSM voice channel for end to end security Global System for Mobile Communications (GSM) technology still plays a key role because of its availability, reliability and robustness. Recently, additional consumer applications are proposed in which GSM is used as a backup or data transmission service. Unfortunately sending data via GSM channel is a challenging task since it is speech sensitive and suppresses other forms of signals. In this paper, a systematic method is proposed to develop a modem that transmits data over GSM voice channel (DoGSMV) using speech like (SL) symbols. Unlike the previous approaches an artificial search space is produced to find best SL symbols and analyses by synthesis (AbyS) method is introduced for parameter decoding. As a result 1.6 kbps simulation data rate is achieved when wireless communication errors are ignored.

negative: Frequency-Reconfigurable Low-Profile Circular Monopolar Patch Antenna A novel frequency-reconfigurable antenna is presented based on a circular monopolar patch antenna. The antenna comprises of a center-fed circular patch surrounded by four sector-shaped patches. Eight varactor diodes are introduced to bridge the gaps between the circular patch and the sector-shaped patches. By changing the reverse bias voltage of the varactor diodes, the antenna can be switched in the operating frequency. A fully functional prototype is developed and tested, which exhibits a continuously tunable frequency band from 1.64 GHz to 2.12 GHz. The measured efficiency rises from about 45% to about 85% as the operating frequency increases from 1.64 GHz to 2.12 GHz. Stable monopole like radiation patterns are achieved at all operating frequencies. In addition, the antenna owns a low-profile structure (0.04 free-space wavelengths at 1.9 GHz). The frequency selective feature and stable radiation patterns make the antenna potentially suitable for future wireless communication systems, such as cognitive radio.

negative:
Three-dimensional soft-tissue and hard-tissue changes in the treatment of bimaxillary protrusion. INTRODUCTION
Facial convexity related to bimaxillary protrusion is prevalent in many populations. Underlying skeletal protrusion combined with increased dentoalveolar protrusion contributes to facial muscle imbalance and lip incompetence, which is undesirable for many patients. In this study, we evaluated the relationship between soft-tissue and hard-tissue changes in an orthodontically treated Asian population.

METHODS
Twenty-four consecutive adult Asian patients (mean age, 24 years), diagnosed with severe bimaxillary dentoalveolar protrusion, were evaluated using pretreatment and posttreatment cone-beam computed tomography. The patients were treated with 4 first premolar extractions followed by anterior retraction with either skeletal or intraoral anchorage. Serial cone-beam computed tomography radiographs were registered on the entire cranial base and fossa. Soft-tissue and hard-tissue changes were determined through landmark displacement and color mapping.

RESULTS
Upper lip retraction was concentrated between the nasolabial folds and commissures. Lower lip retraction was accompanied by significant redistribution of soft tissues at pogonion. Soft-tissue changes correlated well with regional facial muscle activity. Significant retractions (2-4 mm) of the soft tissues occurred beyond the midsagittal region. Use of skeletal anchorage resulted in 1.5 mm greater lower lip retraction than intraoral anchorage, with greater retraction of the maxillary and mandibular incisor root apices.

CONCLUSIONS
Profound soft-tissue changes accompanied retraction of the anterior dentition with both treatment modalities.

negative: Management of a Large Frontal Encephalocoele With Supraorbital Bar Remodeling and Advancement. Of all the craniofacial abnormalities, facial clefts are the most disfiguring. Facial clefts are classified according to the affected anatomical area as described by Tessier. Through this classification, the location and extent of the cleft can be designated numerically.A 2-month-old male infant was referred to authors' craniofacial unit, from a hospital in a rural province of South Africa, with a problem of a supranasal encephalocoele. Bilateral raised eyebrows were noted as was a right-sided upper lid central third coloboma. Computed tomography and magnetic resonance imaging scans confirmed the presence of a supranasal encephalocoele with a large frontal bone defect and splayed nasal bones. Bilateral enlarged orbits were noted with tented orbital roofs that we classified as Tessier number 10 facial clefts. The child was booked for an encephalocoele excision and calvarial reconstruction at 4 months of age.As a result of the encephalocoele, the supraorbital bar with its adjacent nasal bones was cleaved in 2, resulting in a significant frontal bone defect. Osteotomies were performed to remove the supraorbital bar and nasal bones from the calvarium. The supraorbital bar segment was remodeled and plated with absorbable poly-L-lactic acid plates. Osteotomies of the nasal bones allowed them to be united centrally, also with absorbable plates. This entire construct was transferred and secured to the calvarium, but in a more caudal position thereby obliterating the frontal bone and Tessier number 10 facial cleft defects with a naturally contoured construct.

negative: Parallel Cat Swarm Optimization We investigate a parallel structure of cat swarm optimization (CSO) in this paper, and we call it parallel cat swarm optimization (PCSO). In the experiments, we compare particle swarm optimization (PSO) with CSO and PCSO. The experimental results indicate that both CSO and PCSO perform well. Moreover, PCSO is an effective scheme to improve the convergent speed of cat swarm optimization in case the population size is small and the whole iteration is less.

negative:
WiFi-Nano: reclaiming WiFi efficiency through 800 ns slots The increase in WiFi physical layer transmission speeds from 1~Mbps to 1 Gbps has reduced transmission times for a 1500 byte packet from 12 ms to 12 us. However, WiFi MAC overheads such as channel access and acks have not seen similar reductions and cumulatively contribute about 150 us on average per packet. Thus, the efficiency of WiFi has deteriorated from over 80% at 1 Mbps to under 10% at 1 Gbps.
 In this paper, we propose WiFi-Nano, a system that uses 800 ns slots} to significantly improve WiFi efficiency. Reducing slot time from 9 us to 800 ns makes backoffs efficient, but clear channel assessment can no longer be completed in one slot since preamble detection can now take multiple slots. Instead of waiting for multiple slots for detecting preambles, nodes speculatively transmit preambles as their backoff counters expire, while continuing to detect premables using self-interference cancellation. Upon detection of preambles from other transmitters, nodes simply abort their own preamble transmissions, thereby allowing the earliest transmitter to succeed. Further, receivers speculatively transmit their ack preambles at the end of packet reception, thereby reducing ack overhead. We validate the effectiveness of WiFi-Nano through implementation on an FPGA-based software defined radio platform, and through extensive simulations, demonstrate efficiency gains of up to 100%.

negative: Who, What, When, and Where: Multi-Dimensional Collaborative Recommendations Using Tensor Factorization on Sparse User-Generated Data Given the abundance of online information available to mobile users, particularly tourists and weekend travelers, recommender systems that effectively filter this information and suggest interesting participatory opportunities will become increasingly important. Previous work has explored recommending interesting locations; however, users would also benefit from recommendations for activities in which to participate at those locations along with suitable times and days. Thus, systems that provide collaborative recommendations involving multiple dimensions such as location, activities and time would enhance the overall experience of users.The relationship among these dimensions can be modeled by higher-order matrices called tensors which are then solved by tensor factorization. However, these tensors can be extremely sparse. In this paper, we present a system and an approach for performing multi-dimensional collaborative recommendations for Who (User), What (Activity), When (Time) and Where (Location), using tensor factorization on sparse user-generated data. We formulate an objective function which simultaneously factorizes coupled tensors and matrices constructed from heterogeneous data sources. We evaluate our system and approach on large-scale real world data sets consisting of 588,000 Flickr photos collected from three major metro regions in USA. We compare our approach with several state-of-the-art baselines and demonstrate that it outperforms all of them.

negative: A System to Verify Network Behavior of Known Cryptographic Clients Numerous exploits of client-server protocols and applications involve modifying clients to behave in ways that untampered clients would not, such as crafting malicious packets. In this paper, we develop a system for verifying in near real-time that a cryptographic client’s message sequence is consistent with its known implementation. Moreover, we accomplish this without knowing all of the client-side inputs driving its behavior. Our toolchain for verifying a client’s messages explores multiple candidate execution paths in the client concurrently, an innovation useful for aspects of certain cryptographic protocols such as message padding (which will be permitted in TLS 1.3). In addition, our toolchain includes a novel approach to symbolically executing the client software in multiple passes that defers expensive functions until their inputs can be inferred and concretized. We demonstrate client verification on OpenSSL and BoringSSL to show that, e.g., Heartbleed exploits can be detected without Heartbleed-specific filtering and within seconds of the first malicious packet. On legitimate traffic our verification keeps pace with Gmail-shaped workloads, with a median lag of 0.85s.

negative: Visually-Enabled Active Deep Learning for (Geo) Text and Image Classification: A Review This paper investigates recent research on active learning for (geo) text and image classification, with an emphasis on methods that combine visual analytics and/or deep learning. Deep learning has attracted substantial attention across many domains of science and practice, because it can find intricate patterns in big data; but successful application of the methods requires a big set of labeled data. Active learning, which has the potential to address the data labeling challenge, has already had success in geospatial applications such as trajectory classification from movement data and (geo) text and image classification. This review is intended to be particularly relevant for extension of these methods to GISience, to support work in domains such as geographic information retrieval from text and image repositories, interpretation of spatial language, and related geo-semantics challenges. Specifically, to provide a structure for leveraging recent advances, we group the relevant work into five categories: active learning, visual analytics, active learning with visual analytics, active deep learning, plus GIScience and Remote Sensing (RS) using active learning and active deep learning. Each category is exemplified by recent influential work. Based on this framing and our systematic review of key research, we then discuss some of the main challenges of integrating active learning with visual analytics and deep learning, and point out research opportunities from technical and application perspectives—for application-based opportunities, with emphasis on those that address big data with geospatial components.

negative: Managing Employee Compliance with Information Security Policies: The Critical Role of Top Management and Organizational Culture We develop an individual behavioral model that integrates the role of top management and organizational culture into the theory of planned behavior in an attempt to better understand how top management can influence security compliance behavior of employees. Using survey data and structural equation modeling, we test hypotheses on the relationships among top management participation, organizational culture, and key determinants of employee compliance with information security policies. We find that top management participation in information security initiatives has significant direct and indirect influences on employees’ attitudes towards, subjective norm of, and perceived behavioral control over compliance with information security policies. We also find that the top management participation strongly influences organizational culture which in turn impacts employees’ attitudes towards and perceived behavioral control over compliance with information security policies. Furthermore, we find that the effects of top management participation and organizational culture on employee behavioral intentions are fully mediated by employee cognitive beliefs about compliance with information security policies. Our findings extend information security research literature by showing how top management can play a proactive role in shaping employee compliance behavior in addition to the deterrence oriented remedies advocated in the extant literature. Our findings also refine the theories about the role of organizational culture in shaping employee compliance behavior. Significant theoretical and practical implications of ∗This project was partially funded by a grant to the authors from the Defense Information Systems Agency (DISA) of the Department of Defense (DoD). The authors express their thanks to the editor, senior editor, associate editor, and two anonymous reviewers for their detailed and constructive comments and suggestions throughout the review process. †Corresponding author.

negative: A Social Network-Based Recommender System (SNRS) Social influence plays an important role in product marketing. However, it has rarely been considered in traditional recommender systems. In this paper we present a new paradigm of recommender systems which can utilize information in social networks, including user preferences, an item's general acceptance, and influence from social friends. A probabilistic model is developed to make personalized recommendations from such information. We extract data from a real online social network, and our analysis of this large dataset reveals that friends have a tendency to select the same items and give similar ratings. Experimental results on this dataset show that our proposed system not only improves the prediction accuracy of recommender systems but also remedies the data sparsity and cold-start issues inherent in collaborative filtering. Furthermore, we propose to improve the performance of our system by applying semantic filtering of social networks, and validate its improvement via a class project experiment. In this experiment we demonstrate how relevant friends can be selected for inference based on the semantics of friend relationships and finer-grained user ratings. Such technologies can be deployed by most

negative: CRCTOL: A semantic-based domain ontology learning system Keywords: ontologies < controlled vocabularies < index languages < language types < (language), knowledge acquisition < knowledge engineering < artificial intelligence < computer applications < computer operations < (activities and operations)

negative: Learning Inference Models for Computer Vision Computer vision can be understood as the ability to perform inference on image data. Breakthroughs in computer vision technology are often marked by advances in inference techniques, as even the model design is often dictated by the complexity of inference in them. This thesis proposes learning based inference schemes and demonstrates applications in computer vision. We propose techniques for inference in both generative and discriminative computer vision models. Despite their intuitive appeal, the use of generative models in vision is hampered by the difficulty of posterior inference, which is often too complex or too slow to be practical. We propose techniques for improving inference in two widely used techniques: Markov Chain Monte Carlo (MCMC) sampling and message-passing inference. Our inference strategy is to learn separate discriminative models that assist Bayesian inference in a generative model. Experiments on a range of generative vision models show that the proposed techniques accelerate the inference process and/or converge to better solutions. A main complication in the design of discriminative models is the inclusion of prior knowledge in a principled way. For better inference in discriminative models, we propose techniques that modify the original model itself, as inference is simple evaluation of the model. We concentrate on convolutional neural network (CNN) models and propose a generalization of standard spatial convolutions, which are the basic building blocks of CNN architectures, to bilateral convolutions. First, we generalize the existing use of bilateral filters and then propose new neural network architectures with learnable bilateral filters, which we call ‘Bilateral Neural Networks’. We show how the bilateral filtering modules can be used for modifying existing CNN architectures for better image segmentation and propose a neural network approach for temporal information propagation in videos. Experiments demonstrate the potential of the proposed bilateral networks on a wide range of vision tasks and datasets. In summary, we propose learning based techniques for better inference in several computer vision models ranging from inverse graphics to freely parameterized neural networks. In generative vision models, our inference techniques alleviate some of the crucial hurdles in Bayesian posterior inference, paving new ways for the use of model based machine learning in vision. In discriminative CNN models, the proposed filter generalizations aid in the design of new neural network architectures that can handle sparse high-dimensional data as well as provide a way for incorporating prior knowledge into CNNs.

negative: Advances in Variational Inference Many modern unsupervised or semi-supervised machine learning algorithms rely on Bayesian probabilistic models. These models are usually intractable and thus require approximate inference. Variational inference (VI) lets us approximate a high-dimensional Bayesian posterior with a simpler variational distribution by solving an optimization problem. This approach has been successfully used in various models and large-scale applications. In this review, we give an overview of recent trends in variational inference. We first introduce standard mean field variational inference, then review recent advances focusing on the following aspects: (a) scalable VI, which includes stochastic approximations, (b) generic VI, which extends the applicability of VI to a large class of otherwise intractable models, such as non-conjugate models, (c) accurate VI, which includes variational models beyond the mean field approximation or with atypical divergences, and (d) amortized VI, which implements the inference over local latent variables with inference networks. Finally, we provide a summary of promising future research directions.

negative: Bayesian Modeling with Gaussian Processes using the GPstuff Toolbox Gaussian processes (GP) are powerful tools for probabilistic modeling purposes. They can be used to define prior distributions over latent functions in hierarchical Bayesian models. The prior over functions is defined implicitly by the mean and covariance function, which determine the smoothness and variability of the function. The inference can then be conducted directly in the function space by evaluating or approximating the posterior process. Despite their attractive theoretical properties GPs provide practical challenges in their implementation. GPstuff is a versatile collection of computational tools for GP models compatible with Linux and Windows MATLAB and Octave. It includes, among others, various inference methods, sparse approximations and tools for model assessment. In this work, we review these tools and demonstrate the use of GPstuff in several models.

negative: Likelihood-informed dimension reduction for nonlinear inverse problems The intrinsic dimensionality of an inverse problem is affected by prior information, the accuracy and number of observations, and the smoothing properties of the forward operator. From a Bayesian perspective, changes from the prior to the posterior may, in many problems, be confined to a relatively lowdimensional subspace of the parameter space. We present a dimension reduction approach that defines and identifies such a subspace, called the “likelihoodinformed subspace” (LIS), by characterizing the relative influences of the prior and the likelihood over the support of the posterior distribution. This identification enables new and more efficient computational methods for Bayesian inference with nonlinear forward models and Gaussian priors. In particular, we approximate the posterior distribution as the product of a lower-dimensional posterior defined on the LIS and the prior distribution marginalized onto the complementary subspace. Markov chain Monte Carlo sampling can then proceed in lower dimensions, with significant gains in computational efficiency. We also introduce a Rao-Blackwellization strategy that de-randomizes Monte Carlo estimates of posterior expectations for additional variance reduction. We demonstrate the efficiency of our methods using two numerical examples: inference of permeability in a groundwater system governed by an elliptic PDE, and an atmospheric remote sensing problem based on Global Ozone Monitoring System (GOMOS) observations.

negative: Communication-Efficient Distributed Statistical Inference We present a Communication-efficient Surrogate Likelihood (CSL) framework for solving distributed statistical inference problems. CSL provides a communication-efficient surrogate to the global likelihood that can be used for low-dimensional estimation, high-dimensional regularized estimation and Bayesian inference. For low-dimensional estimation, CSL provably improves upon naive averaging schemes and facilitates the construction of confidence intervals. For high-dimensional regularized estimation, CSL leads to a minimax-optimal estimator with controlled communication cost. For Bayesian inference, CSL can be used to form a communication-efficient quasi-posterior distribution that converges to the true posterior. This quasi-posterior procedure significantly improves the computational efficiency of MCMC algorithms even in a non-distributed setting. We present both theoretical analysis and experiments to explore the properties of the CSL approximation.

negative: Variational Bayesian Inference for Big Data Marketing Models 1 Hierarchical Bayesian approaches play a central role in empirical marketing as they yield individual-level parameter estimates that can be used for targeting decisions. MCMC methods have been the methods of choice for estimating hierarchical Bayesian models as they are capable of providing accurate individual-level estimates. However, MCMC methods are computationally prohibitive and do not scale well when applied to massive data sets that have become common in the current era of “Big Data”. We introduce to the marketing literature a new class of Bayesian estimation techniques known as variational Bayesian (VB) inference. These methods tackle the scalability challenge via a deterministic optimization approach to approximate the posterior distribution and yield accurate estimates at a fraction of the computational cost associated with simulation-based MCMC methods. We exploit and extend recent developments in variational Bayesian inference and highlight how two VB estimation approaches – Mean-field VB (that is analogous to Gibbs sampling) for conjugate models and Fixed-form VB (which is analogous to Metropolis-Hasting) for nonconjugate models – can be effectively combined for estimating complex marketing models. We also show how recent advances in parallel computing and in stochastic optimization can be used to further enhance the speed of these VB methods. Using simulated as well as real data sets, we apply the VB approaches to several commonly used marketing models (e.g. mixed linear, logit, selection, and hierarchical ordinal logit models), and demonstrate how the VB inference is widely applicable for marketing problems.

negative: Parameter estimation for text analysis Presents parameter estimation methods common with discrete probability distributions, which is of particular interest in text modeling. Starting with maximum likelihood, a posteriori and Bayesian estimation, central concepts like conjugate distributions and Bayesian networks are reviewed. As an application, the model of latent Dirichlet allocation (LDA) is explained in detail with a full derivation of an approximate inference algorithm based on Gibbs sampling, including a discussion of Dirichlet hyperparameter estimation. History: version 1: May 2005, version 2.4: August 2008.