Amazon (United Kingdom)

Recent advances in Natural Language Processing and Machine Learning provide us with the tools to build predictive models that can be used to unveil patterns driving judicial decisions. This can be useful, for both lawyers and judges, as an assisting tool to rapidly identify cases and extract patterns which lead to certain decisions. This paper presents the first systematic study on predicting the outcome of cases tried by the European Court of Human Rights based solely on textual content. We formulate a binary classification task where the input of our classifiers is the textual content extracted from a case and the target output is the actual judgment as to whether there has been a violation of an article of the convention of human rights. Textual information is represented using contiguous word sequences, i.e., N-grams, and topics. Our models can predict the court’s decisions with a strong accuracy (79% on average). Our empirical analysis indicates that the formal facts of a case are the most important predictive factor. This is consistent with the theory of legal realism suggesting that judicial decision-making is significantly affected by the stimulus of the facts. We also observe that the topical content of a case is another important feature in this classification task and explore this relationship further by conducting a qualitative analysis.

Transferring knowledge from a teacher neural network pretrained on the same or a similar task to a student neural network can significantly improve the performance of the student neural network. Existing knowledge transfer approaches match the activations or the corresponding hand-crafted features of the teacher and the student networks. We propose an information-theoretic framework for knowledge transfer which formulates knowledge transfer as maximizing the mutual information between the teacher and the student networks. We compare our method with existing knowledge transfer methods on both knowledge distillation and transfer learning tasks and show that our method consistently outperforms existing methods. We further demonstrate the strength of our method on knowledge transfer across heterogeneous network architectures by transferring knowledge from a convolutional neural network (CNN) to a multi-layer perceptron (MLP) on CIFAR-10. The resulting MLP significantly outperforms the-state-of-the-art methods and it achieves similar performance to the CNN with a single convolutional layer.

Multi-fidelity modelling enables accurate inference of quantities of interest by synergistically combining realizations of low-cost/low-fidelity models with a small set of high-fidelity observations. This is particularly effective when the low- and high-fidelity models exhibit strong correlations, and can lead to significant computational gains over approaches that solely rely on high-fidelity models. However, in many cases of practical interest, low-fidelity models can only be well correlated to their high-fidelity counterparts for a specific range of input parameters, and potentially return wrong trends and erroneous predictions if probed outside of their validity regime. Here we put forth a probabilistic framework based on Gaussian process regression and nonlinear autoregressive schemes that is capable of learning complex nonlinear and space-dependent cross-correlations between models of variable fidelity, and can effectively safeguard against low-fidelity models that provide wrong trends. This introduces a new class of multi-fidelity information fusion algorithms that provide a fundamental extension to the existing linear autoregressive methodologies, while still maintaining the same algorithmic complexity and overall computational cost. The performance of the proposed methods is tested in several benchmark problems involving both synthetic and real multi-fidelity datasets from computational fluid dynamics simulations.

We assess costs and efficiency of state-of-the-art high-performance cloud computing and compare the results to traditional on-premises compute clusters. Our use case is atomistic simulations carried out with the GROMACS molecular dynamics (MD) toolkit with a particular focus on alchemical protein-ligand binding free energy calculations. We set up a compute cluster in the Amazon Web Services (AWS) cloud that incorporates various different instances with Intel, AMD, and ARM CPUs, some with GPU acceleration. Using representative biomolecular simulation systems, we benchmark how GROMACS performs on individual instances and across multiple instances. Thereby we assess which instances deliver the highest performance and which are the most cost-efficient ones for our use case. We find that, in terms of total costs, including hardware, personnel, room, energy, and cooling, producing MD trajectories in the cloud can be about as cost-efficient as an on-premises cluster given that optimal cloud instances are chosen. Further, we find that high-throughput ligand-screening can be accelerated dramatically by using global cloud resources. For a ligand screening study consisting of 19 872 independent simulations or ∼200 μs of combined simulation trajectory, we made use of diverse hardware available in the cloud at the time of the study. The computations scaled-up to reach peak performance using more than 4 000 instances, 140 000 cores, and 3 000 GPUs simultaneously. Our simulation ensemble finished in about 2 days in the cloud, while weeks would be required to complete the task on a typical on-premises cluster consisting of several hundred nodes.

The Gaussian mechanism is an essential building block used in multitude of\ndifferentially private data analysis algorithms. In this paper we revisit the\nGaussian mechanism and show that the original analysis has several important\nlimitations. Our analysis reveals that the variance formula for the original\nmechanism is far from tight in the high privacy regime ($\\varepsilon \\to 0$)\nand it cannot be extended to the low privacy regime ($\\varepsilon \\to \\infty$).\nWe address these limitations by developing an optimal Gaussian mechanism whose\nvariance is calibrated directly using the Gaussian cumulative density function\ninstead of a tail bound approximation. We also propose to equip the Gaussian\nmechanism with a post-processing step based on adaptive estimation techniques\nby leveraging that the distribution of the perturbation is known. Our\nexperiments show that analytical calibration removes at least a third of the\nvariance of the noise compared to the classical Gaussian mechanism, and that\ndenoising dramatically improves the accuracy of the Gaussian mechanism in the\nhigh-dimensional regime.\n

This paper serves as a survey and empirical evaluation of the state-of-the-art in activity recognition methods using accelerometers. The paper is particularly focused on long-term activity recognition in real-world settings. In these environments, data collection is not a trivial matter; thus, there are performance trade-offs between prediction accuracy, which is not the sole system objective, and keeping the maintenance overhead at minimum levels. We examine research that has focused on the selection of activities, the features that are extracted from the accelerometer data, the segmentation of the time-series data, the locations of accelerometers, the selection and configuration trade-offs, the test/retest reliability, and the generalisation performance. Furthermore, we study these questions from an experimental platform and show, somewhat surprisingly, that many disparate experimental configurations yield comparable predictive performance on testing data. Our understanding of these results is that the experimental setup directly and indirectly defines a pathway for context to be delivered to the classifier, and that, in some settings, certain configurations are more optimal than alternatives. We conclude by identifying how the main results of this work can be used in practice, specifically in experimental configurations in challenging experimental conditions.

We examined the effects of acute low‐pH exposure on ion balance (Na+, Cl−, K+) in several species of fish captured from the Rio Negro, a dilute, acidic tributary of the Amazon. At pH 5.5 (untreated Rio Negro water), the four Rio Negro species tested (piranha preta, Serrasalmus rhombeus; piranha branca, Serrasalmus cf. holandi; aracu, Leporinus fasciatus; and pacu, Myleus sp.) were at or near ion balance; upon exposure to pH 3.5, while Na+ and Cl− loss rates became significant, they were relatively mild. In comparison, tambaqui (Colossoma macropo‐mum), which were obtained from aquaculture and held and tested under the same conditions as the other fish, had loss rates seven times higher than all the Rio Negro species. At pH 3.0, rates of Na+ and Cl− loss for the Rio Negro fish increased three‐ to fivefold but were again much less than those observed in tambaqui. Raising water Ca2+ concentration from 10 mmol L−1 to 100 mmol L−1 during exposure to the same low pH's had no effect on rates of ion loss in the three species tested (piranha preta, piranha branca, aracu), which suggests that either they have such a high branchial affinity for Ca2+ that all sites are saturated at 10 mmol L−1 and additional Ca2+ had no effect, or that Ca2+ may not be involved in regulation of branchial ion permeability. For a final Rio Negro species, the cardinal tetra (Paracheirodon axelrodi), we monitored body Na+ concentration during 5 d of exposure to pH 6.0, 4.0, or 3.5. These pH's had no effect on body Na+ concentration. These data together suggest that exceptional acid tolerance is a general characteristic of fish that inhabit the dilute acidic Rio Negro and raise questions about the role of Ca2+ in regulation of branchial ion permeability in these fish.

The role of straw bale as a construction material for reducing the whole-life impacts of housing is examined. The embodied and operational CO2 emissions in a recently completed UK social housing project are compared using alternative domestic external wall constructions and the effects on the resulting CO2 emissions. It is estimated that over 15 tonnes of CO2 may be stored in biotic materials of each of the semi-detached houses, of which around 6 tonnes are sequestered by straw and the remaining by wood and wood products. This suggests the carbon lock-up potential of renewable construction materials is capable of reducing the case study house's whole-life CO2 emissions of the house over its 60-year design life by 61% when compared with the case without sequestration. The practical implications of construction, detailing, maintenance, cost and self-build potentials of straw-bale construction are also considered. The potential for load-bearing straw-bale walls is examined through the whole-life performance of straw-bale construction with alternative conventional external walling systems. Le rôle de la balle de paille comme matériau de construction pour réduire l'impact des logements en termes de cycle de vie est examiné. Les émissions de CO2 intrinsèques et opérationnelles dans un ensemble de logements sociaux récemment achevé au Royaume-Uni sont comparées, en utilisant des méthodes différentes de construction des murs extérieurs des logements et les effets des émissions de CO2 qui en résultent. Il est estimé qu'il est possible de stocker plus de 15 tonnes de CO2 dans les matériaux biotiques de chacun de ces pavillons jumelés, dont 6 tonnes environ sont séquestrées par la paille, le reste l'étant par le bois et les produits ligneux. Ceci suggère que les possibilités de séquestration du carbone offertes par les matériaux de construction renouvelables sont capables de réduire de 61 % les émissions de CO2 durant le cycle de vie des maisons de cette étude de cas, pour une durée de vie nominale des maisons de 60 ans, par comparaison avec le cas de figure sans séquestration. Sont également envisagées les implications pratiques des possibilités qu'offre une construction utilisant des balles de paille en termes de construction, de détails de construction, d'entretien, de coût et d'autoconstruction. Le potentiel d'utilisation de balles de paille pour les murs porteurs est étudié sous l'angle des performances, tout au long du cycle de vie, de constructions en balles de paille faisant appel à différents systèmes classiques de construction des murs extérieurs. Mots clés: émissions de CO2, coût, énergie intrinsèque, énergie opérationnelle, CO2 séquestré, logement social, balle de paille

Formal methods encompass a wide choice of techniques and tools for the specification, development, analysis, and verification of software and hardware systems. Formal methods are widely applied in industry, in activities ranging from the elicitation of requirements and the early design phases all the way to the deployment, configuration, and runtime monitoring of actual systems. Formal methods allow one to precisely specify the environment in which a system operates, the requirements and properties that the system should satisfy, the models of the system used during the various design steps, and the code embedded in the final implementation, as well as to express conformance relations between these specifications. We present a broad scope of successful applications of formal methods in industry, not limited to the well-known success stories from the safety-critical domain, like railways and other transportation systems, but also covering other areas such as lithography manufacturing and cloud security in e-commerce, to name but a few. We also report testimonies from a number of representatives from industry who, either directly or indirectly, use or have used formal methods in their industrial project endeavours. These persons are spread geographically, including Europe, Asia, North and South America, and the involved projects witness the large coverage of applications of formal methods, not limited to the safety-critical domain. We thus make a case for the importance of formal methods, and in particular of the capacity to abstract and mathematical reasoning that are taught as part of any formal methods course. These are fundamental Computer Science skills that graduates should profit from when working as computer scientists in industry, as confirmed by industry representatives.

Prosody Transfer (PT) is a technique that aims to use the prosody from a source audio as a reference while synthesising speech. Fine-grained PT aims at capturing prosodic aspects like rhythm, emphasis, melody, duration, and loudness, from a source audio at a very granular level and transferring them when synthesising speech in a different target speaker's voice. Current approaches for fine-grained PT suffer from source speaker leakage, where the synthesised speech has the voice identity of the source speaker as opposed to the target speaker. In order to mitigate this issue, they compromise on the quality of PT. In this paper, we propose CopyCat, a novel, many-to-many PT system that is robust to source speaker leakage, without using parallel data. We achieve this through a novel reference encoder architecture capable of capturing temporal prosodic representations which are robust to source speaker leakage. We compare CopyCat against a state-of-the-art fine-grained PT model through various subjective evaluations, where we show a relative improvement of $47\%$ in the quality of prosody transfer and $14\%$ in preserving the target speaker identity, while still maintaining the same naturalness.

This paper accompanies FOSSIL: a software tool for the synthesis of Lyapunov functions and of barrier certificates (or functions) for dynamical systems modelled as differential equations. Lyapunov functions are formal certificates for stability analysis, whereas barrier functions are formal certificates for the safety of dynamical models. FOSSIL is sound and automatic thanks to a counterexample-guided inductive synthesis loop. This method exploits the flexibility of candidate functions generated by training neural network templates, the formal assertions provided by a verifier (namely, an SMT solver), and finally new procedures to ease the exchange of information between the two mentioned components. We endow the tool with features of usability, scalability, and robustness---all of which are showcased on benchmarks.

Abstract The ability to generalize well is one of the primary desiderata for models of natural language processing (NLP), but what ‘good generalization’ entails and how it should be evaluated is not well understood. In this Analysis we present a taxonomy for characterizing and understanding generalization research in NLP. The proposed taxonomy is based on an extensive literature review and contains five axes along which generalization studies can differ: their main motivation, the type of generalization they aim to solve, the type of data shift they consider, the source by which this data shift originated, and the locus of the shift within the NLP modelling pipeline. We use our taxonomy to classify over 700 experiments, and we use the results to present an in-depth analysis that maps out the current state of generalization research in NLP and make recommendations for which areas deserve attention in the future.

Gastronomy is the art and science of good eating and drinking: a concept that extends outwards to embrace wider notions of tradition, culture, society and civilisation. This book provides a rigorous, well researched and much needed treatment of the subject, systematically outlining: * the development of European gastronomic tradition, and the social, economic, philosophical and geographical contexts of change* the experiences, philosophies and relative contributions of great gastronomes, past and present* the interplay of traditional and contemporary influences on modern gastronomy* the relationship between gastronomy and and travel and tourism* salient issues of nutrition, food hygiene and health promotionTaking an all-encompassing look at the subject of gastronomy past, present and future, 'European Gastronomy into the 21st Century' uses example menus and case studies to demonstrate the theory. It also provides an insight into the business arena, using key destination restaurants to illustrate management techniques and marketing issues. Accessible and highly structured, the book guides the reader through its wide-ranging and thought-provoking content.

Abstract In the past two decades, there has been increasing use of semantic networks in engineering design for supporting various activities, such as knowledge extraction, prior art search, idea generation and evaluation. Leveraging large-scale pre-trained graph knowledge databases to support engineering design-related natural language processing (NLP) tasks has attracted a growing interest in the engineering design research community. Therefore, this paper aims to provide a survey of the state-of-the-art semantic networks for engineering design and propositions of future research to build and utilize large-scale semantic networks as knowledge bases to support engineering design research and practice. The survey shows that WordNet, ConceptNet and other semantic networks, which contain common-sense knowledge or are trained on non-engineering data sources, are primarily used by engineering design researchers to develop methods and tools. Meanwhile, there are emerging efforts in constructing engineering and technical-contextualized semantic network databases, such as B-Link and TechNet, through retrieving data from technical data sources and employing unsupervised machine learning approaches. On this basis, we recommend six strategic future research directions to advance the development and uses of large-scale semantic networks for artificial intelligence applications in engineering design.

This paper presents a generic Bayesian framework that enables any deep learning model to actively learn from targeted crowds. Our framework inherits from recent advances in Bayesian deep learning, and extends existing work by considering the targeted crowdsourcing approach, where multiple annotators with unknown expertise contribute an uncontrolled amount (often limited) of annotations. Our framework leverages the low-rank structure in annotations to learn individual annotator expertise, which then helps to infer the true labels from noisy and sparse annotations. It provides a unified Bayesian model to simultaneously infer the true labels and train the deep learning model in order to reach an optimal learning efficacy. Finally, our framework exploits the uncertainty of the deep learning model during prediction as well as the annotators» estimated expertise to minimize the number of required annotations and annotators for optimally training the deep learning model. We evaluate the effectiveness of our framework for intent classification in Alexa (Amazon»s personal assistant), using both synthetic and real-world datasets. Experiments show that our framework can accurately learn annotator expertise, infer true labels, and effectively reduce the amount of annotations in model training as compared to state-of-the-art approaches. We further discuss the potential of our proposed framework in bridging machine learning and crowdsourcing towards improved human-in-the-loop systems.

MOTIVATION: The negative binomial distribution has been shown to be a good model for counts data from both bulk and single-cell RNA-sequencing (RNA-seq). Gaussian process (GP) regression provides a useful non-parametric approach for modelling temporal or spatial changes in gene expression. However, currently available GP regression methods that implement negative binomial likelihood models do not scale to the increasingly large datasets being produced by single-cell and spatial transcriptomics. RESULTS: The GPcounts package implements GP regression methods for modelling counts data using a negative binomial likelihood function. Computational efficiency is achieved through the use of variational Bayesian inference. The GP function models changes in the mean of the negative binomial likelihood through a logarithmic link function and the dispersion parameter is fitted by maximum likelihood. We validate the method on simulated time course data, showing better performance to identify changes in over-dispersed counts data than methods based on Gaussian or Poisson likelihoods. To demonstrate temporal inference, we apply GPcounts to single-cell RNA-seq datasets after pseudotime and branching inference. To demonstrate spatial inference, we apply GPcounts to data from the mouse olfactory bulb to identify spatially variable genes and compare to two published GP methods. We also provide the option of modelling additional dropout using a zero-inflated negative binomial. Our results show that GPcounts can be used to model temporal and spatial counts data in cases where simpler Gaussian and Poisson likelihoods are unrealistic. AVAILABILITY AND IMPLEMENTATION: GPcounts is implemented using the GPflow library in Python and is available at https://github.com/ManchesterBioinference/GPcounts along with the data, code and notebooks required to reproduce the results presented here. The version used for this paper is archived at https://doi.org/10.5281/zenodo.5027066. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

With the advent of the SARS-CoV-2 pandemic, Wastewater-Based Epidemiology (WBE) has been applied to track community infection in cities worldwide and has proven succesful as an early warning system for identification of hotspots and changingprevalence of infections (both symptomatic and asymptomatic) at a city or sub-city level. Wastewater is only one of environmental compartments that requires consideration. In this manuscript, we have critically evaluated the knowledge-base and preparedness for building early warning systems in a rapidly urbanising world, with particular attention to Africa, which experiences rapid population growth and urbanisation. We have proposed a Digital Urban Environment Fingerprinting Platform (DUEF) - a new approach in hazard forecasting and early-warning systems for global health risks and an extension to the existing concept of smart cities. The urban environment (especially wastewater) contains a complex mixture of substances including toxic chemicals, infectious biological agents and human excretion products. DUEF assumes that these specific endo- and exogenous residues, anonymously pooled by communities' wastewater, are indicative of community-wide exposure and the resulting effects. DUEF postulates that the measurement of the substances continuously and anonymously pooled by the receiving environment (sewage, surface water, soils and air), can provide near real-time dynamic information about the quantity and type of physical, biological or chemical stressors to which the surveyed systems are exposed, and can create a risk profile on the potential effects of these exposures. Successful development and utilisation of a DUEF globally requires a tiered approach including: Stage I: network building, capacity building, stakeholder engagement as well as a conceptual model, followed by Stage II: DUEF development, Stage III: implementation, and Stage IV: management and utilization. We have identified four key pillars required for the establishment of a DUEF framework: (1) Environmental fingerprints, (2) Socioeconomic fingerprints, (3) Statistics and modelling and (4) Information systems. This manuscript critically evaluates the current knowledge base within each pillar and provides recommendations for further developments with an aim of laying grounds for successful development of global DUEF platforms.

As conversational agents like Siri and Alexa gain in popularity and use, conversation is becoming a more and more important mode of interaction for search. Conversational search shares some features with traditional search, but differs in some important respects: conversational search systems are less likely to return ranked lists of results (a SERP), more likely to involve iterated interactions, and more likely to feature longer, well-formed user queries in the form of natural language questions. Because of these differences, traditional methods for search evaluation (such as the Cranfield paradigm) do not translate easily to conversational search. In this work, we propose a framework for offline evaluation of conversational search, which includes a methodology for creating test collections with relevance judgments, an evaluation measure based on a user interaction model, and an approach to collecting user interaction data to train the model. The framework is based on the idea of “subtopics”, often used to model novelty and diversity in search and recommendation, and the user model is similar to the geometric browsing model introduced by RBP and used in ERR. As far as we know, this is the first work to combine these ideas into a comprehensive framework for offline evaluation of conversational search.

Abstract We introduce an automated, formal, counterexample-based approach to synthesise Barrier Certificates (BC) for the safety verification of continuous and hybrid dynamical models. The approach is underpinned by an inductive framework: this is structured as a sequential loop between a learner, which manipulates a candidate BC structured as a neural network, and a sound verifier, which either certifies the candidate’s validity or generates counter-examples to further guide the learner. We compare the approach against state-of-the-art techniques, over polynomial and non-polynomial dynamical models: the outcomes show that we can synthesise sound BCs up to two orders of magnitude faster, with in particular a stark speedup on the verification engine (up to three orders less), whilst needing a far smaller data set (up to three orders less) for the learning part. Beyond improvements over the state of the art, we further challenge the new approach on a hybrid dynamical model and on larger-dimensional models, and showcase the numerical robustness of our algorithms and codebase.

Multilingual BERT (mBERT) has shown reasonable capability for zero-shot cross-lingual transfer when fine-tuned on downstream tasks. Since mBERT is not pre-trained with explicit cross-lingual supervision, transfer performance can further be improved by aligning mBERT with cross-lingual signal. Prior work proposes several approaches to align contextualised embeddings. In this paper we analyse how different forms of cross-lingual supervision and various alignment methods influence the transfer capability of mBERT in zero-shot setting. Specifically, we compare parallel corpora vs. dictionary-based supervision and rotational vs. fine-tuning based alignment methods. We evaluate the performance of different alignment methodologies across eight languages on two tasks: Name Entity Recognition and Semantic Slot Filling. In addition, we propose a novel normalisation method which consistently improves the performance of rotation-based alignment including a notable 3% F1 improvement for distant and typologically dissimilar languages. Importantly we identify the biases of the alignment methods to the type of task and proximity to the transfer language. We also find that supervision from parallel corpus is generally superior to dictionary alignments.