Supélec

International audience

We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including: part-of-speech tagging, chunking, named entity recognition, and semantic role labeling. This versatility is achieved by trying to avoid task-specific engineering and therefore disregarding a lot of prior knowledge. Instead of exploiting man-made input features carefully optimized for each task, our system learns internal representations on the basis of vast amounts of mostly unlabeled training data. This work is then used as a basis for building a freely available tagging system with good performance and minimal computational requirements.

Editor: We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including: part-of-speech tagging, chunking, named entity recognition, and semantic role labeling, achieving or exceeding state-of-theart performance in each on four benchmark tasks. Our goal was to design a flexible architecture that can learn representations useful for the tasks, thus avoiding excessive taskspecific feature engineering (and therefore disregarding a lot of prior knowledge). Instead of exploiting man-made input features carefully optimized for each task, our system learns internal representations on the basis of vast amounts of mostly unlabelled training data. This work is then used as a basis for building a freely available tagging system with excellent performance while requiring minimal computational resources. Keywords:

This article offers an empirical exploration on the use of character-level convolutional networks (ConvNets) for text classification. We constructed several large-scale datasets to show that character-level convolutional networks could achieve state-of-the-art or competitive results. Comparisons are offered against traditional models such as bag of words, n-grams and their TFIDF variants, and deep learning models such as word-based ConvNets and recurrent neural networks.

We introduce flat systems, which are equivalent to linear ones via a special type of feedback called endogenous. Their physical properties are subsumed by a linearizing output and they might be regarded as providing another nonlinear extension of Kalman’s controllability. The distance to flatness is measured by a non-negative integer, the defect. We utilize differential algebra which suits well to the fact that, in accordance with Willems ’ standpoint, flatness and defect are best defined without distinguishing between input, state, output and other variables. Many realistic classes of examples are flat. We treat two popular ones: the crane and the car with n trailers, the motion planning of which is obtained via elementary properties of planar curves. The three non-flat examples, the simple, double and variable length pendulums, are borrowed from nonlinear physics. A high frequency control strategy is proposed such that the averaged systems become flat. ∗This work was partially supported by the G.R. “Automatique ” of the CNRS and by the D.R.E.D. of the “Ministère de l’Éducation Nationale”. 1 1

We consider the uplink (UL) and downlink (DL) of non-cooperative multi-cellular time-division duplexing (TDD) systems, assuming that the number N of antennas per base station (BS) and the number K of user terminals (UTs) per cell are large. Our system model accounts for channel estimation, pilot contamination, and an arbitrary path loss and antenna correlation for each link. We derive approximations of achievable rates with several linear precoders and detectors which are proven to be asymptotically tight, but accurate for realistic system dimensions, as shown by simulations. It is known from previous work assuming uncorrelated channels, that as N→∞ while K is fixed, the system performance is limited by pilot contamination, the simplest precoders/detectors, i.e., eigenbeamforming (BF) and matched filter (MF), are optimal, and the transmit power can be made arbitrarily small. We analyze to which extent these conclusions hold in the more realistic setting where N is not extremely large compared to K. In particular, we derive how many antennas per UT are needed to achieve η% of the ultimate performance limit with infinitely many antennas and how many more antennas are needed with MF and BF to achieve the performance of minimum mean-square error (MMSE) detection and regularized zero-forcing (RZF), respectively.

ABSTRACT Many financial markets are characterized by strong relationships and networks, rather than arm's‐length, spot market transactions. We examine the performance consequences of this organizational structure in the context of relationships established when VCs syndicate portfolio company investments. We find that better‐networked VC firms experience significantly better fund performance, as measured by the proportion of investments that are successfully exited through an IPO or a sale to another company. Similarly, the portfolio companies of better‐networked VCs are significantly more likely to survive to subsequent financing and eventual exit. We also provide initial evidence on the evolution of VC networks.

Human ability to understand language is general, flexible, and robust. In contrast, most NLU models above the word level are designed for a specific task and struggle with out-of-domain data. If we aspire to develop models with understanding beyond the detection of superficial correspondences between inputs and outputs, then it is critical to develop a unified model that can execute a range of linguistic tasks across different domains. To facilitate research in this direction, we present the General Language Understanding Evaluation (GLUE, gluebenchmark.com): a benchmark of nine diverse NLU tasks, an auxiliary dataset for probing models for understanding of specific linguistic phenomena, and an online platform for evaluating and comparing models. For some benchmark tasks, training data is plentiful, but for others it is limited or does not match the genre of the test set. GLUE thus favors models that can represent linguistic knowledge in a way that facilitates sample-efficient learning and effective knowledge-transfer across tasks. While none of the datasets in GLUE were created from scratch for the benchmark, four of them feature privately-held test data, which is used to ensure that the benchmark is used fairly. We evaluate baselines that use ELMo (Peters et al., 2018), a powerful transfer learning technique, as well as state-of-the-art sentence representation models. The best models still achieve fairly low absolute scores. Analysis with our diagnostic dataset yields similarly weak performance over all phenomena tested, with some exceptions.

A key challenge of future mobile communication research is to strike an attractive compromise between wireless network's area spectral efficiency and energy efficiency. This necessitates a clean-slate approach to wireless system design, embracing the rich body of existing knowledge, especially on multiple-input-multiple-ouput (MIMO) technologies. This motivates the proposal of an emerging wireless communications concept conceived for single-radio-frequency (RF) large-scale MIMO communications, which is termed as SM. The concept of SM has established itself as a beneficial transmission paradigm, subsuming numerous members of the MIMO system family. The research of SM has reached sufficient maturity to motivate its comparison to state-of-the-art MIMO communications, as well as to inspire its application to other emerging wireless systems such as relay-aided, cooperative, small-cell, optical wireless, and power-efficient communications. Furthermore, it has received sufficient research attention to be implemented in testbeds, and it holds the promise of stimulating further vigorous interdisciplinary research in the years to come. This tutorial paper is intended to offer a comprehensive state-of-the-art survey on SM-MIMO research, to provide a critical appraisal of its potential advantages, and to promote the discussion of its beneficial application areas and their research challenges leading to the analysis of the technological issues associated with the implementation of SM-MIMO. The paper is concluded with the description of the world's first experimental activities in this vibrant research field.

This article explores one of the key enablers of beyond 4G wireless networks leveraging small cell network deployments, proactive caching. Endowed with predictive capabilities and harnessing recent developments in storage, context awareness, and social networks, peak traffic demands can be substantially reduced by proactively serving predictable user demands via caching at base stations and users' devices. In order to show the effectiveness of proactive caching, we examine two case studies that exploit the spatial and social structure of the network, where proactive caching plays a crucial role. First, in order to alleviate backhaul congestion, we propose a mechanism whereby files are proactively cached during off-peak periods based on file popularity and correlations among user and file patterns. Second, leveraging social networks and D2D communications, we propose a procedure that exploits the social structure of the network by predicting the set of influential users to (proactively) cache strategic contents and disseminate them to their social ties via D2D communications. Exploiting this proactive caching paradigm, numerical results show that important gains can be obtained for each case study, with backhaul savings and a higher ratio of satisfied users of up to 22 and 26 percent, respectively. Higher gains can be further obtained by increasing the storage capability at the network edge.

In this letter, the efficient deployment of multiple unmanned aerial vehicles (UAVs) acting as wireless base stations that provide coverage for ground users is analyzed. First, the downlink coverage probability for UAVs as a function of the altitude and the antenna gain is derived. Next, using circle packing theory, the 3-D locations of the UAVs is determined in a way that the total coverage area is maximized while maximizing the coverage lifetime of the UAVs. Our results show that, in order to mitigate interference, the altitude of the UAVs must be properly adjusted based on the beamwidth of the directional antenna as well as coverage requirements. Furthermore, the minimum number of UAVs needed to guarantee a target coverage probability for a given geographical area is determined. Numerical results evaluate various tradeoffs.

We report on a series of experiments with convolutional neural networks (CNN) trained on top of pre-trained word vectors for sentence-level classification tasks. We show that a simple CNN with little hyperparameter tuning and static vectors achieves excellent results on multiple benchmarks. Learning task-specific vectors through fine-tuning offers further gains in performance. We additionally propose a simple modification to the architecture to allow for the use of both task-specific and static vectors. The CNN models discussed herein improve upon the state of the art on 4 out of 7 tasks, which include sentiment analysis and question classification.

3GPP LTE-Advanced has recently been investigating heterogeneous network (HetNet) deployments as a cost effective way to deal with the unrelenting traffic demand. HetNets consist of a mix of macrocells, remote radio heads, and low-power nodes such as picocells, femtocells, and relays. Leveraging network topology, increasing the proximity between the access network and the end users, has the potential to provide the next significant performance leap in wireless networks, improving spatial spectrum reuse and enhancing indoor coverage. Nevertheless, deployment of a large number of small cells overlaying the macrocells is not without new technical challenges. In this article, we present the concept of heterogeneous networks and also describe the major technical challenges associated with such network architecture. We focus in particular on the standardization activities within the 3GPP related to enhanced intercell interference coordination.

OBJECTIVE: Remission in rheumatoid arthritis (RA) is an increasingly attainable goal, but there is no widely used definition of remission that is stringent but achievable and could be applied uniformly as an outcome measure in clinical trials. This work was undertaken to develop such a definition. METHODS: A committee consisting of members of the American College of Rheumatology, the European League Against Rheumatism, and the Outcome Measures in Rheumatology Initiative met to guide the process and review prespecified analyses from RA clinical trials. The committee requested a stringent definition (little, if any, active disease) and decided to use core set measures including, as a minimum, joint counts and levels of an acute-phase reactant to define remission. Members were surveyed to select the level of each core set measure that would be consistent with remission. Candidate definitions of remission were tested, including those that constituted a number of individual measures of remission (Boolean approach) as well as definitions using disease activity indexes. To select a definition of remission, trial data were analyzed to examine the added contribution of patient-reported outcomes and the ability of candidate measures to predict later good radiographic and functional outcomes. RESULTS: Survey results for the definition of remission suggested indexes at published thresholds and a count of core set measures, with each measure scored as 1 or less (e.g., tender and swollen joint counts, C-reactive protein [CRP] level, and global assessments on a 0-10 scale). Analyses suggested the need to include a patient-reported measure. Examination of 2-year followup data suggested that many candidate definitions performed comparably in terms of predicting later good radiographic and functional outcomes, although 28-joint Disease Activity Score-based measures of remission did not predict good radiographic outcomes as well as the other candidate definitions did. Given these and other considerations, we propose that a patient's RA can be defined as being in remission based on one of two definitions: (a) when scores on the tender joint count, swollen joint count, CRP (in mg/dl), and patient global assessment (0-10 scale) are all ≤ 1, or (b) when the score on the Simplified Disease Activity Index is ≤ 3.3. CONCLUSION: We propose two new definitions of remission, both of which can be uniformly applied and widely used in RA clinical trials. We recommend that one of these be selected as an outcome measure in each trial and that the results on both be reported for each trial.

We describe an image compression method, consisting of a nonlinear analysis transformation, a uniform quantizer, and a nonlinear synthesis transformation. The transforms are constructed in three successive stages of convolutional linear filters and nonlinear activation functions. Unlike most convolutional neural networks, the joint nonlinearity is chosen to implement a form of local gain control, inspired by those used to model biological neurons. Using a variant of stochastic gradient descent, we jointly optimize the entire model for rate-distortion performance over a database of training images, introducing a continuous proxy for the discontinuous loss function arising from the quantizer. Under certain conditions, the relaxed loss function may be interpreted as the log likelihood of a generative model, as implemented by a variational autoencoder. Unlike these models, however, the compression model must operate at any given point along the rate-distortion curve, as specified by a trade-off parameter. Across an independent set of test images, we find that the optimized method generally exhibits better rate-distortion performance than the standard JPEG and JPEG 2000 compression methods. More importantly, we observe a dramatic improvement in visual quality for all images at all bit rates, which is supported by objective quality estimates using MS-SSIM.

Blending theoretical results with practical applications, this book provides an introduction to random matrix theory and shows how it can be used to tackle a variety of problems in wireless communications. The Stieltjes transform method, free probability theory, combinatoric approaches, deterministic equivalents and spectral analysis methods for statistical inference are all covered from a unique engineering perspective. Detailed mathematical derivations are presented throughout, with thorough explanation of the key results and all fundamental lemmas required for the reader to derive similar calculus on their own. These core theoretical concepts are then applied to a wide range of real-world problems in signal processing and wireless communications, including performance analysis of CDMA, MIMO and multi-cell networks, as well as signal detection and estimation in cognitive radio networks. The rigorous yet intuitive style helps demonstrate to students and researchers alike how to choose the correct approach for obtaining mathematically accurate results.

In the last year, new models and methods for pretraining and transfer learning have driven striking performance improvements across a range of language understanding tasks. The GLUE benchmark, introduced a little over one year ago, offers a single-number metric that summarizes progress on a diverse set of such tasks, but performance on the benchmark has recently surpassed the level of non-expert humans, suggesting limited headroom for further research. In this paper we present SuperGLUE, a new benchmark styled after GLUE with a new set of more difficult language understanding tasks, a software toolkit, and a public leaderboard. SuperGLUE is available at super.gluebenchmark.com.

The authors provide a theoretical interpretation of two features of international data: the countercyclical movements in net exports and the tendency for the trade balance to be negatively correlated with current and future movements in terms of trade but positively correlated with past movements. They document the same properties in a two-country stochastic growth model in which trade fluctuations reflect, in large part, the dynamics of capital formation. The authors find that their general-equilibrium perspective is essential: the relation between the trade balance and the terms of trade depends critically on the source of fluctuations. Copyright 1994 by American Economic Association.

The use of large-scale antenna arrays can bring substantial improvements in energy and/or spectral efficiency to wireless systems due to the greatly improved spatial resolution and array gain. Recent works in the field of massive multiple-input multiple-output (MIMO) show that the user channels decorrelate when the number of antennas at the base stations (BSs) increases, thus strong signal gains are achievable with little interuser interference. Since these results rely on asymptotics, it is important to investigate whether the conventional system models are reasonable in this asymptotic regime. This paper considers a new system model that incorporates general transceiver hardware impairments at both the BSs (equipped with large antenna arrays) and the single-antenna user equipments (UEs). As opposed to the conventional case of ideal hardware, we show that hardware impairments create finite ceilings on the channel estimation accuracy and on the downlink/uplink capacity of each UE. Surprisingly, the capacity is mainly limited by the hardware at the UE, while the impact of impairments in the large-scale arrays vanishes asymptotically and interuser interference (in particular, pilot contamination) becomes negligible. Furthermore, we prove that the huge degrees of freedom offered by massive MIMO can be used to reduce the transmit power and/or to tolerate larger hardware impairments, which allows for the use of inexpensive and energy-efficient antenna elements.

The paradox of a host specificity of the human faecal microbiota otherwise acknowledged as characterized by global functionalities conserved between humans led us to explore the existence of a phylogenetic core. We investigated the presence of a set of bacterial molecular species that would be altogether dominant and prevalent within the faecal microbiota of healthy humans. A total of 10 456 non-chimeric bacterial 16S rRNA sequences were obtained after cloning of PCR-amplified rDNA from 17 human faecal DNA samples. Using alignment or tetranucleotide frequency-based methods, 3180 operational taxonomic units (OTUs) were detected. The 16S rRNA sequences mainly belonged to the phyla Firmicutes (79.4%), Bacteroidetes (16.9%), Actinobacteria (2.5%), Proteobacteria (1%) and Verrumicrobia (0.1%). Interestingly, while most of OTUs appeared individual-specific, 2.1% were present in more than 50% of the samples and accounted for 35.8% of the total sequences. These 66 dominant and prevalent OTUs included members of the genera Faecalibacterium, Ruminococcus, Eubacterium, Dorea, Bacteroides, Alistipes and Bifidobacterium. Furthermore, 24 OTUs had cultured type strains representatives which should be subjected to genome sequence with a high degree of priority. Strikingly, 52 of these 66 OTUs were detected in at least three out of four recently published human faecal microbiota data sets, obtained with very different experimental procedures. A statistical model confirmed these OTUs prevalence. Despite the species richness and a high individual specificity, a limited number of OTUs is shared among individuals and might represent the phylogenetic core of the human intestinal microbiota. Its role in human health deserves further study.