Brno University of Technology

We propose two novel model architectures for computing continuous vector representations of words from very large data sets. The quality of these representations is measured in a word similarity task, and the results are compared to the previously best performing techniques based on different types of neural networks. We observe large improvements in accuracy at much lower computational cost, i.e. it takes less than a day to learn high quality word vectors from a 1.6 billion words data set. Furthermore, we show that these vectors provide state-of-the-art performance on our test set for measuring syntactic and semantic word similarities.

We propose two novel model architectures for computing continuous vector representations of words from very large data sets. The quality of these representations is measured in a word similarity task, and the results are compared to the previ-ously best performing techniques based on different types of neural networks. We observe large improvements in accuracy at much lower computational cost, i.e. it takes less than a day to learn high quality word vectors from a 1.6 billion words data set. Furthermore, we show that these vectors provide state-of-the-art perfor-mance on our test set for measuring syntactic and semantic word similarities.

A new recurrent neural network based language model (RNN LM) with applications to speech recognition is presented. Results indicate that it is possible to obtain around 50% reduction of perplexity by using mixture of several RNN LMs, compared to a state of the art backoff language model. Speech recognition experiments show around 18% reduction of word error rate on the Wall Street Journal task when comparing models trained on the same amount of data, and around 5% on the much harder NIST RT05 task, even when the backoff model is trained on much more data than the RNN LM. We provide ample empirical evidence to suggest that connectionist language models are superior to standard n-gram techniques, except their high computational (training) complexity. Index Terms: language modeling, recurrent neural networks, speech recognition

Abstract In this article, we review special features of Gwyddion—a modular, multiplatform, open-source software for scanning probe microscopy data processing, which is available at http://gwyddion.net/. We describe its architecture with emphasis on modularity and easy integration of the provided algorithms into other software. Special functionalities, such as data processing from non-rectangular areas, grain and particle analysis, and metrology support are discussed as well. It is shown that on the basis of open-source software development, a fully functional software package can be created that covers the needs of a large part of the scanning probe microscopy user community.

There are two widely known issues with properly training Recurrent Neural Networks, the vanishing and the exploding gradient problems detailed in Bengio et al. (1994). In this paper we attempt to improve the understanding of the underlying issues by exploring these problems from an analytical, a geometric and a dynamical systems perspective. Our analysis is used to justify a simple yet effective solution. We propose a gradient norm clipping strategy to deal with exploding gradients and a soft constraint for the vanishing gradients problem. We validate empirically our hypothesis and proposed solutions in the experimental section.

Continuous space language models have recently demonstrated outstanding results across a variety of tasks. In this paper, we examine the vector-space word representations that are implicitly learned by the input-layer weights. We find that these representations are surprisingly good at capturing syntactic and semantic regularities in language, and that each relationship is characterized by a relation-specific vector offset. This allows vector-oriented reasoning based on the offsets between words. For example, the male/female relationship is automatically learned, and with the induced vector representations, “King-Man + Woman ” results in a vector very close to “Queen. ” We demonstrate that the word vectors capture syntactic regularities by means of syntactic analogy questions (provided with this paper), and are able to correctly answer almost 40 % of the questions. We demonstrate that the word vectors capture semantic regularities by using the vector offset method to answer SemEval-2012 Task 2 questions. Remarkably, this method outperforms the best previous systems. 1

Plasma-liquid interactions represent a growing interdisciplinary area of research involving plasma science, fluid dynamics, heat and mass transfer, photolysis, multiphase chemistry and aerosol science. This review provides an assessment of the state-of-the-art of this multidisciplinary area and identifies the key research challenges. The developments in diagnostics, modeling and further extensions of cross section and reaction rate databases that are necessary to address these challenges are discussed. The review focusses on non-equilibrium plasmas.

We present several modifications of the original recurrent neural network language model (RNN LM).While this model has been shown to significantly outperform many competitive language modeling techniques in terms of accuracy, the remaining problem is the computational complexity. In this work, we show approaches that lead to more than 15 times speedup for both training and testing phases. Next, we show importance of using a backpropagation through time algorithm. An empirical comparison with feedforward networks is also provided. In the end, we discuss possibilities how to reduce the amount of parameters in the model. The resulting RNN model can thus be smaller, faster both during training and testing, and more accurate than the basic one.

Abstract. A mathematical model of the prototype of mem-ristor, manufactured in 2008 in Hewlett-Packard Labs, is described in the paper. It is shown that the hitherto pub-lished approaches to the modeling of boundary conditions need not conform with the requirements for the behavior of a practical circuit element. The described SPICE model of the memristor is thus constructed as an open model, ena-bling additional modifications of non-linear boundary conditions. Its functionality is illustrated on computer simulations.

The understanding of the basic physical relationships between nano-scale structural variables and the macroscale properties of polymer nanocomposites remains in its infancy. The primary objective of this article is to ascertain the state of the art regarding the understanding and prediction of the macroscale properties of polymers reinforced with nanometer-sized solid inclusions over a wide temperature range. We emphasize that the addition of nanoparticles with large specific surface area to polymer matrices leads to amplification of a number of rather distinct molecular processes resulting from interactions between chains and solid surfaces. This results in a “non-classical” response of these systems to mechanical and electro-optical excitations when measured on the macroscale. For example, nanoparticles are expected to be particularly effective at modifying the intrinsic nano-scale dynamic heterogeneity of polymeric glass-formation and, correspondingly, recent simulations indicate that both the strength of particle interaction with the polymer matrix and the particle concentration can substantially influence the dynamic fragility of polymer glass-formation, a measure of the strength of the temperature dependence of the viscosity or structural relaxation time. Another basic characteristic of nanoparticles in polymer matrices is the tendency for the particles to associate into extended structures that can dominate the rheological, viscoelastic and mechanical properties of the nanocomposite so that thermodynamic factors that effect nanoparticle dispersion can be crucially important. Opportunities to exploit knowledge gained from understanding biomechanics of hierarchical biological protein materials and potential applications in materials design and nanotechnology are among future research challenges. Research on nanocomposites formed from block copolymers and nanoparticles offers huge promise in molecular electronics and photovoltaics. The surface functionalization of nanoparticles by the grafting of polymer brushes is expected to play important role in the designing of novel organic/inorganic nanocomposite materials. The formation of bulk heterojunctions at the nanometer scale leads to efficient dissociation of the charge pairs generated under sunlight. Based on the presentations and discussion, we make recommendations for future work in this area by the physics, chemistry, and engineering communities.

Single nucleotide variants represent a prevalent form of genetic variation. Mutations in the coding regions are frequently associated with the development of various genetic diseases. Computational tools for the prediction of the effects of mutations on protein function are very important for analysis of single nucleotide variants and their prioritization for experimental characterization. Many computational tools are already widely employed for this purpose. Unfortunately, their comparison and further improvement is hindered by large overlaps between the training datasets and benchmark datasets, which lead to biased and overly optimistic reported performances. In this study, we have constructed three independent datasets by removing all duplicities, inconsistencies and mutations previously used in the training of evaluated tools. The benchmark dataset containing over 43,000 mutations was employed for the unbiased evaluation of eight established prediction tools: MAPP, nsSNPAnalyzer, PANTHER, PhD-SNP, PolyPhen-1, PolyPhen-2, SIFT and SNAP. The six best performing tools were combined into a consensus classifier PredictSNP, resulting into significantly improved prediction performance, and at the same time returned results for all mutations, confirming that consensus prediction represents an accurate and robust alternative to the predictions delivered by individual tools. A user-friendly web interface enables easy access to all eight prediction tools, the consensus classifier PredictSNP and annotations from the Protein Mutant Database and the UniProt database. The web server and the datasets are freely available to the academic community at http://loschmidt.chemi.muni.cz/predictsnp.

Free radicals are chemical particles containing one or more unpaired electrons, which may be part of the molecule. They cause the molecule to become highly reactive. The free radicals are also known to play a dual role in biological systems, as they can be either beneficial or harmful for living systems. It is clear that there are numerous mechanisms participating on the protection of a cell against free radicals. In this review, our attention is paid to metallothioneins (MTs) as small, cysteine-rich and heavy metal-binding proteins, which participate in an array of protective stress responses. The mechanism of the reaction of metallothioneins with oxidants and electrophilic compounds is discussed. Numerous reports indicate that MT protects cells from exposure to oxidants and electrophiles, which react readily with sulfhydryl groups. Moreover, MT plays a key role in regulation of zinc levels and distribution in the intracellular space. The connections between zinc, MT and cancer are highlighted.

Changing network conditions pose severe problems to video streaming in the Internet. HTTP adaptive streaming (HAS) is a technology employed by numerous video services that relieves these issues by adapting the video to the current network conditions. It enables service providers to improve resource utilization and Quality of Experience (QoE) by incorporating information from different layers in order to deliver and adapt a video in its best possible quality. Thereby, it allows taking into account end user device capabilities, available video quality levels, current network conditions, and current server load. For end users, the major benefits of HAS compared to classical HTTP video streaming are reduced interruptions of the video playback and higher bandwidth utilization, which both generally result in a higher QoE. Adaptation is possible by changing the frame rate, resolution, or quantization of the video, which can be done with various adaptation strategies and related client- and server-side actions. The technical development of HAS, existing open standardized solutions, but also proprietary solutions are reviewed in this paper as fundamental to derive the QoE influence factors that emerge as a result of adaptation. The main contribution is a comprehensive survey of QoE related works from human computer interaction and networking domains, which are structured according to the QoE impact of video adaptation. To be more precise, subjective studies that cover QoE aspects of adaptation dimensions and strategies are revisited. As a result, QoE influence factors of HAS and corresponding QoE models are identified, but also open issues and conflicting results are discussed. Furthermore, technical influence factors, which are often ignored in the context of HAS, affect perceptual QoE influence factors and are consequently analyzed. This survey gives the reader an overview of the current state of the art and recent developments. At the same time, it targets networking researchers who develop new solutions for HTTP video streaming or assess video streaming from a user centric point of view. Therefore, this paper is a major step toward truly improving HAS.

Manipulation and navigation of micro and nanoswimmers in different fluid environments can be achieved by chemicals, external fields, or even motile cells. Many researchers have selected magnetic fields as the active external actuation source based on the advantageous features of this actuation strategy such as remote and spatiotemporal control, fuel-free, high degree of reconfigurability, programmability, recyclability, and versatility. This review introduces fundamental concepts and advantages of magnetic micro/nanorobots (termed here as "MagRobots") as well as basic knowledge of magnetic fields and magnetic materials, setups for magnetic manipulation, magnetic field configurations, and symmetry-breaking strategies for effective movement. These concepts are discussed to describe the interactions between micro/nanorobots and magnetic fields. Actuation mechanisms of flagella-inspired MagRobots (i.e., corkscrew-like motion and traveling-wave locomotion/ciliary stroke motion) and surface walkers (i.e., surface-assisted motion), applications of magnetic fields in other propulsion approaches, and magnetic stimulation of micro/nanorobots beyond motion are provided followed by fabrication techniques for (quasi-)spherical, helical, flexible, wire-like, and biohybrid MagRobots. Applications of MagRobots in targeted drug/gene delivery, cell manipulation, minimally invasive surgery, biopsy, biofilm disruption/eradication, imaging-guided delivery/therapy/surgery, pollution removal for environmental remediation, and (bio)sensing are also reviewed. Finally, current challenges and future perspectives for the development of magnetically powered miniaturized motors are discussed.

Carbon nanotubes (CNTs) have been under scientific investigation for more than fifteen years because of their unique properties that predestine them for many potential applications. The field of nanotechnology and nanoscience push their investigation forward to produce CNTs with suitable parameters for future applications. It is evident that new approaches of their synthesis need to be developed and optimized. In this paper we review history, types, structure and especially the different synthesis methods for CNTs preparation including arc discharge, laser ablation and chemical vapour deposition. Moreover, we mention some rarely used ways of arc discharge deposition which involves arc discharge in liquid solutions in contrary to standard used deposition in a gas atmosphere. In addition, the methods for uniform vertically aligned CNTs synthesis using lithographic techniques for catalyst deposition as well as a method utilizing a nanoporous anodized aluminium oxide as a pattern for selective CNTs growth are reported too.

Sequence-discriminative training of deep neural networks (DNNs) is investigated on a 300 hour American English conversational telephone speech task. Different sequence-discriminative criteria ndash;- maximum mutual information (MMI), minimum phone error (MPE), state-level minimum Bayes risk (sMBR), and boosted MMI ndash;- are compared. Two different heuristics are investigated to improve the performance of the DNNs trained using sequence-based criteria ndash;- lattices are re-generated after the first iteration of training; and, for MMI and BMMI, the frames where the numerator and denominator hypotheses are disjoint are removed from the gradient computation. Starting from a competitive DNN baseline trained using cross-entropy, different sequence-discriminative criteria are shown to lower word error rates by 8-9% relative, on average. Little difference is noticed between the different sequence-based criteria that are investigated. The experiments are done using the open-source Kaldi toolkit, which makes it possible for the wider community to reproduce these results.

The increasing number of scientific publications focusing on magnetic materials indicates growing interest in the broader scientific community. Substantial progress was made in the synthesis of magnetic materials of desired size, morphology, chemical composition, and surface chemistry. Physical and chemical stability of magnetic materials is acquired by the coating. Moreover, surface layers of polymers, silica, biomolecules, etc. can be designed to obtain affinity to target molecules. The combination of the ability to respond to the external magnetic field and the rich possibilities of coatings makes magnetic materials universal tool for magnetic separations of small molecules, biomolecules and cells. In the biomedical field, magnetic particles and magnetic composites are utilized as the drug carriers, as contrast agents for magnetic resonance imaging (MRI), and in magnetic hyperthermia. However, the multifunctional magnetic particles enabling the diagnosis and therapy at the same time are emerging. The presented review article summarizes the findings regarding the design and synthesis of magnetic materials focused on biomedical applications. We highlight the utilization of magnetic materials in separation/preconcentration of various molecules and cells, and their use in diagnosis and therapy.

Bevacizumab, an antibody against vascular endothelial growth factor (VEGF), is a promising, yet controversial, drug in human glioblastoma treatment (GBM). Its effects on tumor burden, recurrence, and vascular physiology are unclear. We therefore determined the tumor response to bevacizumab at the phenotypic, physiological, and molecular level in a clinically relevant intracranial GBM xenograft model derived from patient tumor spheroids. Using anatomical and physiological magnetic resonance imaging (MRI), we show that bevacizumab causes a strong decrease in contrast enhancement while having only a marginal effect on tumor growth. Interestingly, dynamic contrast-enhanced MRI revealed a significant reduction of the vascular supply, as evidenced by a decrease in intratumoral blood flow and volume and, at the morphological level, by a strong reduction of large- and medium-sized blood vessels. Electron microscopy revealed fewer mitochondria in the treated tumor cells. Importantly, this was accompanied by a 68% increase in infiltrating tumor cells in the brain parenchyma. At the molecular level we observed an increase in lactate and alanine metabolites, together with an induction of hypoxia-inducible factor 1α and an activation of the phosphatidyl-inositol-3-kinase pathway. These data strongly suggest that vascular remodeling induced by anti-VEGF treatment leads to a more hypoxic tumor microenvironment. This favors a metabolic change in the tumor cells toward glycolysis, which leads to enhanced tumor cell invasion into the normal brain. The present work underlines the need to combine anti-angiogenic treatment in GBMs with drugs targeting specific signaling or metabolic pathways linked to the glycolytic phenotype.

Recurrent neural network language models (RNNLMs) have recently demonstrated state-of-the-art performance across a variety of tasks. In this paper, we improve their performance by providing a contextual real-valued input vector in association with each word. This vector is used to convey contextual information about the sentence being modeled. By performing Latent Dirichlet Allocation using a block of preceding text, we achieve a topic-conditioned RNNLM. This approach has the key advantage of avoiding the data fragmentation associated with building multiple topic models on different data subsets. We report perplexity results on the Penn Treebank data, where we achieve a new state-of-the-art. We further apply the model to the Wall Street Journal speech recognition task, where we observe improvements in word-error-rate.

Agriculture in the 21st century is facing multiple challenges, such as those related to soil fertility, climatic fluctuations, environmental degradation, urbanization, and the increase in food demand for the increasing world population. In the meanwhile, the scientific community is facing key challenges in increasing crop production from the existing land base. In this regard, traditional farming has witnessed enhanced per acre crop yields due to irregular and injudicious use of agrochemicals, including pesticides and synthetic fertilizers, but at a substantial environmental cost. Another major concern in modern agriculture is that crop pests are developing pesticide resistance. Therefore, the future of sustainable crop production requires the use of alternative strategies that can enhance crop yields in an environmentally sound manner. The application of rhizobacteria, specifically, plant growth-promoting rhizobacteria (PGPR), as an alternative to chemical pesticides has gained much attention from the scientific community. These rhizobacteria harbor a number of mechanisms through which they promote plant growth, control plant pests, and induce resistance to various abiotic stresses. This review presents a comprehensive overview of the mechanisms of rhizobacteria involved in plant growth promotion, biocontrol of pests, and bioremediation of contaminated soils. It also focuses on the effects of PGPR inoculation on plant growth survival under environmental stress. Furthermore, the pros and cons of rhizobacterial application along with future directions for the sustainable use of rhizobacteria in agriculture are discussed in depth.