Amazon (United States)

Abstract Array programming provides a powerful, compact and expressive syntax for accessing, manipulating and operating on data in vectors, matrices and higher-dimensional arrays. NumPy is the primary array programming library for the Python language. It has an essential role in research analysis pipelines in fields as diverse as physics, chemistry, astronomy, geoscience, biology, psychology, materials science, engineering, finance and economics. For example, in astronomy, NumPy was an important part of the software stack used in the discovery of gravitational waves 1 and in the first imaging of a black hole 2 . Here we review how a few fundamental array concepts lead to a simple and powerful programming paradigm for organizing, exploring and analysing scientific data. NumPy is the foundation upon which the scientific Python ecosystem is constructed. It is so pervasive that several projects, targeting audiences with specialized needs, have developed their own NumPy-like interfaces and array objects. Owing to its central position in the ecosystem, NumPy increasingly acts as an interoperability layer between such array computation libraries and, together with its application programming interface (API), provides a flexible framework to support the next decade of scientific and industrial analysis.

Reliability at massive scale is one of the biggest challenges we face at Amazon.com, one of the largest e-commerce operations in the world; even the slightest outage has significant financial consequences and impacts customer trust. The Amazon.com platform, which provides services for many web sites worldwide, is implemented on top of an infrastructure of tens of thousands of servers and network components located in many datacenters around the world. At this scale, small and large components fail continuously and the way persistent state is managed in the face of these failures drives the reliability and scalability of the software systems.

To be agile and cost effective, data centers should allow dynamic resource allocation across large server pools. In particular, the data center network should enable any server to be assigned to any service. To meet these goals, we present VL2, a practical network architecture that scales to support huge data centers with uniform high capacity between servers, performance isolation between services, and Ethernet layer-2 semantics. VL2 uses (1) flat addressing to allow service instances to be placed anywhere in the network, (2) Valiant Load Balancing to spread traffic uniformly across network paths, and (3) end-system based address resolution to scale to large server pools, without introducing complexity to the network control plane. VL2's design is driven by detailed measurements of traffic and fault data from a large operational cloud service provider. VL2's implementation leverages proven network technologies, already available at low cost in high-speed hardware implementations, to build a scalable and reliable network architecture. As a result, VL2 networks can be deployed today, and we have built a working prototype. We evaluate the merits of the VL2 design using measurement, analysis, and experiments. Our VL2 prototype shuffles 2.7 TB of data among 75 servers in 395 seconds - sustaining a rate that is 94% of the maximum possible.

, water, and energy exchange between the biosphere and the atmosphere, and other meteorological and biological measurements, from 212 sites around the globe (over 1500 site-years, up to and including year 2014). These sites, independently managed and operated, voluntarily contributed their data to create global datasets. Data were quality controlled and processed using uniform methods, to improve consistency and intercomparability across sites. The dataset is already being used in a number of applications, including ecophysiology studies, remote sensing studies, and development of ecosystem and Earth system models. FLUXNET2015 includes derived-data products, such as gap-filled time series, ecosystem respiration and photosynthetic uptake estimates, estimation of uncertainties, and metadata about the measurements, presented for the first time in this paper. In addition, 206 of these sites are for the first time distributed under a Creative Commons (CC-BY 4.0) license. This paper details this enhanced dataset and the processing methods, now made available as open-source codes, making the dataset more accessible, transparent, and reproducible.

BACKGROUND: Comparative analysis of RNA sequences is the basis for the detailed and accurate predictions of RNA structure and the determination of phylogenetic relationships for organisms that span the entire phylogenetic tree. Underlying these accomplishments are very large, well-organized, and processed collections of RNA sequences. This data, starting with the sequences organized into a database management system and aligned to reveal their higher-order structure, and patterns of conservation and variation for organisms that span the phylogenetic tree, has been collected and analyzed. This type of information can be fundamental for and have an influence on the study of phylogenetic relationships, RNA structure, and the melding of these two fields. RESULTS: We have prepared a large web site that disseminates our comparative sequence and structure models and data. The four major types of comparative information and systems available for the three ribosomal RNAs (5S, 16S, and 23S rRNA), transfer RNA (tRNA), and two of the catalytic intron RNAs (group I and group II) are: (1) Current Comparative Structure Models; (2) Nucleotide Frequency and Conservation Information; (3) Sequence and Structure Data; and (4) Data Access Systems. CONCLUSIONS: This online RNA sequence and structure information, the result of extensive analysis, interpretation, data collection, and computer program and web development, is accessible at our Comparative RNA Web (CRW) Site http://www.rna.icmb.utexas.edu. In the future, more data and information will be added to these existing categories, new categories will be developed, and additional RNAs will be studied and presented at the CRW Site.

Much of the recent progress made in image classification research can be credited to training procedure refinements, such as changes in data augmentations and optimization methods. In the literature, however, most refinements are either briefly mentioned as implementation details or only visible in source code. In this paper, we will examine a collection of such refinements and empirically evaluate their impact on the final model accuracy through ablation study. We will show that, by combining these refinements together, we are able to improve various CNN models significantly. For example, we raise ResNet-50's top-1 validation accuracy from 75.3% to 79.29% on ImageNet. We will also demonstrate that improvement on image classification accuracy leads to better transfer learning performance in other application domains such as object detection and semantic segmentation.

Previous studies have shown that the method of hydrogen mass repartitioning (HMR) is a potentially useful tool for accelerating molecular dynamics (MD) simulations. By repartitioning the mass of heavy atoms into the bonded hydrogen atoms, it is possible to slow the highest-frequency motions of the macromolecule under study, thus allowing the time step of the simulation to be increased by up to a factor of 2. In this communication, we investigate further how this mass repartitioning allows the simulation time step to be increased in a stable fashion without significantly increasing discretization error. To this end, we ran a set of simulations with different time steps and mass distributions on a three-residue peptide to get a comprehensive view of the effect of mass repartitioning and time step increase on a system whose accessible phase space is fully explored in a relatively short amount of time. We next studied a 129-residue protein, hen egg white lysozyme (HEWL), to verify that the observed behavior extends to a larger, more-realistic, system. Results for the protein include structural comparisons from MD trajectories, as well as comparisons of pKa calculations via constant-pH MD. We also calculated a potential of mean force (PMF) of a dihedral rotation for the MTS [(1-oxyl-2,2,5,5-tetramethyl-pyrroline-3-methyl)methanethiosulfonate] spin label via umbrella sampling with a set of regular MD trajectories, as well as a set of mass-repartitioned trajectories with a time step of 4 fs. Since no significant difference in kinetics or thermodynamics is observed by the use of fast HMR trajectories, further evidence is provided that long-time-step HMR MD simulations are a viable tool for accelerating MD simulations for molecules of biochemical interest.

Being able to spot defective parts is a critical component in large-scale industrial manufacturing. A particular challenge that we address in this work is the cold-start problem: fit a model using nominal (non-defective) example images only. While handcrafted solutions per class are possible, the goal is to build systems that work well simultaneously on many different tasks automatically. The best peforming approaches combine embeddings from ImageNet models with an outlier detection model. In this paper, we extend on this line of work and propose PatchCore, which uses a maximally representative memory bank of nominal patch-features. PatchCore offers competitive inference times while achieving state-of-the-art performance for both detection and localization. On the challenging, widely used MVTec AD benchmark PatchCore achieves an image-level anomaly detection AUROC score of up to 99.6%, more than halving the error compared to the next best competitor. We further report competitive results on two additional datasets and also find competitive results in the few samples regime. Code: github.com/amazon-research/patchcore-inspection.

Large, pre-trained language models (PLMs) such as BERT and GPT have drastically changed the Natural Language Processing (NLP) field. For numerous NLP tasks, approaches leveraging PLMs have achieved state-of-the-art performance. The key idea is to learn a generic, latent representation of language from a generic task once, then share it across disparate NLP tasks. Language modeling serves as the generic task, one with abundant self-supervised text available for extensive training. This article presents the key fundamental concepts of PLM architectures and a comprehensive view of the shift to PLM-driven NLP techniques. It surveys work applying the pre-training then fine-tuning, prompting, and text generation approaches. In addition, it discusses PLM limitations and suggested directions for future research.

Reliability at massive scale is one of the biggest challenges we face at Amazon.com, one of the largest e-commerce operations in the world; even the slightest outage has significant financial consequences and impacts customer trust. The Amazon.com platform, which provides services for many web sites worldwide, is implemented on top of an infrastructure of tens of thousands of servers and network components located in many datacenters around the world. At this scale, small and large components fail continuously and the way persistent state is managed in the face of these failures drives the reliability and scalability of the software systems. This paper presents the design and implementation of Dynamo, a highly available key-value storage system that some of Amazon's core services use to provide an "always-on" experience. To achieve this level of availability, Dynamo sacrifices consistency under certain failure scenarios. It makes extensive use of object versioning and application-assisted conflict resolution in a manner that provides a novel interface for developers to use.

We describe the input/output automaton model, a model for concurrent and distributed discrete event systems. We define the model, illustrate the model with several examples concerning vending machines and a leader election algorithm, and survey the ways in which the model has been used.

We use a mechanistically based ecosystem simulation model to describe and analyze the spatial and temporal patterns of terrestrial net primary productivity (NPP) in South America. The Terrestrial Ecosystem Model (TEM) is designed to predict major carbon and nitrogen fluxes and pool sizes in terrestrial ecosystems at continental to global scales. Information from intensively studies field sites is used in combination with continental—scale information on climate, soils, and vegetation to estimate NPP in each of 5888 non—wetland, 0.5° latitude °0.5° longitude grid cells in South America, at monthly time steps. Preliminary analyses are presented for the scenario of natural vegetation throughout the continent, as a prelude to evaluating human impacts on terrestrial NPP. The potential annual NPP of South America is estimated to be 12.5 Pg/yr of carbon (26.3 Pg/yr of organic matter) in a non—wetland area of 17.0 ° 10 6 km 2 . More than 50% of this production occurs in the tropical and subtropical evergreen forest region. Six independent model runs, each based on an independently derived set of model parameters, generated mean annual NPP estimates for the tropical evergreen forest region ranging from 900 to 1510 g°m — 2 °yr — 1 of carbon, with an overall mean of 1170 g°m — 2 °yr — 1 . Coefficients of variation in estimated annual NPP averaged 20% for any specific location in the evergreen forests, which is probably within the confidence limits of extant NPP measurements. Predicted rates of mean annual NPP in other types of vegetation ranged from 95 g°m — 2 °yr — 1 in arid shrublands to 930 g°m @ ?yr — 1 in savannas, and were within the ranges measured in empirical studies. The spatial distribution of predicted NPP was directly compared with estimates made using the Miami mode of Lieth (1975). Overall, TEM predictions were °10% lower than those of the Miami model, but the two models agreed closely on the spatial patterns of NPP in south America. Unlike previous models, however, TEM estimates NPP monthly, allowing for the evaluation of seasonal phenomena. This is an important step toward integration of ecosystem models with remotely sensed information, global climate models, and atmospheric transport models, all of which are evaluated at comparable spatial and temporal scales. Seasonal patterns of NPP in South America are correlated with moisture availability in most vegetation types, but are strongly influenced by seasonal differences in cloudiness in the tropical evergreen forests. On an annual basis, moisture availability was the factor that was correlated most strongly with annual NPP in South America, but differences were again observed among vegetation types. These results allow for the investigation and analysis of climatic controls over NPP at continental scales, within and among vegetation types, and within years. Further model validation is needed. Nevertheless, the ability to investigate NPP—environment interactions with a high spatial and temporal resolution at continental scales should prove useful if not essential for rigorous analysis of the potential effects of global climate changes on terrestrial ecosystems.

We present a general and flexible video-level framework for learning action models in videos. This method, called temporal segment network (TSN), aims to model long-range temporal structure with a new segment-based sampling and aggregation scheme. This unique design enables the TSN framework to efficiently learn action models by using the whole video. The learned models could be easily deployed for action recognition in both trimmed and untrimmed videos with simple average pooling and multi-scale temporal window integration, respectively. We also study a series of good practices for the implementation of the TSN framework given limited training samples. Our approach obtains the state-the-of-art performance on five challenging action recognition benchmarks: HMDB51 (71.0 percent), UCF101 (94.9 percent), THUMOS14 (80.1 percent), ActivityNet v1.2 (89.6 percent), and Kinetics400 (75.7 percent). In addition, using the proposed RGB difference as a simple motion representation, our method can still achieve competitive accuracy on UCF101 (91.0 percent) while running at 340 FPS. Furthermore, based on the proposed TSN framework, we won the video classification track at the ActivityNet challenge 2016 among 24 teams.

Deep embeddings answer one simple question: How similar are two images? Learning these embeddings is the bedrock of verification, zero-shot learning, and visual search. The most prominent approaches optimize a deep convolutional network with a suitable loss function, such as contrastive loss or triplet loss. While a rich line of work focuses solely on the loss functions, we show in this paper that selecting training examples plays an equally important role. We propose distance weighted sampling, which selects more informative and stable examples than traditional approaches. In addition, we show that a simple margin based loss is sufficient to outperform all other loss functions. We evaluate our approach on the Stanford Online Products, CAR196, and the CUB200-2011 datasets for image retrieval and clustering, and on the LFW dataset for face verification. Our method achieves state-of-the-art performance on all of them.

Most studies of the human microbiome have focused on westernized people with life-style practices that decrease microbial survival and transmission, or on traditional societies that are currently in transition to westernization. We characterize the fecal, oral, and skin bacterial microbiome and resistome of members of an isolated Yanomami Amerindian village with no documented previous contact with Western people. These Yanomami harbor a microbiome with the highest diversity of bacteria and genetic functions ever reported in a human group. Despite their isolation, presumably for >11,000 years since their ancestors arrived in South America, and no known exposure to antibiotics, they harbor bacteria that carry functional antibiotic resistance (AR) genes, including those that confer resistance to synthetic antibiotics and are syntenic with mobilization elements. These results suggest that westernization significantly affects human microbiome diversity and that functional AR genes appear to be a feature of the human microbiome even in the absence of exposure to commercial antibiotics. AR genes are likely poised for mobilization and enrichment upon exposure to pharmacological levels of antibiotics. Our findings emphasize the need for extensive characterization of the function of the microbiome and resistome in remote nonwesternized populations before globalization of modern practices affects potentially beneficial bacteria harbored in the human body.

Transformers have recently shown superior performances on various vision tasks. The large, sometimes even global, receptive field endows Transformer models with higher representation power over their CNN counterparts. Nevertheless, simply enlarging receptive field also gives rise to several concerns. On the one hand, using dense attention e.g., in ViT, leads to excessive memory and computational cost, and features can be influenced by irrelevant parts which are beyond the region of interests. On the other hand, the sparse attention adopted in PVT or Swin Transformer is data agnostic and may limit the ability to model long range relations. To mitigate these issues, we propose a novel deformable selfattention module, where the positions of key and value pairs in selfattention are selected in a data-dependent way. This flexible scheme enables the self-attention module to focus on relevant re-gions and capture more informative features. On this basis, we present Deformable Attention Transformer, a general backbone model with deformable attention for both image classification and dense prediction tasks. Extensive experi-ments show that our models achieve consistently improved results on comprehensive benchmarks. Code is available at https://github.com/LeapLabTHU/DAT.

Deep neural networks have achieved remarkable performance in both image classification and object detection problems, at the cost of a large number of parameters and computational complexity. In this work, we show how to reduce the redundancy in these parameters using a sparse decomposition. Maximum sparsity is obtained by exploiting both inter-channel and intra-channel redundancy, with a fine-tuning step that minimize the recognition loss caused by maximizing sparsity. This procedure zeros out more than 90% of parameters, with a drop of accuracy that is less than 1% on the ILSVRC2012 dataset. We also propose an efficient sparse matrix multiplication algorithm on CPU for Sparse Convolutional Neural Networks (SCNN) models. Our CPU implementation demonstrates much higher efficiency than the off-the-shelf sparse matrix libraries, with a significant speedup realized over the original dense network. In addition, we apply the SCNN model to the object detection problem, in conjunction with a cascade model and sparse fully connected layers, to achieve significant speedups.

Datacenter workloads demand high computational capabilities, flexibility, power efficiency, and low cost. It is challenging to improve all of these factors simultaneously. To advance datacenter capabilities beyond what commodity server designs can provide, we have designed and built a composable, reconfigurablefabric to accelerate portions of large-scale software services. Each instantiation of the fabric consists of a 6x8 2-D torus of high-end Stratix V FPGAs embedded into a half-rack of 48 machines. One FPGA is placed into each server, accessible through PCIe, and wired directly to other FPGAs with pairs of 10 Gb SAS cables In this paper, we describe a medium-scale deployment of this fabric on a bed of 1,632 servers, and measure its efficacy in accelerating the Bing web search engine. We describe the requirements and architecture of the system, detail the critical engineering challenges and solutions needed to make the system robust in the presence of failures, and measure the performance, power, and resilience of the system when ranking candidate documents. Under high load, the largescale reconfigurable fabric improves the ranking throughput of each server by a factor of 95% for a fixed latency distribution--- or, while maintaining equivalent throughput, reduces the tail latency by 29%

Soybeans represent a recent and powerful threat to tropical biodiversity in Brazil. Developing effective strategies to contain and minimize the environmental impact of soybean cultivation requires understanding of both the forces that drive the soybean advance and the many ways that soybeans and their associated infrastructure catalyse destructive processes. The present paper presents an up-to-date review of the advance of soybeans in Brazil, its environmental and social costs and implications for development policy. Soybeans are driven by global market forces, making them different from many of the land-use changes that have dominated the scene in Brazil so far, particularly in Amazonia. Soybeans are much more damaging than other crops because they justify massive transportation infrastructure projects that unleash a chain of events leading to destruction of natural habitats over wide areas in addition to what is directly cultivated for soybeans. The capacity of global markets to absorb additional production represents the most likely limit to the spread of soybeans, although Brazil may someday come to see the need for discouraging rather than subsidizing this crop because many of its effects are unfavourable to national interests, including severe concentration of land tenure and income, expulsion of population to Amazonian frontier, and gold-mining, as well as urban areas, and the opportunity cost of substantial drains on government resources. The multiple impacts of soybean expansion on biodiversity and other development considerations have several implications for policy: (1) protected areas need to be created in advance of soybean frontiers, (2) elimination of the many subsidies that speed soybean expansion beyond what would occur otherwise from market forces is to be encouraged, (3) studies to assess the costs of social and environmental impacts associated with soybean expansion are urgently required, and (4) the environmental-impact regulatory system requires strengthening, including mechanisms for commitments not to implant specific infrastructure projects that are judged to have excessive impacts.

To be agile and cost effective, data centers should allow dynamic resource allocation across large server pools. In particular, the data center network should enable any server to be assigned to any service. To meet these goals, we present VL2, a practical network architecture that scales to support huge data centers with uniform high capacity between servers, performance isolation between services, and Ethernet layer-2 semantics. VL2 uses (1) flat addressing to allow service instances to be placed anywhere in the network, (2) Valiant Load Balancing to spread traffic uniformly across network paths, and (3) end-system based address resolution to scale to large server pools, without introducing complexity to the network control plane. VL2's design is driven by detailed measurements of traffic and fault data from a large operational cloud service provider. VL2's implementation leverages proven network technologies, already available at low cost in high-speed hardware implementations, to build a scalable and reliable network architecture. As a result, VL2 networks can be deployed today, and we have built a working prototype. We evaluate the merits of the VL2 design using measurement, analysis, and experiments. Our VL2 prototype shuffles 2.7 TB of data among 75 servers in 395 seconds - sustaining a rate that is 94% of the maximum possible.