Huawei Technologies (United Kingdom)

Language model pre-training, such as BERT, has significantly improved the performances of many natural language processing tasks. However, pre-trained language models are usually computationally expensive, so it is difficult to efficiently execute them on resourcerestricted devices. To accelerate inference and reduce model size while maintaining accuracy, we first propose a novel Transformer distillation method that is specially designed for knowledge distillation (KD) of the Transformer-based models. By leveraging this new KD method, the plenty of knowledge encoded in a large "teacher" BERT can be effectively transferred to a small "student" Tiny-BERT. Then, we introduce a new two-stage learning framework for TinyBERT, which performs Transformer distillation at both the pretraining and task-specific learning stages. This framework ensures that TinyBERT can capture the general-domain as well as the task-specific knowledge in BERT.

Due to the increased popularity of augmented and virtual reality experiences, the interest in capturing the real world in multiple dimensions and in presenting it to users in an immersible fashion has never been higher. Distributing such representations enables users to freely navigate in multisensory 3D media experiences. Unfortunately, such representations require a large amount of data, not feasible for transmission on today's networks. Efficient compression technologies well adopted in the content chain are in high demand and are key components to democratize augmented and virtual reality applications. Moving Picture Experts Group, as one of the main standardization groups dealing with multimedia, identified the trend and started recently the process of building an open standard for compactly representing 3D point clouds, which are the 3D equivalent of the very well-known 2D pixels. This paper introduces the main developments and technical aspects of this ongoing standardization effort.

3GPP has completed a study on coordinated multipoint transmission and reception techniques to facilitate cooperative communications across multiple transmission and reception points (e.g., cells) for the LTE-Advanced system. In CoMP operation, multiple points coordinate with each other in such a way that the transmission signals from/to other points do not incur serious interference or even can be exploited as a meaningful signal. The goal of the study is to evaluate the potential performance benefits of CoMP techniques and the implementation aspects including the complexity of the standards support for CoMP. This article discusses some of the deployment scenarios in which CoMP techniques will likely be most beneficial and provides an overview of CoMP schemes that might be supported in LTE-Advanced given the modern silicon/DSP technologies and backhaul designs available today. In addition, practical implementation and operational challenges are discussed. We also assess the performance benefits of CoMP in these deployment scenarios with traffic varying from low to high load.

In this paper, we propose virtual data center (VDC) as the unit of resource allocation for multiple tenants in the cloud. VDCs are more desirable than physical data centers because the resources allocated to VDCs can be rapidly adjusted as tenants' needs change. To enable the VDC abstraction, we design a data center network virtualization architecture called SecondNet. SecondNet achieves scalability by distributing all the virtual-to-physical mapping, routing, and bandwidth reservation state in server hypervisors. Its port-switching based source routing (PSSR) further makes SecondNet applicable to arbitrary network topologies using commodity servers and switches. SecondNet introduces a centralized VDC allocation algorithm for bandwidth guaranteed virtual to physical mapping. Simulations demonstrate that our VDC allocation achieves high network utilization and low time complexity. Our implementation and experiments show that we can build SecondNet on top of various network topologies, and SecondNet provides bandwidth guarantee and elasticity, as designed.

5G cellular networks are assumed to be the key enabler and infrastructure provider in the ICT industry, by offering a variety of services with diverse requirements. The standardization of 5G cellular networks is being expedited, which also implies more of the candidate technologies will be adopted. Therefore, it is worthwhile to provide insight into the candidate techniques as a whole and examine the design philosophy behind them. In this article, we try to highlight one of the most fundamental features among the revolutionary techniques in the 5G era, i.e., there emerges initial intelligence in nearly every important aspect of cellular networks, including radio resource management, mobility management, service provisioning management, and so on. However, faced with ever-increasingly complicated configuration issues and blossoming new service requirements, it is still insufficient for 5G cellular networks if it lacks complete AI functionalities. Hence, we further introduce fundamental concepts in AI and discuss the relationship between AI and the candidate techniques in 5G cellular networks. Specifically, we highlight the opportunities and challenges to exploit AI to achieve intelligent 5G networks, and demonstrate the effectiveness of AI to manage and orchestrate cellular network resources. We envision that AI-empowered 5G cellular networks will make the acclaimed ICT enabler a reality.

To mitigate the severe inter-tier interference and enhance the limited cooperative gains resulting from the constrained and non-ideal transmissions between adjacent base stations in HetNets, H-CRANs are proposed as cost-efficient potential solutions through incorporating cloud computing into HetNets. In this article, state-of-the-art research achievements and challenges of H-CRANs are surveyed. In particular, we discuss issues of system architectures, spectral and energy efficiency performance, and promising key techniques. A great emphasis is given toward promising key techniques in HCRANs to improve both spectral and energy efficiencies, including cloud-computing-based coordinated multipoint transmission and reception, large-scale cooperative multiple antenna, cloud-computing-based cooperative radio resource management, and cloud-computingbased self-organizing networks in cloud converging scenarios. The major challenges and open issues in terms of theoretical performance with stochastic geometry, fronthaul-constrained resource allocation, and standard development that may block the promotion of H-CRANs are discussed as well.

In 6G, it is envisaged that the function of sensing and communication will coexist and be fully integrated in one system, sharing the same resources in time, frequency and space domains, as well as key elements including waveform, signal processing, hardware, etc. This paper discusses novel applications, key performance requirements, challenges and future research directions for Integrated Sensing and Communication (ISAC) design in 6G. First, four categories of ISAC use cases are described as new services in 6G together with their corresponding performance requirements on 6G design. In addition, it is demonstrated how the information obtained through sensing can significantly improve the performance of communication. Thereafter, the challenges in the design and evaluation of a practical ISAC system are discussed, including sensing and communication performance tradeoffs, hardware imperfections, as well as the investigation of an appropriate channel model. Finally, future research directions are presented to set the path for designing an efficient ISAC network.

Structural pruning enables model acceleration by removing structurally-grouped parameters from neural networks. However, the parameter-grouping patterns vary widely across different models, making architecture-specific pruners, which rely on manually-designed grouping schemes, non-generalizable to new architectures. In this work, we study a highly-challenging yet barely-explored task, any structural pruning, to tackle general structural pruning of arbitrary architecture like CNNs, RNNs, GNNs and Transformers. The most prominent obstacle towards this goal lies in the structural coupling, which not only forces different layers to be pruned simultaneously, but also expects all removed parameters to be consistently unimportant, thereby avoiding structural issues and significant performance degradation after pruning. To address this problem, we propose a general and fully automatic method, Dependency Graph (DepGraph), to explicitly model the dependency between layers and comprehensively group coupled parameters for pruning. In this work, we extensively evaluate our method on several architectures and tasks, including ResNe(X)t, DenseNet, MobileNet and Vision transformer for images, GAT for graph, DGCNN for 3D point cloud, alongside LSTM for language, and demonstrate that, even with a simple norm-based criterion, the proposed method consistently yields gratifying performances.

Caching is a hot research topic and poised to develop into a key technology for the upcoming 5G wireless networks. However, the successful implementation of caching techniques crucially depends on joint research developments in different scientific domains such as networking, information theory, machine learning, and wireless communications. Moreover, there are business barriers related to the complex interactions between the involved stakeholders: users, cellular operators, and Internet content providers. In this article we discuss several technical misconceptions with the aim of uncovering enabling research directions for caching in wireless systems. Ultimately, we make a speculative stakeholder analysis for wireless caching in 5G.

Network slicing for 5G provides Network-as-a-Service (NaaS) for different use cases, allowing network operators to build multiple virtual networks on a shared infrastructure. With network slicing, service providers can deploy their applications and services flexibly and quickly to accommodate diverse services’ specific requirements. As an emerging technology with a number of advantages, network slicing has raised many issues for the industry and academia alike. Here, the authors discuss this technology’s background and propose a framework. They also discuss remaining challenges and future research directions.

Outlines of the multiphysics behaviors upon penetration and the comparisons of experiment and simulation.

Next-generation [fifth-generation (5G)] mobile systems are broadening their spectrum to higher-frequency bands (above 6 GHz) to support a high data rate up to multigigabits per second. In this article, we have two main contributions to higher-frequency communications. First, we summarize the candidate frequency bands that are promising for 5G research, including licensed and unlicensed frequency bands. Second, a new network architecture for higher-frequency communications is proposed, which is featured with load-centric backhauling (LCB), multiple-frequency transmission, and intelligent control techniques.

Advanced interference mitigation techniques relying on multipoint coordination have attracted significant attention from the wireless industry and academia in the past few years. In 3GPP LTE-Advanced, a work item on Coordinated Multiple Point transmission and reception (CoMP) was initiated in September 2011, and it is one of the core features of Release 11. The objective of this work item is to provide the necessary specification support to efficiently realize the benefits of cooperative transmission in the downlink and cooperative reception in the uplink. This article discusses the specification support for CoMP and the motivations behind the specific design choices. The deployment scenarios that were considered for the application of CoMP in Release 11 are also presented.

With the blossoming of network functions virtualization and software-defined networks, networks are becoming more and more agile with features like resilience, programmability, and open interfaces, which help operators to launch a network or service with more flexibility and shorter time to market. Recently, the concept of network slicing has been proposed to facilitate the building of a dedicated and customized logical network with virtualized resources. In this article, we introduce the concept of hierarchical NSaaS, helping operators to offer customized end-to-end cellular networks as a service. Moreover, the service orchestration and service level agreement mapping for quality assurance are introduced to illustrate the architecture of service management across different levels of service models. Finally, we illustrate the process of network slicing as a service within operators by typical examples. With network slicing as a service, we believe that the supporting system will transform itself to a production system by merging the operation and business domains, and enabling operators to build network slices for vertical industries more agilely.

In this article, a novel self-decoupled multiple-input multiple-output (MIMO) antenna pair with a shared radiator is proposed for fifth-generation (5G) smartphones. In our approach, a radiator is directly excited by two feeding ports, and interestingly, the two ports are naturally isolated across a wide bandwidth without using any extra decoupling structures. To offer a deep physical insight of the self-decoupling mechanism, a mode-cancellation method based on the synthesis of common and differential modes is developed for the first time. The proposed self-decoupled antenna pair shows a good isolation of better than 11.5 dB across the 5G N77 band (3.3-4.2 GHz) with a radiation pattern diversity property. Based on the self-decoupled antenna pair, an 8 × 8 MIMO antenna system, constituted by four sets of antenna pairs, is simulated, fabricated, and measured to validate the concept. The experimental results demonstrate that the proposed 8 × 8 MIMO system can offer an isolation of better than 10.5 dB between all ports and a high total efficiency of 63.1%-85.1% across 3.3-4.2 GHz. With the advantages of self-decoupling, shared radiator, simple structure, wide bandwidth, and high efficiency, the proposed design scheme exhibits promising potential for the future highly integrated MIMO antennas for 5G smartphones.

Real software, the kind working programmers produce by the kLOC to solve real-world problems, tends to be "natural", like speech or natural language; it tends to be highly repetitive and predictable. Researchers have captured this naturalness of software through statistical models and used them to good effect in suggestion engines, porting tools, coding standards checkers, and idiom miners. This suggests that code that appears improbable, or surprising, to a good statistical language model is "unnatural" in some sense, and thus possibly suspicious. In this paper, we investigate this hypothesis. We consider a large corpus of bug fix commits (ca. 7,139), from 10 different Java projects, and focus on its language statistics, evaluating the naturalness of buggy code and the corresponding fixes. We find that code with bugs tends to be more entropic (i.e. unnatural), becoming less so as bugs are fixed. Ordering files for inspection by their average entropy yields cost-effectiveness scores comparable to popular defect prediction methods. At a finer granularity, focusing on highly entropic lines is similar in cost-effectiveness to some well-known static bug finders (PMD, FindBugs) and ordering warnings from these bug finders using an entropy measure improves the cost-effectiveness of inspecting code implicated in warnings. This suggests that entropy may be a valid, simple way to complement the effectiveness of PMD or FindBugs, and that search-based bug-fixing methods may benefit from using entropy both for fault-localization and searching for fixes.

For 6G in 2030 and beyond, key performance metrics long for terabit-per-second, one-tenth of millisecond latency with zero jitter, millimeter-preci-sion sensing and positioning, and seamless connectivity, among others. To meet these demands, the terahertz (0.1–10 THz) band comes into the horizon, with ultra-broad bandwidth and sub-millimeter wavelength, which, however, suffers from distance limitation and power consumption concerns. In this article, four interdisciplinary directions to address the above challenges and unleash the full potential of THz communications are investigated, namely, integrated sensing and communication, ultra-mas-sive MIMO and dynamic hybrid beamforming, intelligent surfaces, and machine/deep learning. Furthermore, open problems as well as integration of these solutions are elaborated. Qualitative and quantitative studies demonstrate the benefits and performance of the proposed solutions.

Network embedding is to learn low-dimensional vector representations for nodes of a given social network, facilitating many tasks in social network analysis such as link prediction. The vast majority of existing embedding algorithms are designed for unsigned social networks or social networks with only positive links. However, networks in social media could have both positive and negative links, and little work exists for signed social networks. From recent findings of signed network analysis, it is evident that negative links have distinct properties and added value besides positive links, which brings about both challenges and opportunities for signed network embedding. In this paper, we propose a deep learning framework SiNE for signed network embedding. The framework optimizes an objective function guided by social theories that provide a fundamental understanding of signed social networks. Experimental results on two real-world datasets of social media demonstrate the effectiveness of the proposed framework SiNE.

Current wireless and cellular networks are destined to undergo a significant change in the transition to the next generation of network technology. The so called wireless powered communication network (WPCN) has been recently emerging as a promising candidate for achieving the target performance of future networks. According to this paradigm, nodes in a WPCN can be equipped with hardware capable of harvesting energy from wireless signals, that is, their battery can be ubiquitously replenished without physical connections. Recent technological advances in the field of wireless power harvesting and transfer are providing strong evidence of the feasibility of this vision, especially for low-power devices. The future deployment of WPCN is more and more concretely foreseen. The aim of this article is therefore to provide a comprehensive review of the basics and backgrounds of WPCN, current major developments, and open research issues. In particular, we first give an overview of WPCN and its structure. We then present three major advanced approaches whose adoption could increase the performance of future WPCN: backscatter communications with energy harvesting; duty-cycle based energy management; and transceiver design for self-sustainable communications. We discuss implementation perspectives and tools for WPCN. Finally, we outline open research problems for WPCN.

Passive optical networks are the most important class of fiber access systems in the world today. This article first reviews the reasons why the PON as a general architecture is so important. We then outline in some depth the technologies used to implement this architecture, including the G-PON and E-PON systems being deployed today, and the advanced PON systems that provide the evolution path to ever higher bandwidths