State Key Laboratory of Robotics

Federated learning (FL) has attracted growing attentions via data-private collaborative training on decentralized clients. However, most existing methods unrealistically assume object classes of the overall framework are fixed over time. It makes the global model suffer from significant catastrophic forgetting on old classes in real-world scenarios, where local clients often collect new classes continuously and have very limited storage memory to store old classes. Moreover, new clients with unseen new classes may participate in the FL training, further aggravating the catastrophic forgetting of global model. To address these challenges, we develop a novel Global-Local Forgetting Compensation (GLFC) model, to learn a global class-incremental model for alleviating the catastrophic forgetting from both local and global perspectives. Specifically, to address local forgetting caused by class imbalance at the local clients, we design a class-aware gradient compensation loss and a class-semantic relation distillation loss to balance the forgetting of old classes and distill consistent inter-class relations across tasks. To tackle the global forgetting brought by the non-i.i.d class imbalance across clients, we propose a proxy server that selects the best old global model to assist the local relation distillation. Moreover, a prototype gradient-based communication mechanism is developed to protect the privacy. Our model outperforms state-of-the-art methods by 4.4%~15.1% in terms of average accuracy on representative benchmark datasets. The code is available at https://github.com/conditionWang/FCIL.

Gelatin/sodium alginate/carboxymethyl chitosan hydrogel mixed with bone mesenchymal stem cells for 3D bioprinting.

Spectrum sensing is the prerequisite of opportunistic spectrum access in cognitive sensor networks (CSNs) as its reliability determines the success of transmission. However, spectrum sensing is an energy-consuming operation that needs to be minimized for CSNs due to resource limitations. This paper considers the case where the cognitive sensors cooperatively sense a licensed channel by using the CoMAC-based cooperative spectrum sensing (CSS) scheme to determine the presence of primary users. Energy efficiency (EE), defined as the ratio of the average throughput to the average energy consumption, is a very important performance metric for CSNs. We formulate an EE-maximization problem for CSS in CSNs subject to the constraint on the detection performance. In order to address the non-convex and non-separable nature of the formulated problem, we first find the optimal expression for the detection threshold and then propose an iterative solution algorithm to obtain an efficient pair of sensing time and the length of the modulated symbol sequence. Simulations demonstrate the convergence and optimality of the proposed algorithm. It is also observed in simulations that the combination of the CoMAC-based CSS scheme and the proposed algorithm yields much higher EE than conventional CSS schemes while guaranteeing the same detection performance.

The main problem of atomic force microscopy (AFM)-based nanomanipulation is the lack of real-time visual feedback. Although this problem has been partially solved by virtual reality technology, the faulty display caused by random drift and modeling errors in the virtual reality interface are still limiting the efficiency of the AFM-based nanomanipulation. Random drift aroused from an uncontrolled manipulation environment generates a position error between the manipulation coordinate and the true environment. Modeling errors due to the uncertainties of the nanoenvironment often result in displaying a wrong position of the object. Since there is no feedback to check the validity of the display, the faulty display cannot be detected in real time and leads to a failed manipulation. In this paper, a real-time fault detection and correction (RFDC) method is proposed to solve these problems by using the AFM tip as an end effector as well as a force sensor during manipulation. Based on the interaction force measured from the AFM tip, the validity of the visual feedback is monitored in real time by the developed Kalman filter. Once the faulty display is detected, it can be corrected online through a quick local scan without interrupting manipulation. In this way, the visual feedback keeps consistent with the true environment changes during manipulation, which makes it possible for several operations to finish without an image scan in between. The theoretical study and the implementation of the RFDC method are elaborated. Experiments of manipulating nanomaterials including nanoparticles and nanorods have been carried out to demonstrate its effectiveness and efficiency.

Grasping a moving target has been investigated extensively for fixed-base manipulator. However, such a task becomes much more challenging when the manipulator is free flying in the air with an UAV. Towards moving target grasping, this paper presents an aerial manipulator system composed of a hex-rotor and a 7-DoF (Degree of Freedom) manipulator. An independent control structure is used in the aerial manipulator control system, i.e., the hex-rotor and the manipulator are controlled separately. In the hex-rotor's controller, the system CoM (Center of Mass) offset motion is used to compensate disturbance of the robotic arm. In the manipulator's controller, the relative kinematics between the target and the aerial vehicle is taken into consideration to grasp the target. At last aerial grasping experiments are conducted to validate the feasibility of the proposed control scheme and the reliability of our aerial manipulator system.

Surface defect detection is of great significance to ensure the quality of steel plate. The surface defects of steel plate are characterized by multiple types, complex and irregular shapes, large scale range, and high similarity with normal regions, resulting in low accuracy of widely used vision based defect detection methods. To overcome these issues, this paper proposes a method of detecting steel plate surface defects based on deformation convolution and background suppression. First, an improved Faster RCNN method with deformable convolution and Region-of-Interest align is proposed to enhance the detection performance for large-scale defects with complex and irregular shapes; Second, a background suppression method is proposed to enhance the discrimination ability between the normal region and the defect region. Experimental results show that, compared with the state-of-the-art methods, the proposed method can significantly improve the defect detection performance of steel plate.

Radio frequency (RF) chain circuits play a major role in digital receiver architectures, allowing passband communication signals to be processed in baseband. When operating at high frequencies, these circuits tend to be costly. This increased cost imposes a major limitation on future multiple-input-multiple-output (MIMO) communication technologies. A common approach to mitigate the increased cost is to utilize hybrid architectures, in which the received signal is combined in analog into a lower dimension, thus reducing the number of RF chains. In this article we study the design and hardware implementation of hybrid architectures via minimizing channel estimation error. We first derive the optimal solution for complex-gain combiners and propose an alternating optimization algorithm for phase-shifter combiners. We then present a hardware prototype implementing analog combining for RF chain reduction. The prototype consists of a specially designed configurable combining board as well as a dedicated experimental setup. Our hardware prototype allows us evaluating the effect of analog combining in MIMO systems using actual communication signals. The experimental study, which focuses on channel estimation accuracy in MIMO channels, demonstrates that using the proposed prototype, the achievable channel estimation performance is within a small gap in a statistical sense from that obtained using a costly receiver in which each antenna is connected to a dedicated RF chain.

Atomic force microscopy (AFM) has found a wide range of bio-applications in the past few decades due to its ability to measure biological samples in natural environments at a high spatial resolution. AFM has become a key platform in biomedical, bioengineering and drug research fields, enabling mechanical and morphological characterization of live biological systems. Hence, we provide a comprehensive review on recent advances in the use of AFM for characterizing the biomechanical properties of multi-scale biological samples, ranging from molecule, cell to tissue levels. First, we present the fundamental principles of AFM and two AFM-based models for the characterization of biomechanical properties of biological samples, covering key AFM devices and AFM bioimaging as well as theoretical models for characterizing the elasticity and viscosity of biomaterials. Then, we elaborate on a series of new experimental findings through analysis of biomechanics. Finally, we discuss the future directions and challenges. It is envisioned that the AFM technique will enable many remarkable discoveries, and will have far-reaching impacts on bio-related studies and applications in the future.

Federated learning-based semantic segmentation (FSS) has drawn widespread attention via decentralized training on local clients. However, most FSS models assume categories are fixed in advance, thus heavily undergoing forgetting on old categories in practical applications where local clients receive new categories incrementally while have no memory storage to access old classes. Moreover, new clients collecting novel classes may join in the global training of FSS, which further exacerbates catastrophic forgetting. To surmount the above challenges, we propose a Forgetting-Balanced Learning (FBL) model to address heterogeneous forgetting on old classes from both intra-client and interclient aspects. Specifically, under the guidance of pseudo labels generated via adaptive class-balanced pseudo labeling, we develop a forgetting-balanced semantic compensation loss and a forgetting-balanced relation consistency loss to rectify intra-client heterogeneous forgetting of old categories with background shift. It performs balanced gradient propagation and relation consistency distillation within local clients. Moreover, to tackle heterogeneous forgetting from inter-client aspect, we propose a task transition monitor. It can identify new classes under privacy protection and store the latest old global model for relation distillation. Qualitative experiments reveal large improvement of our model against comparison methods. The code is available at https://github.com/JiahuaDong/FISS.

Due to the lack of adequate tools for observation, native molecular behaviors at the nanoscale have been poorly understood. The advent of atomic force microscopy (AFM) provides an exciting instrument for investigating physiological processes on individual living cells with molecular resolution, which attracts the attention of worldwide researchers. In the past few decades, AFM has been widely utilized to investigate molecular activities on diverse biological interfaces, and the performances and functions of AFM have also been continuously improved, greatly improving our understanding of the behaviors of single molecules in action and demonstrating the important role of AFM in addressing biological issues with unprecedented spatiotemporal resolution. In this article, we review the related techniques and recent progress about applying AFM to characterize biomolecular systems in situ from single molecules to living cells. The challenges and future directions are also discussed.

The advent of atomic force microscopy (AFM) provides an exciting tool to detect molecular and cellular behaviors under aqueous conditions. AFM is able to not only visualize the surface topography of the specimens, but also can quantify the mechanical properties of the specimens by force spectroscopy assay. Nevertheless, integrating AFM topographic imaging with force spectroscopy assay has long been limited due to the low spatiotemporal resolution. In recent years, the appearance of a new AFM imaging mode called peak force tapping (PFT) has shattered this limit. PFT allows AFM to simultaneously acquire the topography and mechanical properties of biological samples with unprecedented spatiotemporal resolution. The practical applications of PFT in the field of life sciences in the past decade have demonstrated the excellent capabilities of PFT in characterizing the fine structures and mechanics of living biological systems in their native states, offering novel possibilities to reveal the underlying mechanisms guiding physiological/pathological activities. In this paper, the recent progress in cell and molecular biology that has been made with the utilization of PFT is summarized, and future perspectives for further progression and biomedical applications of PFT are provided.

A long history has passed since electromyography (EMG) signals have been explored in human-centered robots for intuitive interaction. However, it still has a gap between scientific research and real-life applications. Previous studies mainly focused on EMG decoding algorithms, leaving a dynamic relationship between the human, robot, and uncertain environment in real-life scenarios seldomly concerned. To fill this gap, this paper presents a comprehensive review of EMG-based techniques in human-robot-environment interaction (HREI) systems. The general processing framework is summarized, and three interaction paradigms, including direct control, sensory feedback, and partial autonomous control, are introduced. EMG-based intention decoding is treated as a module of the proposed paradigms. Five key issues involving precision, stability, user attention, compliance, and environmental awareness in this field are discussed. Several important directions, including EMG decomposition, robust algorithms, HREI dataset, proprioception feedback, reinforcement learning, and embodied intelligence, are proposed to pave the way for future research. To the best of what we know, this is the first time that a review of EMG-based methods in the HREI system is summarized. It provides a novel and broader perspective to improve the practicability of current myoelectric interaction systems, in which factors in human-robot interaction, robot-environment interaction, and state perception by human sensations are considered, which has never been done by previous studies.

OBJECTIVES: No studies have compared monoaxial and polyaxial pedicle screws with regard to the von Mises stress of the instrumentation, intradiscal pressures of the adjacent segment and adjacent segment degeneration. METHODS: Short-segment monoaxial/polyaxial pedicle screw fixation techniques were compared using finite element methods, and the redistributed T11-L1 segment range of motion, largest maximal von Mises stress of the instrumentation, and intradiscal pressures of the adjacent segment under displacement loading were evaluated. Radiographic results of 230 patients with traumatic thoracolumbar fractures treated with these fixations were reviewed, and the sagittal Cobb's angle, vertebral body angle, anterior vertebral body height of the fractured vertebrae and adjacent segment degeneration were calculated and evaluated. RESULTS: The largest maximal values of the von Mises stress were 376.8 MPa for the pedicle screws in the short-segment monoaxial pedicle screw fixation model and 439.9 MPa for the rods in the intermediate monoaxial pedicle screw fixation model. The maximal intradiscal pressures of the upper adjacent segments were all greater than those of the lower adjacent segments. The maximal intradiscal pressures of the monoaxial pedicle screw fixation model were larger than those in the corresponding segments of the normal model. The radiographic results at the final follow-up evaluation showed that the mean loss of correction of the sagittal Cobb's angle, vertebral body angle and anterior vertebral body height were smallest in the intermediate monoaxial pedicle screw fixation group. Adjacent segment degeneration was less likely to be observed in the intermediate polyaxial pedicle screw fixation group but more likely to be observed in the intermediate monoaxial pedicle screw fixation group. CONCLUSION: Smaller von Mises stress in the pedicle screws and lower intradiscal pressure in the adjacent segment were observed in the polyaxial screw model than in the monoaxial pedicle screw fixation spine models. Fracture-level fixation could significantly correct kyphosis and reduce correction loss, and adjacent segment degeneration was less likely to be observed in the intermediate polyaxial pedicle screw fixation group.

. Furthermore, live C. reinhardtii cells trapped by ODEP can form a micrometer-sized motor array. The rotating frequency of the cells ranges from 50 to 120 rpm, which can be reversibly adjusted with a fast response speed by varying the optical intensity. Functional flagella have been demonstrated to play a decisive role in the rotation. The programmable cell array with a rotating motion can be used as a bio-micropump to drive the liquid flow in microfludic chips and may shed new light on bio-actuation.

Single-source domain generalization (SDG) for object detection is a challenging yet essential task as the distribution bias of the unseen domain degrades the algorithm per-formance significantly. However, existing methods attempt to extract domain-invariant features, neglecting that the bi-ased data leads the network to learn biased features that are non-causal and poorly generalizable. To this end, we pro-pose an Unbiased Faster R-CNN (UFR) for generalizable feature learning. Specifically, we formulate SDG in object detection from a causal perspective and construct a Struc-tural Causal Model (SCM) to analyze the data bias andfeature bias in the task, which are caused by scene confounders and object attribute confounders. Based on the SCM, we de-sign a Global-Local Transformation module for data aug-mentation, which effectively simulates domain diversity and mitigates the data bias. Additionally, we introduce a Causal Attention Learning module that incorporates a designed at-tention invariance loss to learn image-level features that are robust to scene confounders. Moreover, we develop a Causal Prototype Learning module with an explicit instance constraint and an implicit prototype constraint, which fur-ther alleviates the negative impact of object attribute con-founders. Experimental results on five scenes demonstrate the prominent generalization ability of our method, with an improvement of 3.9% mAP on the Night-Clear scene.

Cells are frequently studied because they are basic structural, functional, and biological units of living organisms. Extracting features from cellular behaviors can facilitate decision making in medical diagnoses and represents an important aspect in the development of biomedical engineering. Previous studies have just focused on either the individual cell or cell clusters separately, which leads to a great lack of information. Microwell technologies could address the challenges of in vitro cellular studies, from individual cell studies to 3D functional assays, by providing more information from smaller sample volumes and enabling the incorporation of low-cost high-throughput assays in the drug discovery process. To this end, the present study describes an easy-to-use method for fabricating a versatile microwell chip that utilizes a digital micro-mirror device printing system, and the chip can be employed in multidimensional cellular analysis, ranging from the single cell to the 3D spheroid level. The microwell manufacturing process, using a digital mask in place of a conventional physical mask, is based on shadowed light and is full of flexibility. Three different dimensions (single cell (1D), cell monolayer (2D) and cell spheroid (3D)) of one cell type can be formed using a microwell array and the analyses of biological characteristics are achieved separately. Single cells and cell clusters can be controlled via customized geometries of microfabricated selectively adhesive poly(ethylene glycol) diacrylate (PEGDA) wells. The effects of shape on cellular growth and hybrid tissue layers were investigated by peeling off the microwells. Furthermore, 3D multicellular spheroids were successfully established in a controllable and high-throughput format. Preclinical drug screening was investigated and distinct differences were observed in the tolerance response to drug testing between the 2D and 3D conditions. The study results further demonstrate that the high-density microwell chip is an easy-to-use multidimensional cellular analysis and rapid drug screening technique, and it could be easily adapted for a wide range of biological research applications.

Inspired by the human 3D visual perception system, we present an obstacle detection and classification method based on the use of Time-of-Flight (ToF) cameras for robotic navigation in unstructured environments. The ToF camera provides 3D sensing by capturing an image along with per-pixel 3D space information. Based on this valuable feature and human knowledge of navigation, the proposed method first removes irrelevant regions which do not affect robot's movement from the scene. In the second step, regions of interest are detected and clustered as possible obstacles using both 3D information and intensity image obtained by the ToF camera. Consequently, a multiple relevance vector machine (RVM) classifier is designed to classify obstacles into four possible classes based on the terrain traversability and geometrical features of the obstacles. Finally, experimental results in various unstructured environments are presented to verify the robustness and performance of the proposed approach. We have found that, compared with the existing obstacle recognition methods, the new approach is more accurate and efficient.

Wireless sensor networks are gradually employed in many applications that require reliable and real-time data transmission. As hop count is an important factor affecting end-to-end delay and reliability, we investigate the hop constrained relay node placement (HCRNP) problem in this paper. First, to achieve connectivity requirement, we study the connected HCRNP problem. Then, to design survivable network topologies against node failures, we study the 2-connected HCRNP problem. Correspondingly, two polynomial-time algorithms: cover-based 1-connected node placement (C1NP) and cover-based 2-connected node placement (C2NP) are proposed, respectively, to address the above two problems. Through rigorous analysis, we show that 1) C1NP has an approximation ratio better than existing algorithms for the connected HCRNP problem (i.e., O(1) for special settings and O(ln n) for arbitrary settings, where n is the number of SNs) and 2) C2NP is the first algorithm that can provide an explicit performance guarantee for the 2-connected HCRNP problem, i.e., whenever C2NP finds a feasible solution, the ratio of this solution to the optimal solution is guaranteed to be O(ln n). Finally, we verify the effectiveness of the proposed algorithms through extensive simulations.

In the microanalysis of laser-induced breakdown spectroscopy, the influence of surface roughness on spectral stability and quantitative analysis capability was studied for the first time when the laser ablation crater diameter was approximately 10 μm.

Recent progress in introducing rotation invariance (RI) to 3D deep learning methods is mainly made by designing RI features to replace 3D coordinates as input. The key to this strategy lies in how to restore the global information that is lost by the input RI features. Most state-of-the-arts achieve this by incurring additional blocks or complex global representations, which is time-consuming and ineffective. In this paper, we real that the global information loss stems from an unexplored pose information loss problem, i.e., common convolution layers cannot capture the relative poses between RI features, thus hindering the global information to be hierarchically aggregated in the deep networks. To address this problem, we develop a Poseaware Rotation Invariant Convolution (i.e., PaRI-Conv), which dynamically adapts its kernels based on the relative poses. Specifically, in each PaRI-Conv layer, a lightweight Augmented Point Pair Feature (APPF) is designed to fully encode the RI relative pose information. Then, we propose to synthesize a factorized dynamic kernel, which reduces the computational cost and memory burden by decomposing it into a shared basis matrix and a pose-aware diagonal matrix that can be learned from the APPF. Extensive experiments on shape classification and part segmentation tasks show that our PaRI-Conv surpasses the state-of-the-art RI methods while being more compact and efficient.