Centre for Automation and Robotics

This document provides a review of the techniques and therapies used in gait rehabilitation after stroke. It also examines the possible benefits of including assistive robotic devices and brain-computer interfaces in this field, according to a top-down approach, in which rehabilitation is driven by neural plasticity.The methods reviewed comprise classical gait rehabilitation techniques (neurophysiological and motor learning approaches), functional electrical stimulation (FES), robotic devices, and brain-computer interfaces (BCI).From the analysis of these approaches, we can draw the following conclusions. Regarding classical rehabilitation techniques, there is insufficient evidence to state that a particular approach is more effective in promoting gait recovery than other. Combination of different rehabilitation strategies seems to be more effective than over-ground gait training alone. Robotic devices need further research to show their suitability for walking training and their effects on over-ground gait. The use of FES combined with different walking retraining strategies has shown to result in improvements in hemiplegic gait. Reports on non-invasive BCIs for stroke recovery are limited to the rehabilitation of upper limbs; however, some works suggest that there might be a common mechanism which influences upper and lower limb recovery simultaneously, independently of the limb chosen for the rehabilitation therapy. Functional near infrared spectroscopy (fNIRS) enables researchers to detect signals from specific regions of the cortex during performance of motor activities for the development of future BCIs. Future research would make possible to analyze the impact of rehabilitation on brain plasticity, in order to adapt treatment resources to meet the needs of each patient and to optimize the recovery process.

A new method for grating and side lobes suppression in ultrasound images is presented. It is based on an analysis of the phase diversity at the aperture data. Two coherence factors, namely the phase coherence factor (PCF) and the sign coherence factor (SCF), are proposed to weight the coherent sum output. Different from other approaches, phase rather than amplitude information is used to perform the correction action. Besides achieving the main goal, the method obtains improvements in lateral resolution and SNR. Implementation of the SCF technique is quite straightforward, operating in realtime, and can be added to any virtually existing beamformer to improve the resolution, contrast, SNR, and dynamic range of the images. A programmable parameter allows adjusting the sensitivity of the method to out-of-phase signals, from zero to a strict coherence criterion. The theoretical basis for the 2 methods are given and their performances evaluated by simulation. Then, experiments are conducted to provide results that are in good agreement with those expected from theory and simulation.

We present a new method to accurately locate persons indoors by fusing inertial navigation system (INS) techniques with active RFID technology. A foot-mounted inertial measuring units (IMUs)-based position estimation method, is aided by the received signal strengths (RSSs) obtained from several active RFID tags placed at known locations in a building. In contrast to other authors that integrate IMUs and RSS with a loose Kalman filter (KF)-based coupling (by using the residuals of inertial- and RSS-calculated positions), we present a tight KF-based INS/RFID integration, using the residuals between the INS-predicted reader-to-tag ranges and the ranges derived from a generic RSS path-loss model. Our approach also includes other drift reduction methods such as zero velocity updates (ZUPTs) at foot stance detections, zero angular-rate updates (ZARUs) when the user is motionless, and heading corrections using magnetometers. A complementary extended Kalman filter (EKF), throughout its 15-element error state vector, compensates the position, velocity and attitude errors of the INS solution, as well as IMU biases. This methodology is valid for any kind of motion (forward, lateral or backward walk, at different speeds), and does not require an offline calibration for the user gait. The integrated INS <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX"> $+$</tex></formula> RFID methodology eliminates the typical drift of IMU-alone solutions (approximately 1% of the total traveled distance), resulting in typical positioning errors along the walking path (no matter its length) of approximately 1.5 m.

Most ultrawideband (UWB) location systems already proposed for position estimation have only been individually evaluated for particular scenarios. For a fair performance comparison among different solutions, a common evaluation scenario would be desirable. In this paper, we compare three commercially available UWB systems (Ubisense, BeSpoon, and DecaWave) under the same experimental conditions, in order to do a critical performance analysis. We include the characterization of the quality of the estimated tag-to-sensor distances in an indoor industrial environment. This testing space includes areas under line-of-sight (LOS) and diverse non-LOS conditions caused by the reflection, propagation, and the diffraction of the UWB radio signals across different obstacles. The study also includes the analysis of the estimated azimuth and elevation angles for the Ubisense system, which is the only one that incorporates this feature using an array antenna at each sensor. Finally, we analyze the 3-D positioning estimation performance of the three UWB systems using a Bayesian filter implemented with a particle filter and a measurement model that takes into account bad range measurements and outliers. A final conclusion is drawn about which system performs better under these industrial conditions.

Abstract Populations of cortical neurons flexibly perform different functions; for the primary motor cortex (M1) this means a rich repertoire of motor behaviors. We investigate the flexibility of M1 movement control by analyzing neural population activity during a variety of skilled wrist and reach-to-grasp tasks. We compare across tasks the neural modes that capture dominant neural covariance patterns during each task. While each task requires different patterns of muscle and single unit activity, we find unexpected similarities at the neural population level: the structure and activity of the neural modes is largely preserved across tasks. Furthermore, we find two sets of neural modes with task-independent activity that capture, respectively, generic temporal features of the set of tasks and a task-independent mapping onto muscle activity. This system of flexibly combined, well-preserved neural modes may underlie the ability of M1 to learn and generate a wide-ranging behavioral repertoire.

Deep learning is recently showing outstanding results for solving a wide variety of robotic tasks in the areas of perception, planning, localization, and control. Its excellent capabilities for learning representations from the complex data acquired in real environments make it extremely suitable for many kinds of autonomous robotic applications. In parallel, Unmanned Aerial Vehicles (UAVs) are currently being extensively applied for several types of civilian tasks in applications going from security, surveillance, and disaster rescue to parcel delivery or warehouse management. In this paper, a thorough review has been performed on recent reported uses and applications of deep learning for UAVs, including the most relevant developments as well as their performances and limitations. In addition, a detailed explanation of the main deep learning techniques is provided. We conclude with a description of the main challenges for the application of deep learning for UAV-based solutions.

Abstract In this paper, a system that allows applying precision agriculture techniques is described. The application is based on the deployment of a team of unmanned aerial vehicles that are able to take georeferenced pictures in order to create a full map by applying mosaicking procedures for postprocessing. The main contribution of this work is practical experimentation with an integrated tool. Contributions in different fields are also reported. Among them is a new one‐phase automatic task partitioning manager, which is based on negotiation among the aerial vehicles, considering their state and capabilities. Once the individual tasks are assigned, an optimal path planning algorithm is in charge of determining the best path for each vehicle to follow. Also, a robust flight control based on the use of a control law that improves the maneuverability of the quadrotors has been designed. A set of field tests was performed in order to analyze all the capabilities of the system, from task negotiations to final performance. These experiments also allowed testing control robustness under different weather conditions. © 2011 Wiley Periodicals, Inc.

Traffic merging in urban environments is one of the main causes of traffic congestion. From the driver's point of view, the difficulty arises along the on-ramp where the merging vehicle's driver has to discern whether he should accelerate or decelerate to enter the main road. In parallel, the drivers of the vehicles already on the major road may have to modify their speeds to permit the entrance of the merging vehicle, thus affecting the traffic flow. This paper presents an approach to merging from a minor to a major road in congested traffic situations. An automated merging system that was developed with two principal goals, i.e., to permit the merging vehicle to sufficiently fluidly enter the major road to avoid congestion on the minor road and to modify the speed of the vehicles already on the main road to minimize the effect on that already congested main road, is described. A fuzzy controller is developed to act on the vehicles' longitudinal control - throttle and brake pedals - following the references set by a decision algorithm. Data from other vehicles are acquired using wireless vehicle-to-infrastructure (V2I) communication. A system installed in the infrastructure that is capable of assessing road traffic conditions in real time is responsible for transmitting the data of the vehicles in the surrounding area. Three production vehicles were used in the experimental phase to validate the proposed system at the facilities of the Centro de Automática y Robótica with encouraging results.

Falls are the leading cause of injury and death in elderly individuals. Unfortunately, fall detectors are typically based on wearable devices, and the elderly often forget to wear them. In addition, fall detectors based on artificial vision are not yet available on the market. In this paper, we present a new low-cost fall detector for smart homes based on artificial vision algorithms. Our detector combines several algorithms (background subtraction, Kalman filtering and optical flow) as input to a machine learning algorithm with high detection accuracy. Tests conducted on over 50 different fall videos have shown a detection ratio of greater than 96%.

The number of tasks that nowadays are accomplished by using unmanned aerial vehicles is rising across many civil applications, including agriculture. Thus, this work aims at providing the reader with an overview of the agronomical use of unmanned aerial vehicles. The work starts with a historical analysis of the use of aircrafts in agriculture, as pioneers of their use in modern precision agriculture techniques, currently applied by a high number of users. This survey has been carried out by providing a classification of the vehicles according to their typology and main sensorial and performance features. An extensive review of the most common applications and the advantages of using unmanned aerial vehicles is the core of the work. Finally, a brief summary of the key points of the legislation applicable to civil drones that could affect to agricultural applications is analyzed.

Agriculture 4.0 is transforming farming livelihoods thanks to the development and adoption of technologies such as artificial intelligence, the Internet of Things and robotics, traditionally used in other productive sectors. Soft robotics and soft grippers in particular are promising approaches to lead to new solutions in this field due to the need to meet hygiene and manipulation requirements in unstructured environments and in operation with delicate products. This review aims to provide an in-depth look at soft end-effectors for agricultural applications, with a special emphasis on robotic harvesting. To that end, the current state of automatic picking tasks for several crops is analysed, identifying which of them lack automatic solutions, and which methods are commonly used based on the botanical characteristics of the fruits. The latest advances in the design and implementation of soft grippers are also presented and discussed, studying the properties of their materials, their manufacturing processes, the gripping technologies and the proposed control methods. Finally, the challenges that have to be overcome to boost its definitive implementation in the real world are highlighted. Therefore, this review intends to serve as a guide for those researchers working in the field of soft robotics for Agriculture 4.0, and more specifically, in the design of soft grippers for fruit harvesting robots.

Feeding the growing global population requires an annual increase in food production. This requirement suggests an increase in the use of pesticides, which represents an unsustainable chemical load for the environment. To reduce pesticide input and preserve the environment while maintaining the necessary level of food production, the efficiency of relevant processes must be drastically improved. Within this context, this research strived to design, develop, test and assess a new generation of automatic and robotic systems for effective weed and pest control aimed at diminishing the use of agricultural chemical inputs, increasing crop quality and improving the health and safety of production operators. To achieve this overall objective, a fleet of heterogeneous ground and aerial robots was developed and equipped with innovative sensors, enhanced end-effectors and improved decision control algorithms to cover a large variety of agricultural situations. This article describes the scientific and technical objectives, challenges and outcomes achieved in three common crops.

Abstract The developments of information and automation technologies have opened a new era for weed management to fit physical and chemical control treatments to the spatial and temporal heterogeneity of weed distributions in agricultural fields. This review describes the technologies of site‐specific weed management (SSWM) systems, evaluates their ecological and economic benefits and gives a perspective for the implementation in practical farming. Sensor technologies including 3D cameras, multispectral imaging and Artificial Intelligence (AI) for weed classification and computer‐based decision algorithms are described in combination with precise spraying and hoeing operations. Those treatments are targeted for patches of weeds or individual weed plants. Cameras can also guide inter‐row hoes precisely in the centre between two crop rows at much higher driving speed. Camera‐guided hoeing increased selectivity and weed control efficacy compared with manual steered hoeing. Robots combine those guiding systems with in‐row hoeing or spot spraying systems that can selectively control individual weeds within crop rows. Results with patch spraying show at least 50% saving of herbicides in various crops without causing additional costs for weed control in the following years. A challenge with these technologies is the interoperability of sensing and controllers. Most of the current SSWM technologies use their own IT protocols that do not allow connecting different sensors and implements. Plug &amp; play standards for linking detection, decision making and weeding would improve the adoption of new SSWM technologies and reduce operational costs. An important impact of SSWM is the potential contribution to the EU‐ Green Deal targets to reduce pesticide use and increase biodiversity. However, further on‐farm research is needed for integrating those technologies into agricultural practice.

This document provides a survey of the mathematical methods currently used for position estimation in indoor local positioning systems (LPS), particularly those based on radiofrequency signals. The techniques are grouped into four categories: geometry-based methods, minimization of the cost function, fingerprinting, and Bayesian techniques. Comments on the applicability, requirements, and immunity to nonline-of-sight (NLOS) propagation of the signals of each method are provided.

Vehicles equipped with intelligent systems designed to prevent accidents, such as collision warning systems (CWSs) or lane-keeping assistance (LKA), are now on the market. The next step in reducing road accidents is to coordinate such vehicles in advance not only to avoid collisions but to improve traffic flow as well. To this end, vehicle-to-infrastructure (V2I) communications are essential to properly manage traffic situations. This paper describes the AUTOPIA approach toward an intelligent traffic management system based on V2I communications. A fuzzy-based control algorithm that takes into account each vehicle's safe and comfortable distance and speed adjustment for collision avoidance and better traffic flow has been developed. The proposed solution was validated by an IEEE-802.11p-based communications study. The entire system showed good performance in testing in real-world scenarios, first by computer simulation and then with real vehicles.

Nowadays, one important challenge in cyber-physical production systems is updating dynamic production schedules through an automated decision-making performed while the production is running. The condition of the manufacturing equipment may in fact lead to schedule unfeasibility or inefficiency, thus requiring responsiveness to preserve productivity and reduce the operational costs. In order to address current limitations of traditional scheduling methods, this work proposes a new framework that exploits the aggregation of several digital twins, representing different physical assets and their autonomous decision-making, together with a global digital twin, in order to perform production scheduling optimization when it is needed. The decision-making process is supported on a fuzzy inference system using the state or conditions of different assets and the production rate of the whole system. The condition of the assets is predicted by the condition-based monitoring modules in the local digital twins of the workstations, whereas the production rate is evaluated and assured by the global digital twin of the shop floor. This paper presents a framework for decentralized and integrated decision-making for re-scheduling of a cyber-physical production system, and the validation and proof-of-concept of the proposed method in an Industry 4.0 pilot line of assembly process. The experimental results demonstrate that the proposed framework is capable to detect changes in the manufacturing process and to make appropriate decisions for re-scheduling the process.

Collision avoidance is one of the most difficult and challenging automatic driving operations in the domain of intelligent vehicles. In emergency situations, human drivers are more likely to brake than to steer, although the optimal maneuver would, more frequently, be steering alone. This statement suggests the use of automatic steering as a promising solution to avoid accidents in the future. The objective of this paper is to provide a collision avoidance system (CAS) for autonomous vehicles, focusing on pedestrian collision avoidance. The detection component involves a stereo-vision-based pedestrian detection system that provides suitable measurements of the time to collision. The collision avoidance maneuver is performed using fuzzy controllers for the actuators that mimic human behavior and reactions, along with a high-precision Global Positioning System (GPS), which provides the information needed for the autonomous navigation. The proposed system is evaluated in two steps. First, drivers' behavior and sensor accuracy are studied in experiments carried out by manual driving. This study will be used to define the parameters of the second step, in which automatic pedestrian collision avoidance is carried out at speeds of up to 30 km/h. The performed field tests provided encouraging results and proved the viability of the proposed approach.

This paper describes the design, construction and validation of a mobile sensory platform for greenhouse monitoring. The complete system consists of a sensory system on board a small quadrotor (i.e., a four rotor mini-UAV). The goals of this system include taking measures of temperature, humidity, luminosity and CO2 concentration and plotting maps of these variables. These features could potentially allow for climate control, crop monitoring or failure detection (e.g., a break in a plastic cover). The sensors have been selected by considering the climate and plant growth models and the requirements for their integration onboard the quadrotor. The sensors layout and placement have been determined through a study of quadrotor aerodynamics and the influence of the airflows from its rotors. All components of the system have been developed, integrated and tested through a set of field experiments in a real greenhouse. The primary contributions of this paper are the validation of the quadrotor as a platform for measuring environmental variables and the determination of the optimal location of sensors on a quadrotor.

Personal drones are becoming part of every day life. To fully integrate them into society, it is crucial to design safe and intuitive ways to interact with these aerial systems. The recent advances on User-Centered Design (UCD) applied to Natural User Interfaces (NUIs) intend to make use of human innate features, such as speech, gestures and vision to interact with technology in the way humans would with one another. In this paper, a Graphical User Interface (GUI) and several NUI methods are studied and implemented, along with computer vision techniques, in a single software framework for aerial robotics called Aerostack which allows for intuitive and natural human-quadrotor interaction in indoor GPS-denied environments. These strategies include speech, body position, hand gesture and visual marker interactions used to directly command tasks to the drone. The NUIs presented are based on devices like the Leap Motion Controller, microphones and small size monocular on-board cameras which are unnoticeable to the user. Thanks to this UCD perspective, the users can choose the most intuitive and effective type of interaction for their application. Additionally, the strategies proposed allow for multi-modal interaction between multiple users and the drone by being able to integrate several of these interfaces in one single application as is shown in various real flight experiments performed with non-expert users.

Summary Weed monitoring is the first step in any site‐specific weed management programme. A relatively large variety of platforms, cameras, sensors and image analysis procedures are available to detect and map weed presence/abundance at various times and spatial scales. Remote sensing from satellites or aircraft can provide accurate weed maps when the images are obtained at late weed phenological stages. Cameras located on unmanned aerial vehicles (UAVs) have been shown to be adequate for early‐season weed detection in a variety of wide‐row crops, providing images with relatively high spatial resolutions. Alternatively, weed detection/mapping systems from ground‐based platforms can achieve even higher resolutions using a variety of non‐imaging and imaging technologies. These ground systems are suited, in some cases, for real‐time site‐specific weed management. Despite this rich arsenal of technologies, their commercial adoption is, apparently, low. In this study, we describe the state of the art of remotely sensed and ground‐based weed monitoring in arable crops and the current level of adoption of these technologies, exploring major constraints for adoption and trying to identify research gaps and bottlenecks.