Instituto de Microelectrónica de Sevilla

Hardware implementations of spiking neurons can be extremely useful for a large variety of applications, ranging from high-speed modeling of large-scale neural systems to real-time behaving systems, to bidirectional brain-machine interfaces. The specific circuit solutions used to implement silicon neurons depend on the application requirements. In this paper we describe the most common building blocks and techniques used to implement these circuits, and present an overview of a wide range of neuromorphic silicon neurons, which implement different computational models, ranging from biophysically realistic and conductance-based Hodgkin-Huxley models to bi-dimensional generalized adaptive integrate and fire models. We compare the different design methodologies used for each silicon neuron design described, and demonstrate their features with experimental results, measured from a wide range of fabricated VLSI chips.

Modern computation based on the von Neumann architecture is today a mature cutting-edge science. In the Von Neumann architecture, processing and memory units are implemented as separate blocks interchanging data intensively and continuously. This data transfer is responsible for a large part of the power consumption. The next generation computer technology is expected to solve problems at the exascale with 1018 calculations each second. Even though these future computers will be incredibly powerful, if they are based on von Neumann type architectures, they will consume between 20 and 30 megawatts of power and will not have intrinsic physically built-in capabilities to learn or deal with complex data as our brain does. These needs can be addressed by neuromorphic computing systems which are inspired by the biological concepts of the human brain. This new generation of computers has the potential to be used for the storage and processing of large amounts of digital information with much lower power consumption than conventional processors. Among their potential future applications, an important niche is moving the control from data centers to edge devices. The aim of this Roadmap is to present a snapshot of the present state of neuromorphic technology and provide an opinion on the challenges and opportunities that the future holds in the major areas of neuromorphic technology, namely materials, devices, neuromorphic circuits, neuromorphic algorithms, applications, and ethics. The Roadmap is a collection of perspectives where leading researchers in the neuromorphic community provide their own view about the current state and the future challenges for each research area. We hope that this Roadmap will be a useful resource by providing a concise yet comprehensive introduction to readers outside this field, for those who are just entering the field, as well as providing future perspectives for those who are well established in the neuromorphic computing community.

Conventional neuro-computing architectures and artificial neural networks have often been developed with no or loose connections to neuroscience. As a consequence, they have largely ignored key features of biological neural processing systems, such as their extremely low-power consumption features or their ability to carry out robust and efficient computation using massively parallel arrays of limited precision, highly variable, and unreliable components. Recent developments in nano-technologies are making available extremely compact and low power, but also variable and unreliable solid-state devices that can potentially extend the offerings of availing CMOS technologies. In particular, memristors are regarded as a promising solution for modeling key features of biological synapses due to their nanoscale dimensions, their capacity to store multiple bits of information per element and the low energy required to write distinct states. In this paper, we first review the neuro- and neuromorphic computing approaches that can best exploit the properties of memristor and scale devices, and then propose a novel hybrid memristor-CMOS neuromorphic circuit which represents a radical departure from conventional neuro-computing approaches, as it uses memristors to directly emulate the biophysics and temporal dynamics of real synapses. We point out the differences between the use of memristors in conventional neuro-computing architectures and the hybrid memristor-CMOS circuit proposed, and argue how this circuit represents an ideal building block for implementing brain-inspired probabilistic computing paradigms that are robust to variability and fault tolerant by design.

In this paper we review several ways of realizing asynchronous Spike-Timing-Dependent-Plasticity (STDP) using memristors as synapses. Our focus is on how to use individual memristors to implement synaptic weight multiplications, in a way such that it is not necessary to (a) introduce global synchronization and (b) to separate memristor learning phases from memristor performing phases. In the approaches described, neurons fire spikes asynchronously when they wish and memristive synapses perform computation and learn at their own pace, as it happens in biological neural systems. We distinguish between two different memristor physics, depending on whether they respond to the original "moving wall" or to the "filament creation and annihilation" models. Independent of the memristor physics, we discuss two different types of STDP rules that can be implemented with memristors: either the pure timing-based rule that takes into account the arrival time of the spikes from the pre- and the post-synaptic neurons, or a hybrid rule that takes into account only the timing of pre-synaptic spikes and the membrane potential and other state variables of the post-synaptic neuron. We show how to implement these rules in cross-bar architectures that comprise massive arrays of memristors, and we discuss applications for artificial vision.

State-of-the-art image sensors suffer from significant limitations imposed by their very principle of operation. These sensors acquire the visual information as a series of “snapshot” images, recorded at discrete points in time. Visual information gets time quantized at a predetermined frame rate which has no relation to the dynamics present in the scene. Furthermore, each recorded frame conveys the information from all pixels, regardless of whether this information, or a part of it, has changed since the last frame had been acquired. This acquisition method limits the temporal resolution, potentially missing important information, and leads to redundancy in the recorded image data, unnecessarily inflating data rate and volume. Biology is leading the way to a more efficient style of image acquisition. Biological vision systems are driven by events happening within the scene in view, and not, like image sensors, by artificially created timing and control signals. Translating the frameless paradigm of biological vision to artificial imaging systems implies that control over the acquisition of visual information is no longer being imposed externally to an array of pixels but the decision making is transferred to the single pixel that handles its own information individually. In this paper, recent developments in bioinspired, neuromorphic optical sensing and artificial vision are presented and discussed. It is suggested that bioinspired vision systems have the potential to outperform conventional, frame-based vision systems in many application fields and to establish new benchmarks in terms of redundancy suppression and data compression, dynamic range, temporal resolution, and power efficiency. Demanding vision tasks such as real-time 3-D mapping, complex multiobject tracking, or fast visual feedback loops for sensory-motor action, tasks that often pose severe, sometimes insurmountable, challenges to conventional artificial vision systems, are in reach using bioinspired vision sensing and processing techniques.

Dynamic Vision Sensors (DVS) have recently appeared as a new paradigm for vision sensing and processing. They feature unique characteristics such as contrast coding under wide illumination variation, micro-second latency response to fast stimuli, and low output data rates (which greatly improves the efficiency of post-processing stages). They can track extremely fast objects (e.g., time resolution is better than 100 kFrames/s video) without special lighting conditions. Their availability has triggered a new range of vision applications in the fields of surveillance, motion analyses, robotics, and microscopic dynamic observations. One key DVS feature is contrast sensitivity, which has so far been reported to be in the 10-15% range. In this paper, a novel pixel photo sensing and transimpedance pre-amplification stage makes it possible to improve by one order of magnitude contrast sensitivity (down to 1.5%) and power (down to 4 mW), reduce the best reported FPN (Fixed Pattern Noise) by a factor of 2 (down to 0.9%), while maintaining the shortest reported latency (3 μs) and good Dynamic Range (120 dB), and further reducing overall area (down to 30 × 31 μm per pixel). The only penalty is the limitation of intrascene Dynamic Range to 3 decades. A 128 × 128 DVS test prototype has been fabricated in standard 0.35 μm CMOS and extensive experimental characterization results are provided.

Event-driven visual sensors have attracted interest from a number of different research communities. They provide visual information in quite a different way from conventional video systems consisting of sequences of still images rendered at a given "frame rate." Event-driven vision sensors take inspiration from biology. Each pixel sends out an event (spike) when it senses something meaningful is happening, without any notion of a frame. A special type of event-driven sensor is the so-called dynamic vision sensor (DVS) where each pixel computes relative changes of light or "temporal contrast." The sensor output consists of a continuous flow of pixel events that represent the moving objects in the scene. Pixel events become available with microsecond delays with respect to "reality." These events can be processed "as they flow" by a cascade of event (convolution) processors. As a result, input and output event flows are practically coincident in time, and objects can be recognized as soon as the sensor provides enough meaningful events. In this paper, we present a methodology for mapping from a properly trained neural network in a conventional frame-driven representation to an event-driven representation. The method is illustrated by studying event-driven convolutional neural networks (ConvNet) trained to recognize rotating human silhouettes or high speed poker card symbols. The event-driven ConvNet is fed with recordings obtained from a real DVS camera. The event-driven ConvNet is simulated with a dedicated event-driven simulator and consists of a number of event-driven processing modules, the characteristics of which are obtained from individually manufactured hardware modules.

This paper describes CAVIAR, a massively parallel hardware implementation of a spike-based sensing-processing-learning-actuating system inspired by the physiology of the nervous system. CAVIAR uses the asychronous address-event representation (AER) communication framework and was developed in the context of a European Union funded project. It has four custom mixed-signal AER chips, five custom digital AER interface components, 45k neurons (spiking cells), up to 5M synapses, performs 12G synaptic operations per second, and achieves millisecond object recognition and tracking latencies.

Networks with a modular structure are expected to have a lower risk of global failure. However, this theoretical result has remained untested until now. We used an experimental microarthropod metapopulation to test the effect of modularity on the response to perturbation. We perturbed one local population and measured the spread of the impact of this perturbation, both within and between modules. Our results show the buffering capacity of modular networks. To assess the generality of our findings, we then analyzed a dynamical model of our system. We show that in the absence of perturbations, modularity is negatively correlated with metapopulation size. However, even when a small local perturbation occurs, this negative effect is offset by a buffering effect that protects the majority of the nodes from the perturbation.

The four chips presented in the special session on "Activity-driven, event-based vision sensors" quickly output compressed digital data in the form of events. These sensors reduce redundancy and latency and increase dynamic range compared with conventional imagers. The digital sensor output is easily interfaced to conventional digital post processing, where it reduces the latency and cost of post processing compared to imagers. The asynchronous data could spawn a new area of DSP that breaks from conventional Nyquist rate signal processing. This paper reviews the rationale and history of this event-based approach, introduces sensor functionalities, and gives an overview of the papers in this session. The paper concludes with a brief discussion on open questions.

This paper presents a tutorial overview of <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex Notation="TeX">$\Sigma\Delta$</tex></formula> modulators, their operating principles and architectures, circuit errors and models, design methods, and practical issues. A review of the state of the art on nanometer CMOS implementations is described, giving a survey of cutting-edge <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex Notation="TeX">$\Sigma\Delta$</tex></formula> architectures, with emphasis on their application to the next generation of wireless telecom systems.

This paper is an in-depth review on silicon implementations of threshold logic gates that covers several decades. In this paper, we will mention early MOS threshold logic solutions and detail numerous very-large-scale integration (VLSI) implementations including capacitive (switched capacitor and floating gate with their variations), conductance/current (pseudo-nMOS and output-wired-inverters, including a plethora of solutions evolved from them), as well as many differential solutions. At the end, we will briefly mention other implementations, e.g., based on negative resistance devices and on single electron technologies.

The IEEE Mixed-Signal Technical Activity Committee is developing a common set of benchmark circuits for use in researching and evaluating analog fault modeling, test generation, design-for-test, and built-in self-test methodologies. The first release circuits are based on MITEL Semiconductor's 1.5 /spl mu/m and 1.2 /spl mu/m CMOS technologies and they will allow engineers and researchers working in analog and mixed-signal testing to compare test results as is done in the digital domain. This paper presents a set of typical circuits described by netlists in HSPICE format. Schematic diagrams, simulation results and measured results, if available, are provided together with layout and a typical test environment. The full details are available on the web page dedicated to analog and mixed-signal benchmarks.

This brief introduces a new floating memristor emulator circuit based on second-generation current conveyors and passive elements. A mathematical model to characterize the memristor behavior was derived, showing a good accuracy among HSPICE simulations and experimental results. An analysis of the frequency behavior of the memristor is also described, showing that the frequency-dependent pinched hysteresis loop in the current-versus-voltage plane holds up to 20.2 kHz. Theoretical derivations and related results are experimentally validated through implementations from commercially available devices, and the proposed memristor emulator circuit can easily be reproducible at a low cost. Furthermore, the emulator circuit can be used as a teaching aid and for future applications with memristors, such as sensors, cellular neural networks, chaotic systems, programmable analog circuits, and nonvolatile memory devices.

Today, with 0.18-/spl mu/m technologies mature and stable enough for mixed-signal design with a large variety of CMOS compatible optical sensors available and with 0.09-/spl mu/m technologies knocking at the door of designers, we can face the design of integrated systems, instead of just integrated circuits. In fact, significant progress has been made in the last few years toward the realization of vision systems on chips (VSoCs). Such VSoCs are eventually targeted to integrate within a semiconductor substrate the functions of optical sensing, image processing in space and time, high-level processing, and the control of actuators. The consecutive generations of ACE chips define a roadmap toward flexible VSoCs. These chips consist of arrays of mixed-signal processing elements (PEs) which operate in accordance with single instruction multiple data (SIMD) computing architectures and exhibit the functional features of CNN Universal Machines. They have been conceived to cover the early stages of the visual processing path in a fully-parallel manner, and hence more efficiently than DSP-based systems. Across the different generations, different improvements and modifications have been made looking to converge with the newest discoveries of neurobiologists regarding the behavior of natural retinas. This paper presents considerations pertaining to the design of a member of the third generation of ACE chips, namely to the so-called ACE16k chip. This chip, designed in a 0.35-/spl mu/m standard CMOS technology, contains about 3.75 million transistors and exhibits peak computing figures of 330 GOPS, 3.6 GOPS/mm/sup 2/ and 82.5 GOPS/W. Each PE in the array contains a reconfigurable computing kernel capable of calculating linear convolutions on 3/spl times/3 neighborhoods in less than 1.5 /spl mu/s, imagewise Boolean combinations in less than 200 ns, imagewise arithmetic operations in about 5 /spl mu/s, and CNN-like temporal evolutions with a time constant of about 0.5 /spl mu/s. Unfortunately, the many ideas underlying the design of this chip cannot be covered in a single paper; hence, this paper is focused on, first, placing the ACE16k in the ACE chip roadmap and, then, discussing the most significant modifications of ACE16K versus its predecessors in the family.

This paper introduces an event-driven feedforward categorization system, which takes data from a temporal contrast address event representation (AER) sensor. The proposed system extracts bio-inspired cortex-like features and discriminates different patterns using an AER based tempotron classifier (a network of leaky integrate-and-fire spiking neurons). One of the system's most appealing characteristics is its event-driven processing, with both input and features taking the form of address events (spikes). The system was evaluated on an AER posture dataset and compared with two recently developed bio-inspired models. Experimental results have shown that it consumes much less simulation time while still maintaining comparable performance. In addition, experiments on the Mixed National Institute of Standards and Technology (MNIST) image dataset have demonstrated that the proposed system can work not only on raw AER data but also on images (with a preprocessing step to convert images into AER events) and that it can maintain competitive accuracy even when noise is added. The system was further evaluated on the MNIST dynamic vision sensor dataset (in which data is recorded using an AER dynamic vision sensor), with testing accuracy of 88.14%.

The advent of dedicated Deep Learning (DL) accelerators and neuromorphic processors has brought on new opportunities for applying both Deep and Spiking Neural Network (SNN) algorithms to healthcare and biomedical applications at the edge. This can facilitate the advancement of medical Internet of Things (IoT) systems and Point of Care (PoC) devices. In this paper, we provide a tutorial describing how various technologies including emerging memristive devices, Field Programmable Gate Arrays (FPGAs), and Complementary Metal Oxide Semiconductor (CMOS) can be used to develop efficient DL accelerators to solve a wide variety of diagnostic, pattern recognition, and signal processing problems in healthcare. Furthermore, we explore how spiking neuromorphic processors can complement their DL counterparts for processing biomedical signals. The tutorial is augmented with case studies of the vast literature on neural network and neuromorphic hardware as applied to the healthcare domain. We benchmark various hardware platforms by performing a sensor fusion signal processing task combining electromyography (EMG) signals with computer vision. Comparisons are made between dedicated neuromorphic processors and embedded AI accelerators in terms of inference latency and energy. Finally, we provide our analysis of the field and share a perspective on the advantages, disadvantages, challenges, and opportunities that various accelerators and neuromorphic processors introduce to healthcare and biomedical domains.

This paper presents a 128 × 128 dynamic vision sensor. Each pixel detects temporal changes in the local illumination. A minimum illumination temporal contrast of 10% can be detected. A compact preamplification stage has been introduced that allows to improve the minimum detectable contrast over previous designs, while at the same time reducing the pixel area by 1/3. The pixel responds to illumination changes in less than 3.6 μs. The ability of the sensor to capture very fast moving objects, rotating at 10 K revolutions per second, has been verified experimentally. A frame-based sensor capable to achieve this, would require at least 100 K frames per second.

We show and validate a reliable circuit design technique based on source voltage shifting for current-mode signal processing down to femtoamperes. The technique involves specific-current extractors and logarithmic current splitters for obtaining on-chip subpicoampere currents. It also uses a special on-chip sawtooth oscillator to monitor and measure currents down to a few femtoamperes. This way, subpicoampere currents are characterized without driving them off chip and requiring expensive instrumentation with complicated low leakage setups. A special current mirror is also introduced for reliably replicating such low currents. As an example, a simple log-domain first-order low-pass filter is implemented that uses a 100-fF capacitor and a 3.5-fA bias current to achieve a cutoff frequency of 0.5 Hz. A technique for characterizing noise at these currents is also described and verified. Finally, transistor mismatch measurements are provided and discussed. Experimental measurements are shown throughout the paper, obtained from prototypes fabricated in the AMS 0.35-μm three-metal two-poly standard CMOS process.

A large effort is devoted to the research of new computing paradigms associated with innovative nanotechnologies that should complement and/or propose alternative solutions to the classical Von Neumann/CMOS (complementary metal oxide semiconductor) association. Among various propositions, spiking neural network (SNN) seems a valid candidate. i) In terms of functions, SNN using relative spike timing for information coding are deemed to be the most effective at taking inspiration from the brain to allow fast and efficient processing of information for complex tasks in recognition or classification. ii) In terms of technology, SNN may be able to benefit the most from nanodevices because SNN architectures are intrinsically tolerant to defective devices and performance variability. Here, spike‐timing‐dependent plasticity (STDP), a basic and primordial learning function in the brain, is demonstrated with a new class of synapstor (synapse‐transistor), called nanoparticle organic memory field‐effect transistor (NOMFET). This learning function is obtained with a simple hybrid material made of the self‐assembly of gold nanoparticles and organic semiconductor thin films. Beyond mimicking biological synapses, it is also demonstrated how the shape of the applied spikes can tailor the STDP learning function. Moreover, the experiments and modeling show that this synapstor is a memristive device. Finally, these synapstors are successfully coupled with a CMOS platform emulating the pre‐ and postsynaptic neurons, and a behavioral macromodel is developed on usual device simulator.