Brown Institute for Media Innovation

In spite of its familiar phenomenology, the mechanistic basis for mental effort remains poorly understood. Although most researchers agree that mental effort is aversive and stems from limitations in our capacity to exercise cognitive control, it is unclear what gives rise to those limitations and why they result in an experience of control as costly. The presence of these control costs also raises further questions regarding how best to allocate mental effort to minimize those costs and maximize the attendant benefits. This review explores recent advances in computational modeling and empirical research aimed at addressing these questions at the level of psychological process and neural mechanism, examining both the limitations to mental effort exertion and how we manage those limited cognitive resources. We conclude by identifying remaining challenges for theoretical accounts of mental effort as well as possible applications of the available findings to understanding the causes of and potential solutions for apparent failures to exert the mental effort required of us.

OBJECTIVE: Brain-computer interfaces (BCIs) using chronically implanted intracortical microelectrode arrays (MEAs) have the potential to restore lost function to people with disabilities if they work reliably for years. Current sensors fail to provide reliably useful signals over extended periods of time for reasons that are not clear. This study reports a comprehensive retrospective analysis from a large set of implants of a single type of intracortical MEA in a single species, with a common set of measures in order to evaluate failure modes. APPROACH: Since 1996, 78 silicon MEAs were implanted in 27 monkeys (Macaca mulatta). We used two approaches to find reasons for sensor failure. First, we classified the time course leading up to complete recording failure as acute (abrupt) or chronic (progressive). Second, we evaluated the quality of electrode recordings over time based on signal features and electrode impedance. Failure modes were divided into four categories: biological, material, mechanical, and unknown. MAIN RESULTS: Recording duration ranged from 0 to 2104 days (5.75 years), with a mean of 387 days and a median of 182 days (n = 78). Sixty-two arrays failed completely with a mean time to failure of 332 days (median = 133 days) while nine array experiments were electively terminated for experimental reasons (mean = 486 days). Seven remained active at the close of this study (mean = 753 days). Most failures (56%) occurred within a year of implantation, with acute mechanical failures the most common class (48%), largely because of connector issues (83%). Among grossly observable biological failures (24%), a progressive meningeal reaction that separated the array from the parenchyma was most prevalent (14.5%). In the absence of acute interruptions, electrode recordings showed a slow progressive decline in spike amplitude, noise amplitude, and number of viable channels that predicts complete signal loss by about eight years. Impedance measurements showed systematic early increases, which did not appear to affect recording quality, followed by a slow decline over years. The combination of slowly falling impedance and signal quality in these arrays indicates that insulating material failure is the most significant factor. SIGNIFICANCE: This is the first long-term failure mode analysis of an emerging BCI technology in a large series of non-human primates. The classification system introduced here may be used to standardize how neuroprosthetic failure modes are evaluated. The results demonstrate the potential for these arrays to record for many years, but achieving reliable sensors will require replacing connectors with implantable wireless systems, controlling the meningeal reaction, and improving insulation materials. These results will focus future research in order to create clinical neuroprosthetic sensors, as well as valuable research tools, that are able to safely provide reliable neural signals for over a decade.

Brain-computer interfaces (BCIs) have the potential to restore communication for people with tetraplegia and anarthria by translating neural activity into control signals for assistive communication devices. While previous pre-clinical and clinical studies have demonstrated promising proofs-of-concept (Serruya et al., 2002; Simeral et al., 2011; Bacher et al., 2015; Nuyujukian et al., 2015; Aflalo et al., 2015; Gilja et al., 2015; Jarosiewicz et al., 2015; Wolpaw et al., 1998; Hwang et al., 2012; Spüler et al., 2012; Leuthardt et al., 2004; Taylor et al., 2002; Schalk et al., 2008; Moran, 2010; Brunner et al., 2011; Wang et al., 2013; Townsend and Platsko, 2016; Vansteensel et al., 2016; Nuyujukian et al., 2016; Carmena et al., 2003; Musallam et al., 2004; Santhanam et al., 2006; Hochberg et al., 2006; Ganguly et al., 2011; O'Doherty et al., 2011; Gilja et al., 2012), the performance of human clinical BCI systems is not yet high enough to support widespread adoption by people with physical limitations of speech. Here we report a high-performance intracortical BCI (iBCI) for communication, which was tested by three clinical trial participants with paralysis. The system leveraged advances in decoder design developed in prior pre-clinical and clinical studies (Gilja et al., 2015; Kao et al., 2016; Gilja et al., 2012). For all three participants, performance exceeded previous iBCIs (Bacher et al., 2015; Jarosiewicz et al., 2015) as measured by typing rate (by a factor of 1.4-4.2) and information throughput (by a factor of 2.2-4.0). This high level of performance demonstrates the potential utility of iBCIs as powerful assistive communication devices for people with limited motor function.Clinical Trial No: NCT00912041.

Instrumental learning involves corticostriatal circuitry and the dopaminergic system. This system is typically modeled in the reinforcement learning (RL) framework by incrementally accumulating reward values of states and actions. However, human learning also implicates prefrontal cortical mechanisms involved in higher level cognitive functions. The interaction of these systems remains poorly understood, and models of human behavior often ignore working memory (WM) and therefore incorrectly assign behavioral variance to the RL system. Here we designed a task that highlights the profound entanglement of these two processes, even in simple learning problems. By systematically varying the size of the learning problem and delay between stimulus repetitions, we separately extracted WM-specific effects of load and delay on learning. We propose a new computational model that accounts for the dynamic integration of RL and WM processes observed in subjects' behavior. Incorporating capacity-limited WM into the model allowed us to capture behavioral variance that could not be captured in a pure RL framework even if we (implausibly) allowed separate RL systems for each set size. The WM component also allowed for a more reasonable estimation of a single RL process. Finally, we report effects of two genetic polymorphisms having relative specificity for prefrontal and basal ganglia functions. Whereas the COMT gene coding for catechol-O-methyl transferase selectively influenced model estimates of WM capacity, the GPR6 gene coding for G-protein-coupled receptor 6 influenced the RL learning rate. Thus, this study allowed us to specify distinct influences of the high-level and low-level cognitive functions on instrumental learning, beyond the possibilities offered by simple RL models.

The striatal dopaminergic system has been implicated in reinforcement learning (RL), motor performance, and incentive motivation. Various computational models have been proposed to account for each of these effects individually, but a formal analysis of their interactions is lacking. Here we present a novel algorithmic model expanding the classical actor-critic architecture to include fundamental interactive properties of neural circuit models, incorporating both incentive and learning effects into a single theoretical framework. The standard actor is replaced by a dual opponent actor system representing distinct striatal populations, which come to differentially specialize in discriminating positive and negative action values. Dopamine modulates the degree to which each actor component contributes to both learning and choice discriminations. In contrast to standard frameworks, this model simultaneously captures documented effects of dopamine on both learning and choice incentive-and their interactions-across a variety of studies, including probabilistic RL, effort-based choice, and motor skill learning.

One step conversion of lignin from intact biomass to two phenol products, improving access to aromatics and fuels.

Growing evidence suggests that the prefrontal cortex (PFC) is organized hierarchically, with more anterior regions having increasingly abstract representations. How does this organization support hierarchical cognitive control and the rapid discovery of abstract action rules? We present computational models at different levels of description. A neural circuit model simulates interacting corticostriatal circuits organized hierarchically. In each circuit, the basal ganglia gate frontal actions, with some striatal units gating the inputs to PFC and others gating the outputs to influence response selection. Learning at all of these levels is accomplished via dopaminergic reward prediction error signals in each corticostriatal circuit. This functionality allows the system to exhibit conditional if-then hypothesis testing and to learn rapidly in environments with hierarchical structure. We also develop a hybrid Bayesian-reinforcement learning mixture of experts (MoE) model, which can estimate the most likely hypothesis state of individual participants based on their observed sequence of choices and rewards. This model yields accurate probabilistic estimates about which hypotheses are attended by manipulating attentional states in the generative neural model and recovering them with the MoE model. This 2-pronged modeling approach leads to multiple quantitative predictions that are tested with functional magnetic resonance imaging in the companion paper.

Brain-computer interfaces (BCIs) promise to restore independence for people with severe motor disabilities by translating decoded neural activity directly into the control of a computer. However, recorded neural signals are not stationary (that is, can change over time), degrading the quality of decoding. Requiring users to pause what they are doing whenever signals change to perform decoder recalibration routines is time-consuming and impractical for everyday use of BCIs. We demonstrate that signal nonstationarity in an intracortical BCI can be mitigated automatically in software, enabling long periods (hours to days) of self-paced point-and-click typing by people with tetraplegia, without degradation in neural control. Three key innovations were included in our approach: tracking the statistics of the neural activity during self-timed pauses in neural control, velocity bias correction during neural control, and periodically recalibrating the decoder using data acquired during typing by mapping neural activity to movement intentions that are inferred retrospectively based on the user's self-selected targets. These methods, which can be extended to a variety of neurally controlled applications, advance the potential for intracortical BCIs to help restore independent communication and assistive device control for people with paralysis.

OBJECTIVE: Neural interface technology suitable for clinical translation has the potential to significantly impact the lives of amputees, spinal cord injury victims and those living with severe neuromotor disease. Such systems must be chronically safe, durable and effective. APPROACH: We have designed and implemented a neural interface microsystem, housed in a compact, subcutaneous and hermetically sealed titanium enclosure. The implanted device interfaces the brain with a 510k-approved, 100-element silicon-based microelectrode array via a custom hermetic feedthrough design. Full spectrum neural signals were amplified (0.1 Hz to 7.8 kHz, 200× gain) and multiplexed by a custom application specific integrated circuit, digitized and then packaged for transmission. The neural data (24 Mbps) were transmitted by a wireless data link carried on a frequency-shift-key-modulated signal at 3.2 and 3.8 GHz to a receiver 1 m away by design as a point-to-point communication link for human clinical use. The system was powered by an embedded medical grade rechargeable Li-ion battery for 7 h continuous operation between recharge via an inductive transcutaneous wireless power link at 2 MHz. MAIN RESULTS: Device verification and early validation were performed in both swine and non-human primate freely-moving animal models and showed that the wireless implant was electrically stable, effective in capturing and delivering broadband neural data, and safe for over one year of testing. In addition, we have used the multichannel data from these mobile animal models to demonstrate the ability to decode neural population dynamics associated with motor activity. SIGNIFICANCE: We have developed an implanted wireless broadband neural recording device evaluated in non-human primate and swine. The use of this new implantable neural interface technology can provide insight into how to advance human neuroprostheses beyond the present early clinical trials. Further, such tools enable mobile patient use, have the potential for wider diagnosis of neurological conditions and will advance brain research.

In order to understand the exploitation/exploration trade-off in reinforcement learning, previous theoretical and empirical accounts have suggested that increased uncertainty may precede the decision to explore an alternative option. To date, the neural mechanisms that support the strategic application of uncertainty-driven exploration remain underspecified. In this study, electroencephalography (EEG) was used to assess trial-to-trial dynamics relevant to exploration and exploitation. Theta-band activities over middle and lateral frontal areas have previously been implicated in EEG studies of reinforcement learning and strategic control. It was hypothesized that these areas may interact during top-down strategic behavioral control involved in exploratory choices. Here, we used a dynamic reward-learning task and an associated mathematical model that predicted individual response times. This reinforcement-learning model generated value-based prediction errors and trial-by-trial estimates of exploration as a function of uncertainty. Mid-frontal theta power correlated with unsigned prediction error, although negative prediction errors had greater power overall. Trial-to-trial variations in response-locked frontal theta were linearly related to relative uncertainty and were larger in individuals who used uncertainty to guide exploration. This finding suggests that theta-band activities reflect prefrontal-directed strategic control during exploratory choices.

Habits form a crucial component of behavior. In recent years, key computational models have conceptualized habits as arising from model-free reinforcement learning mechanisms, which typically select between available actions based on the future value expected to result from each. Traditionally, however, habits have been understood as behaviors that can be triggered directly by a stimulus, without requiring the animal to evaluate expected outcomes. Here, we develop a computational model instantiating this traditional view, in which habits develop through the direct strengthening of recently taken actions rather than through the encoding of outcomes. We demonstrate that this model accounts for key behavioral manifestations of habits, including insensitivity to outcome devaluation and contingency degradation, as well as the effects of reinforcement schedule on the rate of habit formation. The model also explains the prevalent observation of perseveration in repeated-choice tasks as an additional behavioral manifestation of the habit system. We suggest that mapping habitual behaviors onto value-free mechanisms provides a parsimonious account of existing behavioral and neural data. This mapping may provide a new foundation for building robust and comprehensive models of the interaction of habits with other, more goal-directed types of behaviors and help to better guide research into the neural mechanisms underlying control of instrumental behavior more generally. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Recent neuroimaging evidence has led to the proposal that freezing of gait in Parkinson's disease is due to dysfunctional interactions between frontoparietal cortical regions and subcortical structures, such as the striatum. However, to date, no study has employed task-based functional connectivity analyses to explore this hypothesis. In this study, we used a data-driven multivariate approach to explore the impaired communication between distributed neuronal networks in 10 patients with Parkinson's disease and freezing of gait, and 10 matched patients with no clinical history of freezing behaviour. Patients performed a virtual reality gait task on two separate occasions (once ON and once OFF their regular dopaminergic medication) while functional magnetic resonance imaging data were collected. Group-level independent component analysis was used to extract the subject-specific time courses associated with five well-known neuronal networks: the motor network, the right- and left cognitive control networks, the ventral attention network and the basal ganglia network. We subsequently analysed both the activation and connectivity of these neuronal networks between the two groups with respect to dopaminergic state and cognitive load while performing the virtual reality gait task. During task performance, all patients used the left cognitive control network and the ventral attention network and in addition, showed increased connectivity between the bilateral cognitive control networks. However, patients with freezing demonstrated functional decoupling between the basal ganglia network and the cognitive control network in each hemisphere. This decoupling was also associated with paroxysmal motor arrests. These results support the hypothesis that freezing behaviour in Parkinson's disease is because of impaired communication between complimentary yet competing neural networks.

The evolution of the field of neuroscience has been propelled by the advent of novel technological capabilities, and the pace at which these capabilities are being developed has accelerated dramatically in the past decade. Capitalizing on this momentum, the United States launched the Brain Research through Advancing Innovative Neurotechnologies (BRAIN) Initiative to develop and apply new tools and technologies for revolutionizing our understanding of the brain. In this article, we review the scientific vision for this initiative set forth by the National Institutes of Health and discuss its implications for the future of neuroscience research. Particular emphasis is given to its potential impact on the mapping and study of neural circuits, and how this knowledge will transform our understanding of the complexity of the human brain and its diverse array of behaviours, perceptions, thoughts and emotions.

Neurons in higher cortical areas appear to become active during action observation, either by mirroring observed actions (termed mirror neurons) or by eliciting mental rehearsal of observed motor acts. We report the existence of neurons in the primary motor cortex (M1), an area that is generally considered to initiate and guide movement performance, responding to viewed actions. Multielectrode recordings in monkeys performing or observing a well-learned step-tracking task showed that approximately half of the M1 neurons that were active when monkeys performed the task were also active when they observed the action being performed by a human. These 'view' neurons were spatially intermingled with 'do' neurons, which are active only during movement performance. Simultaneously recorded 'view' neurons comprised two groups: approximately 38% retained the same preferred direction (PD) and timing during performance and viewing, and the remainder (62%) changed their PDs and time lag during viewing as compared with performance. Nevertheless, population activity during viewing was sufficient to predict the direction and trajectory of viewed movements as action unfolded, although less accurately than during performance. 'View' neurons became less active and contained poorer representations of action when only subcomponents of the task were being viewed. M1 'view' neurons thus appear to reflect aspects of a learned movement when observed in others, and form part of a broadly engaged set of cortical areas routinely responding to learned behaviors. These findings suggest that viewing a learned action elicits replay of aspects of M1 activity needed to perform the observed action, and could additionally reflect processing related to understanding, learning or mentally rehearsing action.

The frontal lobes may be organized hierarchically such that more rostral frontal regions modulate cognitive control operations in caudal regions. In our companion paper (Frank MJ, Badre D. 2011. Mechanisms of hierarchical reinforcement learning in corticostriatal circuits I: computational analysis. 22:509-526), we provide novel neural circuit and algorithmic models of hierarchical cognitive control in cortico-striatal circuits. Here, we test key model predictions using functional magnetic resonance imaging (fMRI). Our neural circuit model proposes that contextual representations in rostral frontal cortex influence the striatal gating of contextual representations in caudal frontal cortex. Reinforcement learning operates at each level, such that the system adaptively learns to gate higher order contextual information into rostral regions. Our algorithmic Bayesian "mixture of experts" model captures the key computations of this neural model and provides trial-by-trial estimates of the learner's latent hypothesis states. In the present paper, we used these quantitative estimates to reanalyze fMRI data from a hierarchical reinforcement learning task reported in Badre D, Kayser AS, D'Esposito M. 2010. Frontal cortex and the discovery of abstract action rules. Neuron. 66:315--326. Results validate key predictions of the models and provide evidence for an individual cortico-striatal circuit for reinforcement learning of hierarchical structure at a specific level of policy abstraction. These findings are initially consistent with the proposal that hierarchical control in frontal cortex may emerge from interactions among nested cortico-striatal circuits at different levels of abstraction.

Neural activity in motor cortex during reach and grasp movements shows modulations in a broad range of signals from single-neuron spiking activity (SA) to various frequency bands in broadband local field potentials (LFPs). In particular, spatiotemporal patterns in multiband LFPs are thought to reflect dendritic integration of local and interareal synaptic inputs, attentional and preparatory processes, and multiunit activity (MUA) related to movement representation in the local motor area. Nevertheless, the relationship between multiband LFPs and SA, and their relationship to movement parameters and their relative value as brain-computer interface (BCI) control signals, remain poorly understood. Also, although this broad range of signals may provide complementary information channels in primary (MI) and ventral premotor (PMv) areas, areal differences in information have not been systematically examined. Here, for the first time, the amount of information in SA and multiband LFPs was compared for MI and PMv by recording from dual 96-multielectrode arrays while monkeys made naturalistic reach and grasp actions. Information was assessed as decoding accuracy for 3D arm end point and grip aperture kinematics based on SA or LFPs in MI and PMv, or combinations of signal types across areas. In contrast with previous studies with ≤16 simultaneous electrodes, here ensembles of >16 units (on average) carried more information than multiband, multichannel LFPs. Furthermore, reach and grasp information added by various LFP frequency bands was not independent from that in SA ensembles but rather typically less than and primarily contained within the latter. Notably, MI and PMv did not show a particular bias toward reach or grasp for this task or for a broad range of signal types. For BCIs, our results indicate that neuronal ensemble spiking is the preferred signal for decoding, while LFPs and combined signals from PMv and MI can add robustness to BCI control.

In the last several years, a branch of applied econometrics has developed that is devoted to the development of techniques for the estimation of demand and other functions when the budget constraint is “piecewise-linear,” or “kinky”—that is, when the constraint consists of a number of segments joined together at kink points. Such constraints most frequently arise from government tax and transfer programs. But kinked constraints sometimes arise in nongovernment contexts, a well-known example being the block pricing schedule commonly set by utilities and volume discounting in general. Kinked budget constraints create two difficulties. First, changes in tax and transfer schedules can have unexpected effects that can be exactly the opposite in sign to those expected from economic theory. Examples of this phenomenon are given below. Second, kinked budget constraints make the estimation of demand functions quite difficult.

Journal Article An Experimental Investigation on Rickets Get access Edward Mellanby Edward Mellanby Brown Institute, London University Search for other works by this author on: Oxford Academic PubMed Google Scholar Nutrition Reviews, Volume 34, Issue 11, November 1976, Pages 338–340, https://doi.org/10.1111/j.1753-4887.1976.tb05815.x Published: 01 November 1976

The striatum is critical for the incremental learning of values associated with behavioral actions. The prefrontal cortex (PFC) represents abstract rules and explicit contingencies to support rapid behavioral adaptation in the absence of cumulative experience. Here we test two alternative models of the interaction between these systems, and individual differences thereof, when human subjects are instructed with prior information about reward contingencies that may or may not be accurate. Behaviorally, subjects are overly influenced by prior instructions, at the expense of learning true reinforcement statistics. Computational analysis found that this pattern of data is best accounted for by a confirmation bias mechanism in which prior beliefs--putatively represented in PFC--influence the learning that occurs in the striatum such that reinforcement statistics are distorted. We assessed genetic variants affecting prefrontal and striatal dopaminergic neurotransmission. A polymorphism in the COMT gene (rs4680), associated with prefrontal dopaminergic function, was predictive of the degree to which participants persisted in responding in accordance with prior instructions even as evidence against their veracity accumulated. Polymorphisms in genes associated with striatal dopamine function (DARPP-32, rs907094, and DRD2, rs6277) were predictive of learning from positive and negative outcomes. Notably, these same variants were predictive of the degree to which such learning was overly inflated or neglected when outcomes are consistent or inconsistent with prior instructions. These findings indicate dissociable neurocomputational and genetic mechanisms by which initial biases are strengthened by experience.

Damage to the orbitofrontal cortex (OFC) has been linked to impaired reinforcement processing and maladaptive behavior in changing environments across species. Flexible stimulus-outcome learning, canonically captured by reversal learning tasks, has been shown to rely critically on OFC in rats, monkeys, and humans. However, the precise role of OFC in this learning remains unclear. Furthermore, whether other frontal regions also contribute has not been definitively established, particularly in humans. In the present study, a reversal learning task with probabilistic feedback was administered to 39 patients with focal lesions affecting various sectors of the frontal lobes and to 51 healthy, demographically matched control subjects. Standard groupwise comparisons were supplemented with voxel-based lesion-symptom mapping to identify regions within the frontal lobes critical for task performance. Learning in this dynamic stimulus-reinforcement environment was considered both in terms of overall performance and at the trial-by-trial level. In this challenging, probabilistic context, OFC damage disrupted both initial and reversal learning. Trial-by-trial performance patterns suggest that OFC plays a critical role in interpreting feedback from a particular trial within the broader context of the outcome history across trials rather than in simply suppressing preexisting stimulus-outcome associations. The findings show that OFC, and not other prefrontal regions, plays a necessary role in flexible stimulus-reinforcement learning in humans.