Wellcome / EPSRC Centre for Interventional and Surgical Sciences

Convolutional neural networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they have not demonstrated sufficiently accurate and robust results for clinical use. In addition, they are limited by the lack of image-specific adaptation and the lack of generalizability to previously unseen object classes (a.k.a. zero-shot learning). To address these problems, we propose a novel deep learning-based interactive segmentation framework by incorporating CNNs into a bounding box and scribble-based segmentation pipeline. We propose image-specific fine tuning to make a CNN model adaptive to a specific test image, which can be either unsupervised (without additional user interactions) or supervised (with additional scribbles). We also propose a weighted loss function considering network and interaction-based uncertainty for the fine tuning. We applied this framework to two applications: 2-D segmentation of multiple organs from fetal magnetic resonance (MR) slices, where only two types of these organs were annotated for training and 3-D segmentation of brain tumor core (excluding edema) and whole brain tumor (including edema) from different MR sequences, where only the tumor core in one MR sequence was annotated for training. Experimental results show that: 1) our model is more robust to segment previously unseen objects than state-of-the-art CNNs; 2) image-specific fine tuning with the proposed weighted loss function significantly improves segmentation accuracy; and 3) our method leads to accurate results with fewer user interactions and less user time than traditional interactive segmentation methods.

Automatic segmentation of abdominal anatomy on computed tomography (CT) images can support diagnosis, treatment planning, and treatment delivery workflows. Segmentation methods using statistical models and multi-atlas label fusion (MALF) require inter-subject image registrations, which are challenging for abdominal images, but alternative methods without registration have not yet achieved higher accuracy for most abdominal organs. We present a registration-free deep-learning-based segmentation algorithm for eight organs that are relevant for navigation in endoscopic pancreatic and biliary procedures, including the pancreas, the gastrointestinal tract (esophagus, stomach, and duodenum) and surrounding organs (liver, spleen, left kidney, and gallbladder). We directly compared the segmentation accuracy of the proposed method to the existing deep learning and MALF methods in a cross-validation on a multi-centre data set with 90 subjects. The proposed method yielded significantly higher Dice scores for all organs and lower mean absolute distances for most organs, including Dice scores of 0.78 versus 0.71, 0.74, and 0.74 for the pancreas, 0.90 versus 0.85, 0.87, and 0.83 for the stomach, and 0.76 versus 0.68, 0.69, and 0.66 for the esophagus. We conclude that the deep-learning-based segmentation represents a registration-free method for multi-organ abdominal CT segmentation whose accuracy can surpass current methods, potentially supporting image-guided navigation in gastrointestinal endoscopy procedures.

Despite the state-of-the-art performance for medical image segmentation, deep convolutional neural networks (CNNs) have rarely provided uncertainty estimations regarding their segmentation outputs, e.g., model (epistemic) and image-based (aleatoric) uncertainties. In this work, we analyze these different types of uncertainties for CNN-based 2D and 3D medical image segmentation tasks at both pixel level and structure level. We additionally propose a test-time augmentation-based aleatoric uncertainty to analyze the effect of different transformations of the input image on the segmentation output. Test-time augmentation has been previously used to improve segmentation accuracy, yet not been formulated in a consistent mathematical framework. Hence, we also propose a theoretical formulation of test-time augmentation, where a distribution of the prediction is estimated by Monte Carlo simulation with prior distributions of parameters in an image acquisition model that involves image transformations and noise. We compare and combine our proposed aleatoric uncertainty with model uncertainty. Experiments with segmentation of fetal brains and brain tumors from 2D and 3D Magnetic Resonance Images (MRI) showed that 1) the test-time augmentation-based aleatoric uncertainty provides a better uncertainty estimation than calculating the test-time dropout-based model uncertainty alone and helps to reduce overconfident incorrect predictions, and 2) our test-time augmentation outperforms a single-prediction baseline and dropout-based multiple predictions.

BACKGROUND AND OBJECTIVE: Processing of medical images such as MRI or CT presents different challenges compared to RGB images typically used in computer vision. These include a lack of labels for large datasets, high computational costs, and the need of metadata to describe the physical properties of voxels. Data augmentation is used to artificially increase the size of the training datasets. Training with image subvolumes or patches decreases the need for computational power. Spatial metadata needs to be carefully taken into account in order to ensure a correct alignment and orientation of volumes. METHODS: We present TorchIO, an open-source Python library to enable efficient loading, preprocessing, augmentation and patch-based sampling of medical images for deep learning. TorchIO follows the style of PyTorch and integrates standard medical image processing libraries to efficiently process images during training of neural networks. TorchIO transforms can be easily composed, reproduced, traced and extended. Most transforms can be inverted, making the library suitable for test-time augmentation and estimation of aleatoric uncertainty in the context of segmentation. We provide multiple generic preprocessing and augmentation operations as well as simulation of MRI-specific artifacts. RESULTS: Source code, comprehensive tutorials and extensive documentation for TorchIO can be found at http://torchio.rtfd.io/. The package can be installed from the Python Package Index (PyPI) running pip install torchio. It includes a command-line interface which allows users to apply transforms to image files without using Python. Additionally, we provide a graphical user interface within a TorchIO extension in 3D Slicer to visualize the effects of transforms. CONCLUSION: TorchIO was developed to help researchers standardize medical image processing pipelines and allow them to focus on the deep learning experiments. It encourages good open-science practices, as it supports experiment reproducibility and is version-controlled so that the software can be cited precisely. Due to its modularity, the library is compatible with other frameworks for deep learning with medical images.

BACKGROUND AND OBJECTIVES: Medical image analysis and computer-assisted intervention problems are increasingly being addressed with deep-learning-based solutions. Established deep-learning platforms are flexible but do not provide specific functionality for medical image analysis and adapting them for this domain of application requires substantial implementation effort. Consequently, there has been substantial duplication of effort and incompatible infrastructure developed across many research groups. This work presents the open-source NiftyNet platform for deep learning in medical imaging. The ambition of NiftyNet is to accelerate and simplify the development of these solutions, and to provide a common mechanism for disseminating research outputs for the community to use, adapt and build upon. METHODS: The NiftyNet infrastructure provides a modular deep-learning pipeline for a range of medical imaging applications including segmentation, regression, image generation and representation learning applications. Components of the NiftyNet pipeline including data loading, data augmentation, network architectures, loss functions and evaluation metrics are tailored to, and take advantage of, the idiosyncracies of medical image analysis and computer-assisted intervention. NiftyNet is built on the TensorFlow framework and supports features such as TensorBoard visualization of 2D and 3D images and computational graphs by default. RESULTS: We present three illustrative medical image analysis applications built using NiftyNet infrastructure: (1) segmentation of multiple abdominal organs from computed tomography; (2) image regression to predict computed tomography attenuation maps from brain magnetic resonance images; and (3) generation of simulated ultrasound images for specified anatomical poses. CONCLUSIONS: The NiftyNet infrastructure enables researchers to rapidly develop and distribute deep learning solutions for segmentation, regression, image generation and representation learning applications, or extend the platform to new applications.

We develop a conceptual framework for the understanding of animal personalities in terms of adaptive evolution. We focus on two basic questions. First, why do behavioural types exhibit limited behavioural plasticity, that is, behavioural correlations both across contexts and over time? Second, how can multiple behavioural types coexist within a single population? We emphasize differences in 'state' among individuals in combination with state-dependent behaviour. Some states are inherently stable and individual differences in such states can explain stable differences in suites of behaviour if it is adaptive to make behaviour in various contexts dependent on such states. Behavioural stability and cross-context correlations in behaviour are more difficult to explain if individual states are potentially more variable. In such cases stable personalities can result from state-dependent behaviour if state and behaviour mutually reinforce each other by feedback mechanisms. We discuss various evolutionary mechanisms for the maintenance of variation (in states and/or behaviour), including frequency-dependent selection, spatial variation with incomplete matching between habitat and phenotype, bet-hedging in a temporally fluctuating environment, and non-equilibrium dynamics. Although state differences are important, we also discuss how social conventions and social signalling can give rise to adaptive personality differences in the absence of state differences.

Accurate medical image segmentation is essential for diagnosis, surgical planning and many other applications. Convolutional Neural Networks (CNNs) have become the state-of-the-art automatic segmentation methods. However, fully automatic results may still need to be refined to become accurate and robust enough for clinical use. We propose a deep learning-based interactive segmentation method to improve the results obtained by an automatic CNN and to reduce user interactions during refinement for higher accuracy. We use one CNN to obtain an initial automatic segmentation, on which user interactions are added to indicate mis-segmentations. Another CNN takes as input the user interactions with the initial segmentation and gives a refined result. We propose to combine user interactions with CNNs through geodesic distance transforms, and propose a resolution-preserving network that gives a better dense prediction. In addition, we integrate user interactions as hard constraints into a back-propagatable Conditional Random Field. We validated the proposed framework in the context of 2D placenta segmentation from fetal MRI and 3D brain tumor segmentation from FLAIR images. Experimental results show our method achieves a large improvement from automatic CNNs, and obtains comparable and even higher accuracy with fewer user interventions and less time compared with traditional interactive methods.

One of the fundamental challenges in supervised learning for multimodal image registration is the lack of ground-truth for voxel-level spatial correspondence. This work describes a method to infer voxel-level transformation from higher-level correspondence information contained in anatomical labels. We argue that such labels are more reliable and practical to obtain for reference sets of image pairs than voxel-level correspondence. Typical anatomical labels of interest may include solid organs, vessels, ducts, structure boundaries and other subject-specific ad hoc landmarks. The proposed end-to-end convolutional neural network approach aims to predict displacement fields to align multiple labelled corresponding structures for individual image pairs during the training, while only unlabelled image pairs are used as the network input for inference. We highlight the versatility of the proposed strategy, for training, utilising diverse types of anatomical labels, which need not to be identifiable over all training image pairs. At inference, the resulting 3D deformable image registration algorithm runs in real-time and is fully-automated without requiring any anatomical labels or initialisation. Several network architecture variants are compared for registering T2-weighted magnetic resonance images and 3D transrectal ultrasound images from prostate cancer patients. A median target registration error of 3.6 mm on landmark centroids and a median Dice of 0.87 on prostate glands are achieved from cross-validation experiments, in which 108 pairs of multimodal images from 76 patients were tested with high-quality anatomical labels.

Concentrating on monomorphic populations in demographic steady state, we give three different conditions under which the evolutionarily stable life-history strategy can be characterized as the life-history strategy at which a relatively simple function is maximal. Depending on the way density dependence acts, this function, or fitness measure, can be the life-time production of offspring, the population growth rate, or another quantity from a large range of possible optimization criteria. We illustrate this by examining the optimal age at maturity for a hypothetical organism. All of this demonstrates that, when studying the evolutionary aspects of life-history characteristics, one cannot escape the task of specifying how density dependence limits population growth.

Little direct evidence in man indicates that acute right ventricular loading alters left ventricular configuration. We used the Mueller maneuver (forced inspiration against a closed airway) to increase right ventricular loading and evaluated septal shape and right and left ventricular size in nine normal, semisupine men with phased-array, two-dimensional echocardiography. End-systolic and end-diastolic frames in cross-sectional and longitudinal views of the ventricles were recorded at rest and at various phases during the Mueller period (peak inspiratory effort of 40-60 mm Hg negative pressure). Acute leftward displacement of the septum at end-diastole on cross section during the maximal early Mueller period (first two or three beats after the onset of Mueller maneuver) was evidenced by a substantial increase in the radius of curvature of the septal segment (3.72 0.25 cm vs control, 2.49 0.12 cm,p < 0.001). This leftward septal displacement per- sisted not only during end-diastole, but also during end-systole (3.58 0.45 vs 2.04 0.16 cm;p < 0.01). The septal radius of curvature did not differ from the radius of curvature of the remainder of the left ventricle at rest for systole or diastole (1.94 0.11 and 2.48 0.09 cm, respectively), but differed markedly during the early Mueller phase in both systole (3.58 + 0.45 vs 1.9 0.07 cm; p < 0.005) and diastole (3.72 0.25 vs 2.36 0.07 cm; p < 0.001). Simultaneously, left ventricular end-diastolic cavity areas decreased from control to the early Mueller phase on cross-sectional view from 19.14 1.08 cm2 to 15.73 0.65 cm2 (p < 0.005), and longitudinal view from 29.83 2.08 to 20.74 1.46 cm2; p < 0.001. A significant decrease in end- systolic cavity area was also noted in this view (19.72 i 2.0 to 15.23 1.98 cm2;p < 0.05). Right ventricular end-diastolic diameter increased from control to the early Mueller phase in the cross-sectional view (1.06 0.14 to 1.31 0.17 cm;p < 0.02), as well as in the longitudinal view (1.14 0.23 to 1.80 0.43 cm; p < 0.05). A decrease in left ventricular volume with maintenance of constant shape should result in a shortened radius of curvature for all portions of the ventricle, so the increase in septal radius of curvature in the face of an overall decrease in left ventricular size indicates that right ventricular loading alters left ventricular shape by flattening the septum. This septal flattening persists during systole. Thus, changed septal shape may be an important mechanism of, and evidence for, ventricular interdependence in normal man.

Light is never distributed homogeneously since it forms a gradient over biomass. As a consequence, the common theories on nutrient competition are not applicable to competition for light. In this paper, we investigate a model for light—limited growth and competition among phytoplankton species in a mixed water column. The model is based on standard assumptions such as Lambert—Beer's law of light absorption, a Monod equation for carbon uptake, and constant specific carbon losses. By introducing the concept of quantum return, we show that the dynamics of growth and competition can be quantified not only in terms of depth but also directly in terms of light availability. We argued that the crucial measure for phytoplankton growth is not a "critical depth" but a "critical light intensity," I* o u t . For each species, I* o u t corresponds to the equilibrium light intensity at the bottom of a water column when the species is grown in monoculture. I* o u t plays a role similar to the "critical nutrient concentration" R* used in models of nutrient—limited growth. For a constant light supply, the species with the lowest I* o u t will competitively exclude all other species. There are, however, some important differences between R* and I* o u t . Whereas R* reflects both the local and the total balance between nutrient uptake and nutrient losses, I* o u t only reflects the total carbon balance. Moreover, I* o u t decreases with increasing light supply, whereas R* is independent of the nutrient supply. As a consequence, (1) the outcome of competition for light may depend on the light supply, (2) the compensation point is not a good predictor for the outcome of competition, (3) the resource ratio hypothesis does not apply when species compete for both nutrients and light. The outcome of competition for nutrients and light may depend on the nutrient and light supply, on the mixing depth, and on the background turbidity due to inanimate substances.

To advance the field of soft robotics, a unified database of material constitutive models and experimental characterizations is of paramount importance. This will facilitate the use of finite element analysis to simulate their behavior and optimize the design of soft-bodied robots. Samples from seventeen elastomers, namely Body Double™ SILK, Dragon Skin™ 10 MEDIUM, Dragon Skin 20, Dragon Skin 30, Dragon Skin FX-Pro, Dragon Skin FX-Pro + Slacker, Ecoflex™ 00–10, Ecoflex 00–30, Ecoflex 00–50, Rebound™ 25, Mold Star™ 16 FAST, Mold Star 20T, SORTA-Clear™ 40, RTV615, PlatSil ® Gel-10, Psycho Paint ® , and SOLOPLAST 150318, were subjected to uniaxial tensile tests according to the ASTM D412 standard. Sample preparation and tensile test parameters are described in detail. The tensile test data are used to derive parameters for hyperelastic material models using nonlinear least-squares methods, which are provided to the reader. This article presents the mechanical characterization and the resulting material properties for a wide set of commercially available hyperelastic materials, many of which are recognized and commonly applied in the field of soft robotics, together with some that have never been characterized. The experimental raw data and the algorithms used to determine material parameters are shared on the Soft Robotics Materials Database GitHub repository to enable accessibility, as well as future contributions from the soft robotics community. The presented database is aimed at aiding soft roboticists in designing and modeling soft robots while providing a starting point for future material characterizations related to soft robotics research.

Abstract A fundamental assumption underlying the importance of genetic risks within conservation biology is that inbreeding increases the extinction probability of populations. Although inbreeding has been shown to have a detrimental impact on individual fitness, its contribution to extinction is still poorly understood. We have studied the consequences of different levels of prior inbreeding for the persistence of small populations using Drosophila melanogaster as a model organism. To this end, we determined the extinction rate of small vial populations differing in the level of inbreeding under both optimal and stress conditions, i.e. high temperature stress and ethanol stress. We show that inbred populations have a significantly higher short-term probability of extinction than non-inbred populations, even for low levels of inbreeding, and that the extinction probability increases with increasing inbreeding levels. In addition, we observed that the effects of inbreeding become greatly enhanced under stressful environmental conditions. More importantly, our results show that the impact of environmental stress becomes significantly greater for higher inbreeding levels, demonstrating explicitly that inbreeding and environmental stress are not independent but can act synergistically. These effects seem long lasting as the impact of prior inbreeding was still qualitatively the same after the inbred populations had been expanded to appreciable numbers and maintained as such for approximately 50 generations. Our observations have significant consequences for conservation biology.

High-resolution volume reconstruction from multiple motion-corrupted stacks of 2D slices plays an increasing role for fetal brain Magnetic Resonance Imaging (MRI) studies. Currently existing reconstruction methods are time-consuming and often require user interactions to localize and extract the brain from several stacks of 2D slices. We propose a fully automatic framework for fetal brain reconstruction that consists of four stages: 1) fetal brain localization based on a coarse segmentation by a Convolutional Neural Network (CNN), 2) fine segmentation by another CNN trained with a multi-scale loss function, 3) novel, single-parameter outlier-robust super-resolution reconstruction, and 4) fast and automatic high-resolution visualization in standard anatomical space suitable for pathological brains. We validated our framework with images from fetuses with normal brains and with variable degrees of ventriculomegaly associated with open spina bifida, a congenital malformation affecting also the brain. Experiments show that each step of our proposed pipeline outperforms state-of-the-art methods in both segmentation and reconstruction comparisons including expert-reader quality assessments. The reconstruction results of our proposed method compare favorably with those obtained by manual, labor-intensive brain segmentation, which unlocks the potential use of automatic fetal brain reconstruction studies in clinical practice.

To assess the accuracy of phased-array two-dimensional echocardiography in detecting, localizing and quantifying myocardial injury in man, the relationship of two-dimensional echocardiographic wall motion abnormalities to morphologic evidence of myocardial infarction was evaluated in 20 autopsied patients. Comparisons were made between independent two-dimensional echocardiographic readings of left ventricular segmental wall motion and morphologic evidence of myocardial injury. Of 15 infarcts, 14 were detected by regional akinesis, dyskinesis or hypokinesis. The relationship between abnormal segmental wall motion and morphologic evidence of myocardial necrosis or fibrosis was significant. Seventy-nine of 88 (90%) of infarcted segments showed abnormal wall motion, although 38 of 82 (46%) of morphologically normal segments also demonstrated wall motion abnormalities. Fifty-eight of 65 segments that showed regional akinesis or dyskinesis were transmurally infarcted. Twenty-five of 38 pathologically normal segments seen by two-dimensional echocardiography as akinetic or dyskinetic were adjacent to scar. Hypokinesis was non- specific (31 segments normal, 21 subendocardial infarction). Normal wall motion excluded transmural infarc- tion (0 of 46 segments), but in one patient was associated with subendocardial injury (nine/42 segments).

Mathematical models have played an important role in the development of sexual selection theory. These models come in different flavors and they differ in their assumptions, often in a subtle way. Similar questions can be addressed by modeling frameworks from population genetics, quantitative genetics, evolutionary game theory, or adaptive dynamics, or by individual-based simulations. Confronted with such diversity, nonspecialists may have difficulties judging the scope and limitations of the various approaches. Here we review the major modeling frameworks, highlighting their pros and cons when applied to different research questions. We also discuss recent developments, where classical models are enriched by including more detail regarding genetics, behavior, demography, and population dynamics. It turns out that some seemingly well-established conclusions of sexual selection theory are less general than previously thought. Linking sexual selection to other processes such as sex-ratio evolution or speciation also reveals that enriching the theory can lead to surprising new insights.

When bacteria grow in a medium with two sugars, they first use the preferred sugar and only then start metabolizing the second one. After the first exponential growth phase, a short lag phase of nongrowth is observed, a period called the diauxie lag phase. It is commonly seen as a phase in which the bacteria prepare themselves to use the second sugar. Here we reveal that, in contrast to the established concept of metabolic adaptation in the lag phase, two stable cell types with alternative metabolic strategies emerge and coexist in a culture of the bacterium Lactococcus lactis. Only one of them continues to grow. The fraction of each metabolic phenotype depends on the level of catabolite repression and the metabolic state-dependent induction of stringent response, as well as on epigenetic cues. Furthermore, we show that the production of alternative metabolic phenotypes potentially entails a bet-hedging strategy. This study sheds new light on phenotypic heterogeneity during various lag phases occurring in microbiology and biotechnology and adjusts the generally accepted explanation of enzymatic adaptation proposed by Monod and shared by scientists for more than half a century.

We analyze the consequences of intraguild predation and stage structure for the possible composition of a three-species community consisting of resource, consumer, and predator. Intraguild predation, a special case of omnivory, induces two major differences with traditional linear food chain models: the potential for the occurrence of two alternative stable equilibria at intermediate levels of resource productivity and the extinction of the consumer at high productivities. At low productivities, the consumer dominates, while at intermediate productivities, the predator and the consumer can coexist. The qualitative behavior of the model is robust against addition of an invulnerable size class for the consumer population and against addition of an initial, nonpredatory stage for the predator population, which means that the addition of stage structure does not change the pattern. Unless the top predator is substantially less efficient on the bottom resource, it tends to drive the intermediate species extinct over a surprisingly large range of productivities, thus making coexistence generally impossible. These theoretical results indicate that the conditions for stable food chains involving intraguild predation cannot involve strong competition for the bottommost resource.

OBJECTIVE: The reporting of depressive symptoms following myocardial infarction may be confounded by complaints originating from the myocardial infarction. Therefore, it is difficult to estimate the effects of post-myocardial infarction depression and its treatment on cardiovascular prognosis. The authors' goal was to study the relationship between depressive symptom dimensions following myocardial infarction and both somatic health status and prospective cardiovascular prognosis. METHOD: In two studies of myocardial infarction patients (N=494 and 1,972), the Beck Depression Inventory was used to determine the dimensional structure of depressive symptoms following myocardial infarction. Three symptom dimensions-somatic/affective, cognitive/affective, and appetitive-were compared with baseline left ventricular ejection fraction, Charlson comorbidity index, Killip class, and previous myocardial infarction. The relationship between depressive symptom dimensions and prospective cardiovascular mortality and cardiac-related readmissions was also examined (mean follow-up duration=2.5 years). RESULTS: Somatic/affective symptoms were associated with poor health status (left ventricular ejection fraction, Charlson comorbidity index, Killip class, and previous myocardial infarction) and predicted cardiovascular mortality and cardiac events. Cognitive/affective symptoms were only marginally associated with somatic health status and not with cardiovascular death and cardiac events. Appetitive symptoms were related to somatic health status but did not predict cardiovascular death or cardiac events. CONCLUSIONS: Somatic/affective depressive symptoms following myocardial infarction were confounded by somatic health status yet were prospectively associated with cardiac prognosis even after somatic health status was controlled. Cognitive/affective depressive symptoms were only marginally related to health status and not to cardiac prognosis. These findings suggest that treatment of depression following myocardial infarction might improve cardiovascular prognosis when it reduces somatic/affective symptoms.

One of the central goals of ecology is to predict the distribution and abundance of organisms. Here, we show that, in ecosystems of high biodiversity, the outcome of multispecies competition can be fundamentally unpredictable. We consider a competition model widely applied in phytoplankton ecology and plant ecology in which multiple species compete for three resources. We show that this competition model may have several alternative outcomes, that the dynamics leading to these alternative outcomes may exhibit transient chaos, and that the basins of attraction of these alternative outcomes may have an intermingled fractal geometry. As a consequence of this fractal geometry, it is impossible to predict the winners of multispecies competition in advance.