Max Planck Institute for Intelligent Systems

We present a novel dataset captured from a VW station wagon for use in mobile robotics and autonomous driving research. In total, we recorded 6 hours of traffic scenarios at 10–100 Hz using a variety of sensor modalities such as high-resolution color and grayscale stereo cameras, a Velodyne 3D laser scanner and a high-precision GPS/IMU inertial navigation system. The scenarios are diverse, capturing real-world traffic situations, and range from freeways over rural areas to inner-city scenes with many static and dynamic objects. Our data is calibrated, synchronized and timestamped, and we provide the rectified and raw image sequences. Our dataset also contains object labels in the form of 3D tracklets, and we provide online benchmarks for stereo, optical flow, object detection and other tasks. This paper describes our recording platform, the data format and the utilities that we provide.

We present a learned model of human body shape and pose-dependent shape variation that is more accurate than previous models and is compatible with existing graphics pipelines. Our Skinned Multi-Person Linear model (SMPL) is a skinned vertex-based model that accurately represents a wide variety of body shapes in natural human poses. The parameters of the model are learned from data including the rest pose template, blend weights, pose-dependent blend shapes, identity-dependent blend shapes, and a regressor from vertices to joint locations. Unlike previous models, the pose-dependent blend shapes are a linear function of the elements of the pose rotation matrices. This simple formulation enables training the entire model from a relatively large number of aligned 3D meshes of different people in different poses. We quantitatively evaluate variants of SMPL using linear or dual-quaternion blend skinning and show that both are more accurate than a Blend-SCAPE model trained on the same data. We also extend SMPL to realistically model dynamic soft-tissue deformations. Because it is based on blend skinning, SMPL is compatible with existing rendering engines and we make it available for research purposes.

There is a growing demand for flexible and soft electronic devices. In particular, stretchable, skin‐mountable, and wearable strain sensors are needed for several potential applications including personalized health‐monitoring, human motion detection, human‐machine interfaces, soft robotics, and so forth. This Feature Article presents recent advancements in the development of flexible and stretchable strain sensors. The article shows that highly stretchable strain sensors are successfully being developed by new mechanisms such as disconnection between overlapped nanomaterials, crack propagation in thin films, and tunneling effect, different from traditional strain sensing mechanisms. Strain sensing performances of recently reported strain sensors are comprehensively studied and discussed, showing that appropriate choice of composite structures as well as suitable interaction between functional nanomaterials and polymers are essential for the high performance strain sensing. Next, simulation results of piezoresistivity of stretchable strain sensors by computational models are reported. Finally, potential applications of flexible strain sensors are described. This survey reveals that flexible, skin‐mountable, and wearable strain sensors have potential in diverse applications while several grand challenges have to be still overcome.

Reinforcement learning offers to robotics a framework and set of tools for the design of sophisticated and hard-to-engineer behaviors. Conversely, the challenges of robotic problems provide both inspiration, impact, and validation for developments in reinforcement learning. The relationship between disciplines has sufficient promise to be likened to that between physics and mathematics. In this article, we attempt to strengthen the links between the two research communities by providing a survey of work in reinforcement learning for behavior generation in robots. We highlight both key challenges in robot reinforcement learning as well as notable successes. We discuss how contributions tamed the complexity of the domain and study the role of algorithms, representations, and prior knowledge in achieving these successes. As a result, a particular focus of our paper lies on the choice between model-based and model-free as well as between value-function-based and policy-search methods. By analyzing a simple problem in some detail we demonstrate how reinforcement learning approaches may be profitably applied, and we note throughout open questions and the tremendous potential for future research.

This article reviews both experimental and theoretical advances in the field of active matter which consists of natural and artificial objects capable of self-propulsion. Prime examples of active particles are Brownian particles, biological or manmade microscopic and nanoscopic objects, that can propel themselfes by taking up energy from their environment and converting it into directed motion. The review provides a guided tour through the basic principles and fabrication of active particles and discusses also many interesting future directions these manmade micromachines and nanomachines could take as autonomous agents for healthcare, sustainability, and security applications.

Human pose estimation has made significant progress during the last years. However current datasets are limited in their coverage of the overall pose estimation challenges. Still these serve as the common sources to evaluate, train and compare different models on. In this paper we introduce a novel benchmark "MPII Human Pose" that makes a significant advance in terms of diversity and difficulty, a contribution that we feel is required for future developments in human body models. This comprehensive dataset was collected using an established taxonomy of over 800 human activities [1]. The collected images cover a wider variety of human activities than previous datasets including various recreational, occupational and householding activities, and capture people from a wider range of viewpoints. We provide a rich set of labels including positions of body joints, full 3D torso and head orientation, occlusion labels for joints and body parts, and activity labels. For each image we provide adjacent video frames to facilitate the use of motion information. Given these rich annotations we perform a detailed analysis of leading human pose estimation approaches and gaining insights for the success and failures of these methods.

We propose a framework for analyzing and comparing distributions, which we use to construct statistical tests to determine if two samples are drawn from different distributions. Our test statistic is the largest difference in expectations over functions in the unit ball of a reproducing kernel Hilbert space (RKHS), and is called the maximum mean discrepancy (MMD).We present two distributionfree tests based on large deviation bounds for the MMD, and a third test based on the asymptotic distribution of this statistic. The MMD can be computed in quadratic time, although efficient linear time approximations are available. Our statistic is an instance of an integral probability metric, and various classical metrics on distributions are obtained when alternative function classes are used in place of an RKHS. We apply our two-sample tests to a variety of problems, including attribute matching for databases using the Hungarian marriage method, where they perform strongly. Excellent performance is also obtained when comparing distributions over graphs, for which these are the first such tests.

We describe Human Mesh Recovery (HMR), an end-to-end framework for reconstructing a full 3D mesh of a human body from a single RGB image. In contrast to most current methods that compute 2D or 3D joint locations, we produce a richer and more useful mesh representation that is parameterized by shape and 3D joint angles. The main objective is to minimize the reprojection loss of keypoints, which allows our model to be trained using in-the-wild images that only have ground truth 2D annotations. However, the reprojection loss alone is highly underconstrained. In this work we address this problem by introducing an adversary trained to tell whether human body shape and pose parameters are real or not using a large database of 3D human meshes. We show that HMR can be trained with and without using any paired 2D-to-3D supervision. We do not rely on intermediate 2D keypoint detections and infer 3D pose and shape parameters directly from image pixels. Our model runs in real-time given a bounding box containing the person. We demonstrate our approach on various images in-the-wild and out-perform previous optimization-based methods that output 3D meshes and show competitive results on tasks such as 3D joint location estimation and part segmentation.

The study of quantum phase transitions has been a major focus of theoretical and experimental work in systems of correlated electrons and in correlated ul-tracold atoms in recent years. However, some of the best characterized and understood examples of quantum phase transitions are found in magnetic

We present OctNet, a representation for deep learning with sparse 3D data. In contrast to existing models, our representation enables 3D convolutional networks which are both deep and high resolution. Towards this goal, we exploit the sparsity in the input data to hierarchically partition the space using a set of unbalanced octrees where each leaf node stores a pooled feature representation. This allows to focus memory allocation and computation to the relevant dense regions and enables deeper networks without compromising resolution. We demonstrate the utility of our OctNet representation by analyzing the impact of resolution on several 3D tasks including 3D object classification, orientation estimation and point cloud labeling.

We study the problem of object recognition for categories for which we have no training examples, a task also called zero--data or zero-shot learning. This situation has hardly been studied in computer vision research, even though it occurs frequently; the world contains tens of thousands of different object classes, and image collections have been formed and suitably annotated for only a few of them. To tackle the problem, we introduce attribute-based classification: Objects are identified based on a high-level description that is phrased in terms of semantic attributes, such as the object's color or shape. Because the identification of each such property transcends the specific learning task at hand, the attribute classifiers can be prelearned independently, for example, from existing image data sets unrelated to the current task. Afterward, new classes can be detected based on their attribute representation, without the need for a new training phase. In this paper, we also introduce a new data set, Animals with Attributes, of over 30,000 images of 50 animal classes, annotated with 85 semantic attributes. Extensive experiments on this and two more data sets show that attribute-based classification indeed is able to categorize images without access to any training images of the target classes.

Nonlinear dynamical systems have been used in many disciplines to model complex behaviors, including biological motor control, robotics, perception, economics, traffic prediction, and neuroscience. While often the unexpected emergent behavior of nonlinear systems is the focus of investigations, it is of equal importance to create goal-directed behavior (e.g., stable locomotion from a system of coupled oscillators under perceptual guidance). Modeling goal-directed behavior with nonlinear systems is, however, rather difficult due to the parameter sensitivity of these systems, their complex phase transitions in response to subtle parameter changes, and the difficulty of analyzing and predicting their long-term behavior; intuition and time-consuming parameter tuning play a major role. This letter presents and reviews dynamical movement primitives, a line of research for modeling attractor behaviors of autonomous nonlinear dynamical systems with the help of statistical learning techniques. The essence of our approach is to start with a simple dynamical system, such as a set of linear differential equations, and transform those into a weakly nonlinear system with prescribed attractor dynamics by means of a learnable autonomous forcing term. Both point attractors and limit cycle attractors of almost arbitrary complexity can be generated. We explain the design principle of our approach and evaluate its properties in several example applications in motor control and robotics.

We learn to compute opticalflow by combining a classical spatial-pyramid formulation with deep learning. This estimates large motions in a coarse-to-fine approach by warping one image of a pair at each pyramid level by the current flow estimate and computing an update to the flow. Instead of the standard minimization of an objective function at each pyramid level, we train one deep network per level to compute the flow update. Unlike the recent FlowNet approach, the networks do not need to deal with large motions; these are dealt with by the pyramid. This has several advantages. First, our Spatial Pyramid Network (SPyNet) is much simpler and 96% smaller than FlowNet in terms of model parameters. This makes it more efficient and appropriate for embedded applications. Second, since the flow at each pyramid level is small (<; 1 pixel), a convolutional approach applied to pairs of warped images is appropriate. Third, unlike FlowNet, the learned convolution filters appear similar to classical spatio-temporal filters, giving insight into the method and how to improve it. Our results are more accurate than FlowNet on most standard benchmarks, suggesting a new direction of combining classical flow methods with deep learning.

Image denoising can be described as the problem of mapping from a noisy image to a noise-free image. The best currently available denoising methods approximate this mapping with cleverly engineered algorithms. In this work we attempt to learn this mapping directly with a plain multi layer perceptron (MLP) applied to image patches. While this has been done before, we will show that by training on large image databases we are able to compete with the current state-of-the-art image denoising methods. Furthermore, our approach is easily adapted to less extensively studied types of noise (by merely exchanging the training data), for which we achieve excellent results as well.

This review comprises a detailed survey of ongoing methodologies for soft actuators, highlighting approaches suitable for nanometer- to centimeter-scale robotic applications. Soft robots present a special design challenge in that their actuation and sensing mechanisms are often highly integrated with the robot body and overall functionality. When less than a centimeter, they belong to an even more special subcategory of robots or devices, in that they often lack on-board power, sensing, computation, and control. Soft, active materials are particularly well suited for this task, with a wide range of stimulants and a number of impressive examples, demonstrating large deformations, high motion complexities, and varied multifunctionality. Recent research includes both the development of new materials and composites, as well as novel implementations leveraging the unique properties of soft materials.

We experimentally demonstrate a planar metamaterial analogue of electromagnetically induced transparency at optical frequencies. The structure consists of an optically bright dipole antenna and an optically dark quadrupole antenna, which are cut-out structures in a thin gold film. A pronounced coupling-induced reflectance peak is observed within a broad resonance spectrum. A metamaterial sensor based on these coupling effects is experimentally demonstrated and yields a sensitivity of 588 nm/RIU and a figure of merit of 3.8.

A ubiquitous observation in cell biology is that the diffusive motion of macromolecules and organelles is anomalous, and a description simply based on the conventional diffusion equation with diffusion constants measured in dilute solution fails. This is commonly attributed to macromolecular crowding in the interior of cells and in cellular membranes, summarizing their densely packed and heterogeneous structures. The most familiar phenomenon is a sublinear, power-law increase of the mean-square displacement (MSD) as a function of the lag time, but there are other manifestations like strongly reduced and time-dependent diffusion coefficients, persistent correlations in time, non-Gaussian distributions of spatial displacements, heterogeneous diffusion and a fraction of immobile particles. After a general introduction to the statistical description of slow, anomalous transport, we summarize some widely used theoretical models: Gaussian models like fractional Brownian motion and Langevin equations for visco-elastic media, the continuous-time random walk model, and the Lorentz model describing obstructed transport in a heterogeneous environment. Particular emphasis is put on the spatio-temporal properties of the transport in terms of two-point correlation functions, dynamic scaling behaviour, and how the models are distinguished by their propagators even if the MSDs are identical. Then, we review the theory underlying commonly applied experimental techniques in the presence of anomalous transport like single-particle tracking, fluorescence correlation spectroscopy (FCS) and fluorescence recovery after photobleaching (FRAP). We report on the large body of recent experimental evidence for anomalous transport in crowded biological media: in cyto- and nucleoplasm as well as in cellular membranes, complemented by in vitro experiments where a variety of model systems mimic physiological crowding conditions. Finally, computer simulations are discussed which play an important role in testing the theoretical models and corroborating the experimental findings. The review is completed by a synthesis of the theoretical and experimental progress identifying open questions for future investigation.

We study experimentally and numerically a (quasi-)two-dimensional colloidal suspension of self-propelled spherical particles. The particles are carbon-coated Janus particles, which are propelled due to diffusiophoresis in a near-critical water-lutidine mixture. At low densities, we find that the driving stabilizes small clusters. At higher densities, the suspension undergoes a phase separation into large clusters and a dilute gas phase. The same qualitative behavior is observed in simulations of a minimal model for repulsive self-propelled particles lacking any alignment interactions. The observed behavior is rationalized in terms of a dynamical instability due to the self-trapping of self-propelled particles.

To study the function behind the molecular arrangement of single integrins in cell adhesion, we designed a hexagonally close-packed rigid template of cell-adhesive gold nanodots coated with cyclic RGDfK peptide by using block-copolymer micelle nanolithography. The diameter of the adhesive dots is < 8 nm, which allows the binding of one integrin per dot. These dots are positioned with high precision at 28, 58, 73, and 85 nm spacing at interfaces. A separation of > or = 73 nm between the adhesive dots results in limited cell attachment and spreading, and dramatically reduces the formation of focal adhesion and actin stress fibers. We attribute these cellular responses to restricted integrin clustering rather than insufficient number of ligand molecules in the cell-matrix interface since "micro-nanopatterned" substrates consisting of alternating fields with dense and no nanodots do support cell adhesion. We propose that the range between 58-73 nm is a universal length scale for integrin clustering and activation, since these properties are shared by a variety of cultured cells.

Animals with widely varying body weight, such as flies, spiders, and geckos, can adhere to and move along vertical walls and even ceilings. This ability is caused by very efficient attachment mechanisms in which patterned surface structures interact with the profile of the substrate. An extensive microscopic study has shown a strong inverse scaling effect in these attachment devices. Whereas microm dimensions of the terminal elements of the setae are sufficient for flies and beetles, geckos must resort to sub-microm devices to ensure adhesion. This general trend is quantitatively explained by applying the principles of contact mechanics, according to which splitting up the contact into finer subcontacts increases adhesion. This principle is widely spread in design of natural adhesive systems and may also be transferred into practical applications.