While most deep learning approaches are developed for single images, in real world applications, images are often obtained as a series to inform decision making. Due to hardware (memory) and software (algorithm) limitations, few methods have been developed to integrate multiple images so far. In this study, we present an approach that seamlessly integrates deep learning and traditional machine learning models, to study multiple images and score joint damages in rheumatoid arthritis. This method allows the quantification of joining space narrowing to approach the clinical upper limit. Beyond predictive performance, we integrate the multilevel interconnections across joints and damage types into the machine learning model and reveal the cross-regulation map of joint damages in rheumatoid arthritis. Corresponding author(s) Email:   [email protected] or [email protected]

The cocktail party problem refers to a challenging process when the human sensory system tries to separate a specific voice from a loud mixture of background sound sources. The problem is much more demanding for machines and has become the holy grail in robotic hearing. Despite the many advances in noise suppression, the intrinsic information from the contaminated acoustic channel remains difficult to recover. Here we show a simple-yet-powerful laser-assisted audio system termed REAL (Robot Ear Accomplished by Laser) to probe the vibrations of sound-carrying surfaces (mask, throat and other nearby surfaces) in optical channel, which is intrinsically immune to acoustic background noises. Our results demonstrate that REAL can directly obtain the audio-frequency content from the laser without acoustic channel interference. The signals can be further transcribed into human-recognizable audio by exploiting the internal time and frequency correlations through memory-enabled neural networks. The REAL system would enable a new way in human-robot interaction. Xiaoping Hong  Email:    [email protected]

Two-dimensional metal-organic frameworks (2D-MOFs) have been extensively studied as promising materials in the fields of eletrocatalysis, drug delivery, electronic devicese, etc. However, few studies have explored the application potential of 2D-MOFs in novel neuromorphic computing devices. In this work, we report an optoelectronic neuromorphic transistor based on a 2D-MOFs/polymer charge-trapping layer. We found that, the large specific surface area, stable crystal structure, and highly accessible active sites in 2D-MOFs make them excellent charge-trapping materials for our devices, which are beneficial for mimicking the memory and learning functions observed in the organism's nervous systems. Different types of synaptic behaviors have been realized in our 2D-MOFs-based neuromorphic devices under stimuli signal, e.g., paired-pulse facilitation, excitatory post-synaptic current, short-term memory, and long-term memory. More interestingly, emotion-adjustable learning behavior was realized by changing the value of the source-drain voltage. This work can shed light on the application of 2D-MOFs in neuromorphic computing and will contribute to the further development of neuromorphic computing devices. Corresponding authors Email:  [email protected] (Jia Huang)                                                                [email protected] (Shilei Dai)

"AI & Drug Discovery" mode has significantly promoted drug development and achieved excellent performance, especially with the rapid development of deep learning, making remarkable contributions to protecting human physiological health. However, due to the "black-box" characteristic of the deep learning model, the decision route and predicted results in different research stages assisted by deep models are usually unexplainable, limiting their application in practice and more in-depth research of drug discovery. Focusing on the drug molecules, we propose an explainable fragment-based molecular property attribution technique for analyzing the influence of particular molecule fragments on properties and the relationship between the molecular properties in this paper. Quantitative experiments on 42 benchmark property tasks demonstrate that 325 attribution fragments, which account for 90% of the overall attribution results obtained by the proposed method, have positive relevance to the corresponding property tasks. More impressively, most of the attribution results randomly selected are consistent with the existing mechanism explanations. The discovery mentioned above provides a reference standard for assisting researchers in developing more specific and practical drug molecule studies, such as synthesizing molecular with the targeted property using a fragment obtained from the attribution method.Corresponding author(s) Email:   [email protected]  

The integration of an ingestible dosage form with sensing, actuation and drug delivery capabilities can enable a broad range of surgical-free diagnostic and treatment strategies. However, the gastrointestinal (GI) tract is a highly constrained and complex luminal construct that fundamentally limits the size of an ingestible system. Recent advancements in mesoscale magnetic crawlers have demonstrated the ability to effectively traverse complex and confined systems by leveraging magnetic fields to induce contraction and bending-based locomotion. However, the integration of functional components (e.g., electronics) in the proposed ingestible system remains fundamentally challenging. Here, we demonstrate the creation of a centralized compartment in a magnetic robot by imparting localized flexibility (MR-LF). The centralized compartment enables MR-LF to be readily integrated with modular functional components and payloads, such as commercial off-the-shelf electronics and medication, while preserving its bidirectionality in an ingestible form factor. We demonstrate the ability of MR-LF to incorporate electronics, perform drug delivery, guide continuum devices such as catheters, and navigate air-water environments in confined lumens. The MR-LF enables functional integration to create a highly-integrated ingestible system that can ultimately address a broad range of unmet clinical needs.  Keywords: Ingestible electronics; ingestible robots; soft robots; magnetic robots; magnetic crawlers; drug delivery.  Corresponding author email:   [email protected]    

The substantial increase in global population and climate change, among other factors have led to global food security and supply chain challenges. The United Nations has laid out an agenda to sustainably achieve zero hunger by 2030 as one of its sustainable development goals. However, sustainably achieving improved food yield has become a challenge as excessive use of fertilizers has also led to adverse environmental impact. To address the aforementioned challenges, WisDM Green, an artificial intelligence (AI)-based platform that aims to pinpoint and prioritize compound (e.g. biostimulants) combinations in peat moss, is harnessed to sustainably improve the yield of Amaranthus cruentus (red spinach). In this proof-of-concept study, from a pool of 8 compounds, WisDM Green-pinpointed combinations (6-Benzylaminopurine/Ethylenediaminetetraacetic Acid Iron (III) and Humic Acid/Seaweed Extract) achieve 26.34±15.80 and 33.59±14.60 increase in %Yield, respectively. The study also indicates that compound combinations may exhibit concentration-dependent synergies and thus, properly adjusting the concentration ratios of combinations may further improve plant yield in the context of sustainable farming. P. Wang and K. You contributed to this work equally.Corresponding author(s) Email:   [email protected], [email protected], [email protected] 

Deformability and self-adaptability are important for soft robots in order to deal with uncertain and varying situations and environments during movement and navigation. Droplet-based robots are great candidates to travel inside narrow and constrained spaces without damaging the interfaces due to their extreme deformability and liquid nature, which enables smooth contact between robots and target spaces. Here, we propose magnetic liquid metal droplet robots, comprising liquid metal and carbonyl iron, that can perform reversible telescopic deformation, bending, and on-demand locomotion. The magnetic liquid metal-based robots can perform on demand and reversible coalescence and splitting by intricately applying magnetic fields. Importantly, the liquid metal robot can perform phase transition to fix the desired shape after the programmable shape encoding. The liquid metal-based soft robots can serve as dynamic and recyclable switches for complex circuits, and are capable of repairing damaged sections of microcircuits by remote actuation, controllable coalescence, and on-demand circuit welding. The technology provides a new application scenario of droplet-based soft robots for on-demand circuit welding and transient recyclable electronics. Corresponding author(s) Email:  X.Z.: [email protected]; L.Z.: [email protected]; B.W.: [email protected]

Human receives and transmits various information from the outside world through different sensory systems. The sensory neurons integrate various sensory inputs into a synthetical perception to monitor complex environments, and this fundamentally determines the way how we perceive the world. Developing multifunctional artificial sensory elements that can integrate multisensory perception plays a vital role in future intelligent perception systems, whereas prior spiking neurons reported can only handle single-mode physical signals. Here, we present a bio-inspired haptic-temperature fusion spiking neuron based upon a serial connection of piezoresistive sensor and VO2 volatile memristor. The artificial sensory neuron is capable of detecting and encoding pressure and temperature inputs based on the voltage dividing effect and the intrinsic thermal sensitivity of metal-insulator transition in VO2. Recognition of Braille characters is achieved through multiple piezoresistive sensors, taking advantage of the spatial integration capabilities of such spiking neurons. Notably, the traditionally separate haptic and temperature signals can be fused physically in the sensory neuron when synchronizing the two sensory cues, which is able to recognize multimodal haptic/temperature patterns. The artificial multisensory neuron thus provides a promising approach towards e-skin, neuro-robotics and human-machine interaction technologies.

Developing on-site biomarker enrichment platforms could help to improve the diagnosis of gastrointestinal (GI) tract diseases at early stages.  Medical procedures, such as colonoscopies and imaging techniques, are used to diagnose disease, but are not easily accessible for repeat measurements. In the other hand, liquid biopsies, e.g., blood, urine, or fecal samples, have become important sampling strategies to identify health concerns.  Herein, a robotic pill is designed for collecting relevant biomarkers from the GI over prolonged sampling periods. The robotic pill comprises a magnetic core for locomotion, a delayed gate mechanism that controls sampling location based on changes in its environment, and an enrichment module that traps biomarkers in an absorbent matrix while enabling biofluid to pass through the chamber. The robotic pill was assessed to sample microparticles, proteins, and bacteria from solution. Moreover, the robotic pill was capable of directed locomotion in complex environments and docking in a targeted region against fluid flow. Utilization of an untethered robotic sampling system could provide a tool to investigate aspects of disease initiation and progression for early diagnosis and therapy monitoring.

Avi Hazan1, Elishai Ezra Tsur1*1 Neuro-Biomorphic Engineering Lab, Department of Mathematics and Computer Science, The Open University of Israel, Ra’anana, Israel* Correspondence: [email protected] hardware designs realize neural principles in electronics to provide high-performing, energy-efficient frameworks for machine learning. Here, we propose a neuromorphic analog design for continuous real-time learning. Our hardware design realizes the underlying principles of the neural engineering framework (NEF). NEF brings forth a theoretical framework for the representation and transformation of mathematical constructs with spiking neurons. Thus, providing efficient means for neuromorphic machine learning and the design of intricate dynamical systems. Our analog circuit design implements the neuromorphic prescribed error sensitivity (PES) learning rule with OZ neurons. OZ is an analog implementation of a spiking neuron, which was shown to have complete correspondence with NEF across firing rates, encoding vectors, and intercepts. We demonstrate PES-based neuromorphic representation of mathematical constructs with varying neuron configurations, the transformation of mathematical constructs, and the construction of a dynamical system with the design of an inducible leaky oscillator. We further designed a circuit emulator, allowing the evaluation of our electrical designs on a large scale. We used the circuit emulator in conjunction with a robot simulator to demonstrate adaptive learning-based control of a robotic arm with six degrees of freedom.

A prominent problem in computer vision is occlusion, which occurs when an object’s key features temporarily disappear behind another crossing body, causing the computer to struggle with image detection. While the human brain is capable of compensating for the invisible parts of the blocked object, computers lack such scene interpretation skills. Cloud computing using convolutional neural networks is typically the method of choice for handling such a scenario. However, for mobile applications where energy consumption and computational costs are critical, cloud computing should be minimized. In this regard, we propose a computer vision sensor capable of efficiently detecting and tracking covered objects without heavy reliance on occlusion handling software. Our edge-computing sensor accomplishes this task by self-learning the object prior to the moment of occlusion and uses this information to “reconstruct” the blocked invisible features. Furthermore, the sensor is capable of tracking a moving object by predicting the path it will most likely take while travelling out of sight behind an obstructing body. Finally, sensor operation is demonstrated by exposing the device to various simulated occlusion events. Keywords:  Computer vision, occlusion handling, edge computing, object tracking, dye sensitized solar cell. Corresponding author Email: [email protected]  

Many species can dynamically alter their skin textures to enhance their motility and survivability. Despite the enormous efforts on designing bio-inspired materials with tunable surface textures, developing spatiotemporally programmable and reconfigurable textural morphing without complex control remains challenging. Here we propose a design strategy to achieve metasurfaces with such properties. The metasurfaces comprise an array of unit cells with broadly tailored temporal responses. By arranging the unit cells differently, the metasurfaces can exhibit various spatiotemporal responses, which can be easily reconfigured by disassembling and rearranging the unit cells. Specifically, we adopt viscoelastic shells as the unit cells, which can be pneumatically actuated to a concave state, and recover the initial convex state some time after the load is removed. We computationally and experimentally show that the recovery time can be widely tuned by the geometry and material viscoelasticity of the shells. By assembling such shells with different recovery time, we build metasurfaces with pre-programmed spatiotemporal textural morphing under simple pneumatic actuation, and demonstrate temporal evolution of patterns, such as digit numbers and emoji, and spatiotemporal control of friction. This work opens up new avenues in designing spatiotemporal morphing metasurfaces that could be employed for programming mechanical, optical and electrical properties. Corresponding author: Lihua Jin, Email:   [email protected]  

Being minimally invasive and highly effective, radiofrequency ablation (RFA) is widely used for small size malignant tumors treatment. However, in clinical practice, a large number of tumors are found in irregular shape, while the current RFA devices are hard to control their morphologic appearance of RFA lesions on demand, which usually ends up with unnecessarily excessive tissue ablation and subsequently often brings irreversible damage to the organs’ functions. Here, we introduce active cannulas for each of individually-controlled sub-electrodes to achieve an on-demand shape morphing and thus conformal RFA lesion. The target shape as well as the length of inserted sub-electrodes can be precisely controlled by tuning the active stylets and cannulas. What’s more, owing to independent movement and energy control of each sub-electrodes, our electrode is shown to be not only efficient enough to accomplish accurate trajectory control to target tissue in a single insertion, but also adaptive enough to ablate target tissues with diverse morphologic appearances and locations. On-demand conformal ablation of target tissue is demonstrated as well under the guidance of ultrasound imaging with our device. Potentially, our RFA electrode is a promising minimally invasive treatments of malignant tumors in the future clinical practice. Corresponding author(s) Email:    [email protected]  

Magnetic miniature robots (MMRs) are mobile actuators that can exploit their size to non-invasively access highly confined, enclosed spaces. By leveraging on such unique abilities, MMRs have great prospects to transform robotics, biomedicine and materials science. As having high dexterity is critical for MMRs to enable their targeted applications, existing MMRs have developed numerous soft-bodied gaits to locomote in various environments. However, there exist two critical limitations that have severely restricted their dexterity: (i) MMRs capable of multimodal soft-bodied locomotion have only demonstrated five-degrees-of-freedom (five-DOF) motions because the sixth-DOF rotation about their net magnetic moment axis is uncontrollable; (ii) six-DOF MMRs have only realized one mode of soft-bodied, swimming locomotion. Here we propose a six-DOF MMR that can execute seven modes of soft-bodied locomotion and perform 3-dimensional pick-and-place operations. By optimizing its harmonic magnetization profile, our MMR can produce 1.41-63.9 folds larger sixth-DOF torque than existing MMRs with similar profiles, without compromising their traditional five-DOF actuation capabilities. The proposed MMR demonstrated unprecedented dexterity; it could jump through narrow slots to reach higher grounds; use precise orientation control to roll, two-anchor crawl and swim across tight openings with strict shape constraints; perform undulating crawling across three different planes in convoluted channels. Keywords: Magnetic materials; soft actuators; miniature robots; locomotion. Corresponding author(s) Email:   [email protected]  

The origami technique realizes unique mechanical properties of sheet materials without additional parts. In this study, a self-folded corrugated structure (SCS) is developed based on the reinforcing properties of the origami technique. The corrugated structures are employed as the core material for a high-strength, open-channel sandwich structure. Research on self-folded core materials is scarce; thus, a design concept is proposed, and the mechanical properties of the SCS are evaluated. First, the structural parameters of the SCS fabricated by changing the printing parameters (e.g., linewidth and number of lines/creases), to derive the structural model are determined. The model facilitates the design of an SCS with the desired structure. Thereafter, the mechanical properties of the SCSs are evaluated by conducting three-point bending tests to determine the essential design parameters corresponding to high stiffness. Moreover, SCSs can be stacked without occupying space, thus leading to improved strength. These SCSs fabricated using self-folding paper by ink-jet printing are low cost and eco-friendly. Moreover, they are specialized for rapid design and fabrication, depending on the application. This paper proposes the use of SCS as a novel smart core because it exhibits a high transportation efficiency and stiffness without additional components. Corresponding author(s) Email: [email protected] , [email protected]

Shape memory Nitinol has long been used for actuation. However, utilizing Nitinol to fabricate novel devices for various applications is a challenge, but has shown incredible promise and impacts. Bistable metal strips are widely adopted for shape morphing purposes (primarily in kid’s toys, e.g., snap bracelets) due to their easy and robust transformation between two states. In this paper, we combine Nitinol shape memory alloy and bistable metal strip to fabricate a swimming actuator with both slow moving and fast snapping capability, akin to an octopus swimming slowly in water, but quickly moving upon encountering a threat. The actuator developed here can also swim in multiple directions, all controlled by a wireless module. Furthermore, we demonstrate that an on-board sensor can be incorporated for potential environmental monitoring applications. Taken together, along with the fact that the device developed here has no mechanical parts, makes this  an interesting potential alternative to more expensive, and energy consuming boats.

Memristive devices being applied in neuromorphic computing are envisioned to significantly improve the power consumption and speed of future computing platforms. The materials used to fabricate such devices will play a significant role in their viability. Graphene is a promising material, with superb electrical properties and the ability to be produced sustainably. In this paper, we demonstrate that a fabricated graphene-pentacene memristive device can be used as synapses within Spiking Neural Networks (SNNs) to realise Spike Timing Dependent Plasticity (STDP) for unsupervised learning in an efficient manner. Specifically, we verify operation of two SNN architectures tasked for single digit (0-9) classification: (i) a simple single-layer network, where inputs are presented in 5x5 pixel resolution, and (ii) a larger network capable of classifying the Modified National Institute of Standards and Technology (MNIST) dataset, where inputs are presented in 28x28 pixel resolution. Final results demonstrate that for 100 output neurons, after one training epoch, a test set accuracy of up to 86% can be achieved, which is higher than prior art using the same number of output neurons. We attribute this performance improvement to homeostatic plasticity dynamics that we used to alter the threshold of neurons during training. Our work presents the first investigation of the use of green-fabricated graphene memristive devices to perform a complex pattern classification task. This can pave the way for future research in using graphene devices with memristive capabilities in neuromorphic computing architectures. In favour of reproducible research, we make our code and data publicly available https://anonymous.4open.science/r/c69ab2e2-b672-4ebd-b266-987ee1fd65e7.

Soft fluidic actuators produce continuous and life-like motions that are intrinsically safe, but current designs are not yet mature enough to enable large deployment with high force and low-cost fabrication methods. Here, soft fluidic actuators with two superimposed origami architectures are reported. Driven by a fluid input, the presented dual-origami soft actuators produce quasi-sequential deployment and bending motion that is guided by unsymmetric unfolding of low-stretchable origami components. The dominance between the deployment and bending can be shifted by varying the unfolding behavior, enabling pre-programming of the motion. The proposed origami-inspired soft actuators are directly fabricated by low-cost fused deposition modeling 3D-printing, and subjected to a heat treatment post-processing to enhance the fluid sealing performance. Finally, soft gripper applications are presented and they successfully demonstrate gripping tasks that each requires strength, delicacy, precision and dexterity. The dual-origami approach offers a design guidance for soft robots to embody grow-and-retract motion with a small initial form factor, promising for applications in next-generation soft robotic systems.