-
January 2024: My research team has had three papers accepted for presentation at the 2024 International Conference on Robotics and Automation (ICRA 2024). A big thank you to my students and collaborators!
-
October 2023: I have successfully integrated my In-box Grasp Pose Prediction Pipline on ABB Yumi Mobile manipulator for the Harmony Project. The video is available online Here and the code is available online Here.
-
September 2023: We proposed TiV-ODE: A Neural ODE-based Approach for Controllable Video Generation From Text-Image Pairs . The paper has been accepted for presentation in ICRA 2024 and is available online Here.
-
September 2023: We proposed a framework for Harnessing the Synergy between Pushing, Grasping, and Throwing to Enhance Object Manipulation in Cluttered Scenarios . The paper has been accepted for presentation in ICRA 2024.
-
September 2023: We proposed Robust and Dexterous Dual-arm Tele-Cooperation using Adaptable Impedance Control . The paper has been accepted for presentation in ICRA 2024.
-
September 2023: We proposed NODE-IL: A Data-Efficient Imitation Learning Framework for Long-Horizon Multi-Skills Robot Manipulation . The paper is under review.
-
August 2023: Our paper entitled A Data-efficient Neural ODE Framework for Optimal Control of Soft Manipulators got accepted for presentation at (CoRL 2023). The paper is available online Here.
-
August 2023: Our paper entitled Language-guided Robot Grasping: CLIP-based Referring Grasp Synthesis in Clutter got accepted for presentation at (CoRL 2023). The paper is available online Here.
-
July 2023: We proposed Learning Fine Pinch-Grasp Skills using Tactile Sensing from Real Demonstration Data. The paper is available online Here.
-
June 2023: Our poster, A Novel Extensible Flexible Arm for Robotic got the best poster award in Soft Growing Robots: From Search-and-Rescue to Intraluminal Interventions in (ICRA 2023).
-
March 2023: Our paper entitled Learning Hybrid Locomotion Skills -Learn to Exploit Residual Actions and Modulate Model-based Gait Control got accepted to Frontiers Robotics and AI!
-
February 2023: Mohammadreza Kasaei gave a talk at IPAB workshop in the University of Edinburgh, UK entitled Data-efficient Non-parametric Modelling and Control of an Extensible Soft Manipulator.
-
January 2023: My research team has had four papers accepted for presentation at the 2023 International Conference on Robotics and Automation (ICRA 2023).
-
November 2022:: Mohammadreza Kasaei gave an invited talk at the University of Aveiro, Portugal in a Seminar in Robotics and Intelligent Systems on Robotics for Society: Recent Research in Locomotion and Manipulation.
-
October 2022: Our paper entitled MVGrasp: Real-Time Multi-View 3D Object Grasping in Highly Cluttered Environments got accepted to Robotics and Autonomous Systems (RAS)!!
-
September 2022: We proposed Instance-wise Grasp Synthesis for Robotic Grasping . The paper has been accepted at (ICRA 2023).
-
September 2022: We proposed Agile and Versatile Robot Locomotion via Kernel-based Residual Learning . The paper has been accepted at (ICRA 2023), the video is available online Here.
-
September 2022: We proposed Data-efficient Non-parametric Modelling and Control of an Extensible Soft Manipulator . The paper has been accepted at (ICRA 2023) and the video is available online Here.
-
September 2022: We proposed Throwing Objects into A Moving Basket While Avoiding Obstacles . This paper has been accepted at (ICRA 2023). The paper is available online Here and the video can be watched at Here
-
July 2022: We proposed Design, Modelling, and Control of an Extensible
Flexible Finger for Robotic Manipulation. The paper is under review but the video is available online Here.
-
July 2022: We proposed MVGrasp: Real-Time Multi-View 3D Object Grasping in Highly Cluttered Environments. The paper is available online Here.
-
July 2022: The FCPortugal, a 3D soccer simulation team, a partnership between the universities of Aveiro and Porto, became world champion in the 3D Simulation League, in the RoboCup 2022, held in Thailand. In this league, teams of 11 fully autonomous, simulated humanoid robots play football against each other. It has been my pleasure to contribute to developing walk engines that are agile and robust (final video is available online Here and to read more detail please click Here).
|
|
|
+ Harnessing the Synergy between Pushing, Grasping, and Throwing to Enhance Object Manipulation in Cluttered Scenarios:
In this work, we delve into the intricate synergy among non-prehensile actions like pushing, and prehensile
actions such as grasping and throwing, within the domain of
robotic manipulation. We introduce an innovative approach
to learning these synergies by leveraging model-free deep
reinforcement learning. The robot’s workflow involves detecting the pose of the target object and the basket at each time step, predicting the optimal push configuration to isolate the target object, determining the appropriate grasp configuration, and inferring the necessary parameters for an accurate throw into the basket. This empowers robots to skillfully reconfigure cluttered scenarios through pushing, creating space for collision-free grasping actions. Simultaneously, we integrate throwing behavior, showcasing how this action significantly extends the robot’s
operational reach.
Video
Paper (ICRA)
|
|
+ Language-guided Robot Grasping: CLIP-based Referring Grasp Synthesis in Clutter :
Robots operating in human-centric environments require the integration of visual grounding and grasping capabilities to effectively manipulate objects based on user instructions. This work focuses on the task of referring grasp synthesis, which predicts a grasp pose for an object referred through natural language in cluttered scenes. Existing approaches often employ multi-stage pipelines that first segment the referred object and then propose a suitable grasp, and are evaluated in simple datasets or simulators that do not capture the complexity of natural indoor scenes. To address these limitations, we develop a challenging benchmark based on cluttered indoor scenes from OCID dataset, for which we generate referring expressions and connect them with 4-DoF grasp poses. Further, we propose a novel end-to-end model (CROG) that leverages the visual grounding capabilities of CLIP to learn grasp synthesis directly from image-text pairs. Our results show that vanilla integration of CLIP with pretrained models transfers poorly in our challenging benchmark, while CROG achieves significant improvements both in terms of grounding and grasping.
Extensive robot experiments in both simulation and hardware demonstrate the effectiveness of our approach in challenging interactive object grasping scenarios that include clutter.
Video
Paper (CoRL)
Code
|
|
+ A Data-efficient Neural ODE Framework for Optimal Control of Soft Manipulators:
This work introduces a novel approach for modeling continuous forward kinematic models of soft continuum robots by employing Augmented Neural ODE (ANODE), a cutting-edge family of deep neural network models. To the best of our knowledge, this is the first application of ANODE in modeling soft continuum robots. This formulation introduces auxiliary dimensions, allowing the system's states to evolve in the augmented space which provides a richer set of dynamics that the model can learn, increasing the flexibility and accuracy of the model. Our methodology achieves exceptional sample efficiency, training the continuous forward kinematic model using only 25 scattered data points.
Additionally, we design and implement a fully parallel Model Predictive Path Integral~(MPPI)-based controller running on a GPU, which efficiently manages a non-convex objective function. Through a set of experiments, we showed that the proposed framework (ANODE+MPPI) significantly outperforms state-of-the-art learning based methods such as FNN and RNN in unseen-before scenarios and marginally outperforms them in seen-before scenarios.
Video
Paper (CoRL)
Code
|
|
+ Robust and Dexterous Dual-arm Tele-Cooperation using Adaptable Impedance Control:
This work presents a novel extension to dual-arm tele-cooperation, leveraging the non-linear stiffness and passivity of Fractal Impedance Control (FIC) to adapt to diverse cooperative scenarios. Unlike traditional impedance controllers, our approach ensures stability without relying on energy tanks, as demonstrated in our prior research. In this work, we further extend the FIC framework to bimanual operations, allowing for stable and smooth switching between different dynamic tasks without gain tuning. We also introduce a telemanipulation architecture that offers higher transparency and dexterity, addressing the challenges of signal latency and low-bandwidth communication.
Video
Paper (ICRA)
|
|
+ MVGrasp: Real-Time Multi-View 3D Object Grasping in Highly Cluttered Environments:
In this work, we propose a multi-view deep learning approach to handle robust object grasping in human-centric domains. In particular, our approach takes a point cloud of an arbitrary object as an input, and then, generates orthographic views of the given object. The obtained views are finally used to estimate pixel-wise grasp synthesis for each object. We train the model end-to-end using a synthetic object grasp dataset and test it on both simulation and real-world data without any further fine-tuning.
Video
Paper (RAS)
|
|
+ TiV-ODE: A Neural ODE-based Approach for Controllable Video Generation From Text-Image Pairs:
The work introduces a novel framework, named TiV-ODE, which enables the generation of highly controllable videos using a static image and text caption. The framework relies on the Neural Ordinary Differential Equations (Neural ODEs) to represent complex underlying dynamical systems through a set of nonlinear ordinary differential equations. The proposed method generates videos that possess both the desired dynamics and content. The experimental results demonstrate the effectiveness of the approach in producing visually consistent and highly controllable videos while modeling dynamical systems. This work represents a significant advancement towards developing advanced models for generating controllable videos that can handle complex and dynamic scenes.
Paper (ICRA)
|
|
+ Throwing Objects into A Moving Basket While Avoiding Obstacles:
In this work, we tackle object throwing problem through a deep reinforcement learning approach that enables robots to precisely throw objects into moving baskets while there are obstacles obstructing the path. To the best of our knowledge, we are the first group that addresses throwing
objects with obstacle avoidance. Such a throwing skill not only increases the physical reachability of a robot arm but also improves the execution time.
Video
Paper (ICRA)
|
|
+ Learning Fine Pinch-Grasp Skills using Tactile Sensing from Real Demonstration Data:
This work develops a data-efficient learning from demonstration framework which exploits the use of rich tactile sensing and achieves fine dexterous bimanual manipulation. Specifically, we formulated a convolutional autoencoder network that can effectively extract and encode high-dimensional tactile information. Further, we developed a behaviour cloning network that can learn human-like sensorimotor skills demonstrated directly on the robot hardware in the task space by fusing both proprioceptive and tactile feedback. Our comparison study with the baseline method revealed the effectiveness of the contact information, which enabled successful extraction and replication of the demonstrated motor skills.
Video
Paper (Under review)
|
|
+ Design, Modelling, and Control of an Extensible Flexible Finger for Robotic Manipulation:
This work presents a novel cable-driven soft robot capable of flexing in 3D space with an additional degree of freedom for extension and retraction. In comparison with non-extensible soft robots, the proposed robot provides a larger workspace to reach 3D targeted points. We detail the robot design and prototyping, develop a novel mathematical model for predicting the robot's motion, and employ the model to control the robot that can autonomously follow time-varying trajectories.
Video
|
|
+ Data-efficient Non-parametric Modelling and Control of an Extensible Soft Manipulator:
This work proposed a novel data-efficient and non-parametric approach to develop a continuous model using a small dataset of real robot demonstrations (only 25 points). To the best of our knowledge, the proposed approach is the most sample-efficient method for soft continuum robot. Furthermore, we employed this model to develop a controller to track arbitrary trajectories in the feasible kinematic space.
Video
Paper (ICRA)
|
|
+ Agile and Versatile Robot Locomotion via Kernel-based Residual Learning:
This work developed a kernel-based residual learning framework for quadrupedal robotic locomotion. Initially, a kernel neural network is trained with data collected from an MPC controller. Alongside a frozen kernel network, a residual controller network is trained via reinforcement learning to acquire generalized locomotion skills and resilience against external perturbations. With this proposed framework, a robust quadrupedal locomotion controller is learned with high sample efficiency and controllability, providing omnidirectional locomotion at continuous velocities. Its versatility and robustness are validated on unseen terrains that the expert MPC controller fails to traverse. Furthermore, the learned kernel can produce a range of functional locomotion behaviors and can generalize to unseen gaits.
Video
Paper (ICRA)
|
|
+ Instance-wise Grasp Synthesis for Robotic Grasping:
Generating high-quality instance-wise grasp configurations provides critical information of how to grasp specific objects in a multi-object environment and is of high importance for robot manipulation tasks. This work proposed a novel Single-Stage Grasp (SSG) synthesis network, which performs high-quality instance-wise grasp synthesis in a single stage: instance mask and grasp configurations are generated for each object simultaneously. Our method outperforms state-of-the-art on robotic grasp prediction based on the OCID-Grasp dataset, and performs competitively on the JACQUARD dataset. The benchmarking results showed significant improvements compared to the baseline on the accuracy of generated grasp configurations.
Video
Paper (ICRA)
Code
|
|
+ FC Portugal: RoboCup 2022 3D Simulation League and Technical Challenge Champions:
FC Portugal, a team from the universities of Porto and Aveiro, won the main competition of the **2022 RoboCup 3D Simulation League**, with 17 wins, 1 tie and no losses. During the course of the competition, the team scored 84 goals while conceding only 2. FC Portugal also won the 2022 RoboCup 3D Simulation League Technical Challenge, accumulating the maximum amount of points by ending first in its both events: the Free/Scientific Challenge, and the Fat Proxy Challenge. The team presented in this year’s competition was rebuilt from the ground up since the last RoboCup. No previous code was used or adapted, with the exception of the 6D pose estimation algorithm, and the get-up behaviors, which were re-optimized. In comparison with our previous team, the omnidirectional walk is more stable and went from 0.70 m/s to 0.90 m/s, the long kick from 15 m to 19 m, and the new close-control dribble reaches up to 1.41 m/s.
Video
Paper TDP
|
|
+ Learning Hybrid Locomotion Skills – Learn to Exploit Residual Dynamics and Modulate Model-based Gait Control:
This work aims to combine machine learning and control approaches for legged robots, and developed a hybrid framework to achieve new capabilities of balancing against external perturbations. The framework embeds a kernel which is a fully parametric closed-loop gait generator based on analytical control. On top of that, a neural network with symmetric partial data augmentation learns to automatically adjust the parameters for the gait kernel and to generate compensatory actions for all joints as the residual dynamics, thus significantly augmenting the stability under unexpected perturbations.
Video
Paper (Frontiers Robotics)
|