Natural and robust locomotion with RL

Since our previous control algorithms were only focused on external task performance, the resulting behaviors looked quite unnatural. In this project, we focused on biologically plausible objectives in combination with biomechanically accurate simulators to achieve natural and robust human walking. We specifically do not use demonstrations during training, which makes our learning approach more flexible and robust than other works.

Open-Source Development MyoSuite

Since 2023, I have been collaborating with the MyoSuite development team and act as an open-source developer for the task suite.

Organization of the MyoChallenge 2023

In 2023, I joined the organization committee of the MyoChallenge and am heavily involved in the design of the locomotion track as well as the submission system and the creation of PR material. I further provide two pre-trained RL baselines for the unsolved challenge tasks: relocate and chasetag. Participants may improve upon the baselines during the competition, or just use them to get started.

Open-Source Development Sconegym

Since 2022, I have been collaborating with the company GOATSTREAM in order to create reinforcement learning environments with biomechanical models for my research. The Python RL environments as well as the Hyfydy models will be open-sourced in 2023.

Participation in the MyoChallenge 2022

In 2022, I participated in the MyoChallenge 2022, a NeurIPS competition. The competition involved developing control algorithms for two different tasks with a musculoskeletal hand. In the baoding-balls track, the task was to continuously rotate two balls in one hand, while the reorient-die track involved matching the target position and orientation of a cube. I achieved 2nd and 3rd place in the ball and die track respectively and received the “Honorable Mention” award.

High-dimensional control with RL

Common reinforcement learning algorithms are not well suited for high-dimensional and strongly overactuated control tasks. We propose to use DEP (Differential Extrinsic Plasticity), an algorithm from the domain of self-organization to improve exploration in this regime and acquire robust control algorithms.

Leveraging muscular properties in robotics

We implemented simple muscle models in MuJoCo tasks in order to investigate the effect of muscular properties on robotics tasks with RL. We found that learning algorithms benefit from better robustness, improved learning speed and less sensitivity to hyperparameter changes when muscular properties are present, compared to torque and PD controllers. This project was done jointly with Isabell Wochner who performed these experiments with MPC and the demoa simulator and found similar results.

I am currently working with a Master student to bring these properties to real robotic hardware.