Teaching

Humanoid Robotics (RO5300)

[SS2018] During the summer semester Prof. Dr. Elmar Rueckert is teaching the course Humanoid Robotics (RO5300). In this course he discusses the key components of one of the most complex autonomous systems. These topics are

  1. Kinematics, Dynamics & Locomotion
  2. Representations of Skills & Imitation Learning
  3. Feedback Control, Priorities & Torque Control
  4. Reinforcement Learning & Policy Search
  5. Sensor Integration & Fusion
  6. Cognitive Reasoning & Planning

Here is the current list of accumulated bonus points.

Ask questions at our course related Q&A page here.

The exam will be at July 17th, 2018 in C4-S03 (where the lectures take place). Fill out the exam registration form. The second exam date is on Sept. 27th, 2018 10 am – 12 am in our seminar room Nr.64.

Introduction slides to the course on Humanoid Robotics by Prof. Dr. Elmar Rueckert, University of LuebeckThis course provides a unique overview over central topics in robotics. A particular focus is put in the dependencies and interaction among the components in the control loop illustrated in the image above. These interactions are discussed in the context of state of the art methods including dynamical systems movement primitives, gradient based policy search methods or probabilisitic inference for planning algorithms.

In sum,  the lecture provides a structured and well motivated overview over modern techniques and tools which enable the students to define reward functions, implement robot controller and interaction software and to apply and extend state of the art reinforcement learning and planning approaches.

No special knowledge is required beforehand. All concepts and theories will be developed during the lectures or the tutorials.

The students will also experiment with state of the art machine learning methods and robotic simulation tools in accompanying exercises. Hands on tutorials on programming with Matlab and the simulation tool V-Rep complement the course content.

Follow this link to register for the course: https://moodle.uni-luebeck.de/course/view.php?id=3261.

Location: Room: C4-S03

Course materials (tentative slides, last update Feb. 27, 2018)

  1. Humanoid Robotics Intro (L1: April, 10th 2018)
  2. Kinematics, Dynamics & Locomotion (L2: April, 17th 2018, L3: April, 24th 2018)
  3. Representations of Skills & Imitation Learning (L4: May 8th 2018, L5: May 15th 2018)
  4. Feedback Control, Priorities & Torque Control (L6, L7)
  5. Reinforcement Learning & Policy Search(L8, L9)
  6. Sensor Integration & Fusion (L10, L11)
  7. Cognitive Reasoning & Planning (L12, L13)

Additionally, we will link recordings of the drawings and derivations that were created during the lectures and the tutorials with a tablet and a pen.

Materials for the exercise

For simulating robot manipulation tasks we will use the simulator V-REP. For research and for teaching a free eduction version can be found here. To experiment with state of the art robot control and learning methods Mathworks’ MATLAB will be used. If you do not have it installed yet, please follow the instructions of our IT-Service Center.

  1. Introduction to Matlab and V-Rep (Exercise, Solution)
    Tutorial: April, 18th 2018 in room C4-S03.
  2. Kinematics and Dynamics Assignment (Exercise/Assignment, Solution, Material, Latex Draft)
    Tutorial: May, 2nd 2018, Submission: May 22nd, 2018 at 10 am. 
  3. Movement Representations (Exercise/Assignment, Solution, Material, Data)
    Tutorial: May, 16th 2018, Submission: June 05th, 2018 at 10 am.
    Videos to the exercise

Matlab Files shown during the Tutorial can be found here.

Probabilistic Learning for Robotics (tentative title)

[WS2018/19] In the winter semester, I will teach a course on Probabilistic Learning for Robotics which covers advanced topics including graphical models, factor graphs, probabilistic inference for decision making and planning, and computational models for inference in neuroscience. The content is not yet fixed and may change.

In accompanying exercises and hands on tutorials the students will experiment with state of the art machine learning methods and robotic simulation tools. In particular, Mathworks’ MATLAB,  the robot middleware ROS and the simulation tool V-Rep will be used.

Student Theses

2018

Dittmar, Denny

Distributed Reinforcement Learning with Neural Networks for Robotics (Technical Report)

Technische Universität Darmstadt M.Sc. Thesis, 2018.

(BibTeX | Tags: Reinforcement Learning, RNN, spiking | Links: )

Distributed Reinforcement Learning with Neural Networks for Robotics

2017

Frisch, Yannik

The Effects of Intrinsic Motivation Signals on Reinforcement Learning Strategies (Technical Report)

Technische Universität Darmstadt B.Sc. Thesis, 2017.

(BibTeX | Tags: intrinsic motivation, Reinforcement Learning | Links: )

The Effects of Intrinsic Motivation Signals on Reinforcement Learning Strategies

Pfanschilling, Viktor

Self-Programming Mutation and Crossover in Genetic Programming for Code Generation (Technical Report)

B.Sc. Thesis, 2017.

(BibTeX | Tags: model learning | Links: )

Self-Programming Mutation and Crossover in Genetic Programming for Code Generation

Thiem, Simon-Konstantin

Simulation of the underactuated Sake Robotics Gripper in V-REP and ROS (Technical Report)

Technische Universität Darmstadt B.Sc. Thesis, 2017.

(BibTeX | Tags: Manipulation, Simulation | Links: )

Simulation of the underactuated Sake Robotics Gripper in V-REP and ROS

Nakatenus, Moritz

LSTM Networks for movement planning in humanoids (Technical Report)

Technische Universität Darmstadt M.Sc. Project, 2017.

(BibTeX | Tags: inverse dynamics, RNN)

LSTM Networks for movement planning in humanoids

Gondaliya, Kaushik

Learning to Categorize Issues in Distributed Bug Tracker Systems (Technical Report)

Technische Universität Darmstadt M.Sc. Thesis, 2017.

(BibTeX | Tags: Natural Language Processing, RNN)

Learning to Categorize Issues in Distributed Bug Tracker Systems

Polat, Harun

Nonparametric deep neural networks for movement planning (Technical Report)

Technische Universität Darmstadt B.Sc. Thesis, 2017.

(BibTeX | Tags: model learning, RNN, spiking | Links: )

Nonparametric deep neural networks for movement planning

Plage, Lena

Reinforcement Learning for tactile-based finger gaiting (Technical Report)

Technische Universität Darmstadt B.Sc. Thesis, 2017.

(BibTeX | Tags: Manipulation, Reinforcement Learning | Links: )

Reinforcement Learning for tactile-based finger gaiting

Sharma, David

Adaptive Training Strategies for Brain-Computer-Interfaces (Technical Report)

Technische Universität Darmstadt M.Sc. Thesis, 2017.

(BibTeX | Tags: BCI, human motor control, Reinforcement Learning | Links: )

Adaptive Training Strategies for Brain-Computer-Interfaces

2016

Smyk, Mike

Model-based Control and Planning for Real Robots (Technical Report)

Technische Universität Darmstadt M.Sc. Project, 2016.

(BibTeX | Tags: Probabilistic Inference, SOC | Links: )

Model-based Control and Planning for Real Robots

Stark, Svenja

Learning Probabilistic Feedforward and Feedback Policies for Stable Walking (Technical Report)

Technische Universität Darmstadt M.Sc. Thesis, 2016.

(BibTeX | Tags: locomotion, movement primitives | Links: )

Learning Probabilistic Feedforward and Feedback Policies for Stable Walking

Kohlschuetter, Jan

Learning Probabilistic Classifiers from Electromyography Data for Predicting Knee Abnormalities (Technical Report)

Technische Universität Darmstadt M.Sc. Thesis, 2016.

(BibTeX | Tags: BCI, Reinforcement Learning | Links: )

Learning Probabilistic Classifiers from Electromyography Data for Predicting Knee Abnormalities

2015

Tanneberg, Daniel

Spiking Neural Networks Solve Robot Planning Problems (Technical Report)

Technische Universität Darmstadt M.Sc. Thesis, 2015.

(BibTeX | Tags: RNN, spiking | Links: )

Spiking Neural Networks Solve Robot Planning Problems

2014

Mindt, Max

Probabilistic Inference for Movement Planning in Humanoids (Technical Report)

Technische Universität Darmstadt M.Sc. Thesis, 2014.

(BibTeX | Tags: Probabilistic Inference, SOC | Links: )

Probabilistic Inference for Movement Planning in Humanoids

Mundo, Jan

Structure Learning for Movement Primitives (Technical Report)

Technische Universität Darmstadt M.Sc. Thesis, 2014.

(BibTeX | Tags: graphical models, movement primitives | Links: )

Structure Learning for Movement Primitives

2013

Kniewasser, Gerhard

Reinforcement Learning with Dynamic Movement Primitives - DMPs (Technical Report)

Technische Universität Graz M.Sc. Project, 2013.

(BibTeX | Tags: locomotion, movement primitives | Links: )

Reinforcement Learning with Dynamic Movement Primitives - DMPs

Prevenhueber, Oliver

Monte Carlo Sampling Methods for Motor Control of Constraint High-dimensional Systems (Technical Report)

Technische Universität Graz M.Sc. Thesis, 2013.

(BibTeX | Tags: Probabilistic Inference, Simulation | Links: )

Monte Carlo Sampling Methods for Motor Control of Constraint High-dimensional Systems

Gsenger, Othmar

Probabilistic Models for Learning the Dynamics Model of Robot (Technical Report)

Technische Universität Graz M.Sc. Thesis, 2013.

(BibTeX | Tags: model learning, Probabilistic Inference | Links: )

Probabilistic Models for Learning the Dynamics Model of Robot

2012

Prevenhueber, Oliver

Gibbs Sampling Methods for Motor Control Problems with Hard Constraints (Technical Report)

Technische Universität Graz 2012.

(BibTeX | Tags: graphical models, Simulation | Links: )

Gibbs Sampling Methods for Motor Control Problems with Hard Constraints

2011

Genewein, Tim

Structure Learning for Robotic Motor Control (Technical Report)

Technische Universität Graz M.Sc. Thesis, 2011.

(BibTeX | Tags: graphical models, model learning)

Wiesner, Thomas

Ein Vergleich von Lernalgorithmen für Parametersuche im hochdimensionalen Raum (Technical Report)

Technische Universität Graz B.Sc. Thesis, 2011.

(BibTeX | Tags: Reinforcement Learning, Simulation | Links: )

Ein Vergleich von Lernalgorithmen für Parametersuche im hochdimensionalen Raum