Teaching

Humanoid Robotics (RO5300)

[SS2018] During the summer semester Prof. Dr. Elmar Rueckert is teaching the course Humanoid Robotics (RO5300). In this course he discusses the key components of one of the most complex autonomous systems. These topics are

  1. Kinematics, Dynamics & Locomotion
  2. Representations of Skills & Imitation Learning
  3. Feedback Control, Priorities & Torque Control
  4. Reinforcement Learning & Policy Search
  5. Sensor Integration & Fusion
  6. Cognitive Reasoning & Planning

The exam results are out. Please visit our moodle page to find your grade. The second exam date is on Sept. 27th, 2018 10 am – 12 am in our seminar room Nr.64 in building 64.

Ask questions at our course related Q&A page here.

The exam was at July 17th, 2018 in the Audimax building in room AM S 2. Fill out the exam registration form. 

Introduction slides to the course on Humanoid Robotics by Prof. Dr. Elmar Rueckert, University of LuebeckThis course provides a unique overview over central topics in robotics. A particular focus is put in the dependencies and interaction among the components in the control loop illustrated in the image above. These interactions are discussed in the context of state of the art methods including dynamical systems movement primitives, gradient based policy search methods or probabilisitic inference for planning algorithms.

In sum,  the lecture provides a structured and well motivated overview over modern techniques and tools which enable the students to define reward functions, implement robot controller and interaction software and to apply and extend state of the art reinforcement learning and planning approaches.

No special knowledge is required beforehand. All concepts and theories will be developed during the lectures or the tutorials.

The students will also experiment with state of the art machine learning methods and robotic simulation tools in accompanying exercises. Hands on tutorials on programming with Matlab and the simulation tool V-Rep complement the course content.

Follow this link to register for the course: https://moodle.uni-luebeck.de/course/view.php?id=3261.

Location: Room: C4-S03

Course materials (tentative slides, last update Feb. 27, 2018)

  1. Humanoid Robotics Intro (L1: April, 10th)
  2. Kinematics, Dynamics & Locomotion (L2: April, 17th, L3: April, 24th)
  3. Representations of Skills & Imitation Learning (L4: May 8th, L5: May 15th)
  4. Feedback Control, Priorities & Torque Control (L6: May 30th, L7: June 5th)
  5. Reinforcement Learning & Policy Search(L8: June 19th, L9: June 26th, L10: July 3rd)
  6. Sensor Integration & Fusion and Cognitive Reasoning & Planning (L11: July 10th)
  7. Exam (L12: July 17th, 2018, Handout PDF)

Materials for the exercise

For simulating robot manipulation tasks we will use the simulator V-REP. For research and for teaching a free eduction version can be found here. To experiment with state of the art robot control and learning methods Mathworks’ MATLAB will be used. If you do not have it installed yet, please follow the instructions of our IT-Service Center.

  1. Introduction to Matlab and V-Rep (Exercise, Solution)
    Tutorial: April, 18th 2018 in room C4-S03.
  2. Kinematics and Dynamics Assignment (Exercise/Assignment, Solution, Material, Latex Draft)
    Tutorial: May, 2nd 2018, Submission: May 22nd, 2018 at 10 am. 
  3. Movement Representations (Exercise/Assignment, Solution, Material)
    Tutorial: May, 16th 2018, Submission: June 08th, 2018 at 10 am.
    Videos to the exercise
  4. Feedback Control (Exercise/Assignment, Solution)
    Tutorial: May, 30th 2018, Submission: June 22nd, 2018 at 10 am.
  5. Path Planning and Reinforcement Learning (Exercise/Assignment, Solution, Material)
    Tutorial: June, 20th 2018, Submission: July 10th, 2018 at 10 am 

Matlab Files shown during the Tutorial can be found here.

Probabilistic Learning for Robotics (RO5601)

[WS2018/19] In the winter semester, I will teach a course on Probabilistic Learning for Robotics which covers advanced topics including graphical models, factor graphs, probabilistic inference for decision making and planning, and computational models for inference in neuroscience. The lecture will take place in the Seminarraum Informatik 5 (Von Neumann) 2.132 from 12.00 – 14.00 on selected Thursdays.

In accompanying exercises and hands on tutorials the students will experiment with state of the art machine learning methods and robotic simulation tools. In particular, Mathworks’ MATLAB,  the robot middleware ROS and the simulation tool V-Rep will be used. The exercises and tutorials will also take place in the seminar room  2.132 on selected Fridays (see the course materials and dates below).

Prerequisites (recommended) 

  • Humanoid Robotics (RO5300)
  • Robotics (CS2500)

Follow this link to register for the course: https://moodle.uni-luebeck.de/course/view.php?id=3793.

Location: Room: Seminarraum Informatik 5 (Von Neumann) 2.132

Course materials and dates (tentative slides, last update Sep. 27, 2018)

  1. Probabilistic Learning for Robotics Intro (L1: October, 18th, 12.00 – 14.00)

 

Student Theses

Here is a list of currently offered open topics for Master’s theses.

BCI graphical models human motor control intrinsic motivation inverse dynamics locomotion Manipulation model learning movement primitives Natural Language Processing neural network Probabilistic Inference Reinforcement Learning RNN Simulation SOC spiking

2018

Tanneberg, Daniel; Peters, Jan; Rueckert, Elmar

Intrinsic Motivation and Mental Replay enable Efficient Online Adaptation in Stochastic Recurrent Networks (Journal Article)

Neural Networks - Elsevier, 2018, (Impact Factor of 7.197 - 2017).

(BibTeX | Tags: neural network, Probabilistic Inference, RNN, spiking | Links: )

Intrinsic Motivation and Mental Replay enable Efficient Online Adaptation in Stochastic Recurrent Networks

Dittmar, Denny

Distributed Reinforcement Learning with Neural Networks for Robotics (Technical Report)

Technische Universität Darmstadt M.Sc. Thesis, 2018.

(BibTeX | Tags: Reinforcement Learning, RNN, spiking | Links: )

Distributed Reinforcement Learning with Neural Networks for Robotics

2017

Frisch, Yannik

The Effects of Intrinsic Motivation Signals on Reinforcement Learning Strategies (Technical Report)

Technische Universität Darmstadt B.Sc. Thesis, 2017.

(BibTeX | Tags: intrinsic motivation, Reinforcement Learning | Links: )

The Effects of Intrinsic Motivation Signals on Reinforcement Learning Strategies

Pfanschilling, Viktor

Self-Programming Mutation and Crossover in Genetic Programming for Code Generation (Technical Report)

B.Sc. Thesis, 2017.

(BibTeX | Tags: model learning | Links: )

Self-Programming Mutation and Crossover in Genetic Programming for Code Generation

Thiem, Simon-Konstantin

Simulation of the underactuated Sake Robotics Gripper in V-REP and ROS (Technical Report)

Technische Universität Darmstadt B.Sc. Thesis, 2017.

(BibTeX | Tags: Manipulation, Simulation | Links: )

Simulation of the underactuated Sake Robotics Gripper in V-REP and ROS

Nakatenus, Moritz

LSTM Networks for movement planning in humanoids (Technical Report)

Technische Universität Darmstadt M.Sc. Project, 2017.

(BibTeX | Tags: inverse dynamics, RNN)

LSTM Networks for movement planning in humanoids

Gondaliya, Kaushik

Learning to Categorize Issues in Distributed Bug Tracker Systems (Technical Report)

Technische Universität Darmstadt M.Sc. Thesis, 2017.

(BibTeX | Tags: Natural Language Processing, RNN)

Learning to Categorize Issues in Distributed Bug Tracker Systems

Polat, Harun

Nonparametric deep neural networks for movement planning (Technical Report)

Technische Universität Darmstadt B.Sc. Thesis, 2017.

(BibTeX | Tags: model learning, RNN, spiking | Links: )

Nonparametric deep neural networks for movement planning

Plage, Lena

Reinforcement Learning for tactile-based finger gaiting (Technical Report)

Technische Universität Darmstadt B.Sc. Thesis, 2017.

(BibTeX | Tags: Manipulation, Reinforcement Learning | Links: )

Reinforcement Learning for tactile-based finger gaiting

Sharma, David

Adaptive Training Strategies for Brain-Computer-Interfaces (Technical Report)

Technische Universität Darmstadt M.Sc. Thesis, 2017.

(BibTeX | Tags: BCI, human motor control, Reinforcement Learning | Links: )

Adaptive Training Strategies for Brain-Computer-Interfaces

2016

Smyk, Mike

Model-based Control and Planning for Real Robots (Technical Report)

Technische Universität Darmstadt M.Sc. Project, 2016.

(BibTeX | Tags: Probabilistic Inference, SOC | Links: )

Model-based Control and Planning for Real Robots

Stark, Svenja

Learning Probabilistic Feedforward and Feedback Policies for Stable Walking (Technical Report)

Technische Universität Darmstadt M.Sc. Thesis, 2016.

(BibTeX | Tags: locomotion, movement primitives | Links: )

Learning Probabilistic Feedforward and Feedback Policies for Stable Walking

Kohlschuetter, Jan

Learning Probabilistic Classifiers from Electromyography Data for Predicting Knee Abnormalities (Technical Report)

Technische Universität Darmstadt M.Sc. Thesis, 2016.

(BibTeX | Tags: BCI, Reinforcement Learning | Links: )

Learning Probabilistic Classifiers from Electromyography Data for Predicting Knee Abnormalities

2015

Tanneberg, Daniel

Spiking Neural Networks Solve Robot Planning Problems (Technical Report)

Technische Universität Darmstadt M.Sc. Thesis, 2015.

(BibTeX | Tags: RNN, spiking | Links: )

Spiking Neural Networks Solve Robot Planning Problems

2014

Mindt, Max

Probabilistic Inference for Movement Planning in Humanoids (Technical Report)

Technische Universität Darmstadt M.Sc. Thesis, 2014.

(BibTeX | Tags: Probabilistic Inference, SOC | Links: )

Probabilistic Inference for Movement Planning in Humanoids

Mundo, Jan

Structure Learning for Movement Primitives (Technical Report)

Technische Universität Darmstadt M.Sc. Thesis, 2014.

(BibTeX | Tags: graphical models, movement primitives | Links: )

Structure Learning for Movement Primitives

2013

Kniewasser, Gerhard

Reinforcement Learning with Dynamic Movement Primitives - DMPs (Technical Report)

Technische Universität Graz M.Sc. Project, 2013.

(BibTeX | Tags: locomotion, movement primitives | Links: )

Reinforcement Learning with Dynamic Movement Primitives - DMPs

Prevenhueber, Oliver

Monte Carlo Sampling Methods for Motor Control of Constraint High-dimensional Systems (Technical Report)

Technische Universität Graz M.Sc. Thesis, 2013.

(BibTeX | Tags: Probabilistic Inference, Simulation | Links: )

Monte Carlo Sampling Methods for Motor Control of Constraint High-dimensional Systems

Gsenger, Othmar

Probabilistic Models for Learning the Dynamics Model of Robot (Technical Report)

Technische Universität Graz M.Sc. Thesis, 2013.

(BibTeX | Tags: model learning, Probabilistic Inference | Links: )

Probabilistic Models for Learning the Dynamics Model of Robot

2012

Prevenhueber, Oliver

Gibbs Sampling Methods for Motor Control Problems with Hard Constraints (Technical Report)

Technische Universität Graz 2012.

(BibTeX | Tags: graphical models, Simulation | Links: )

Gibbs Sampling Methods for Motor Control Problems with Hard Constraints

2011

Genewein, Tim

Structure Learning for Robotic Motor Control (Technical Report)

Technische Universität Graz M.Sc. Thesis, 2011.

(BibTeX | Tags: graphical models, model learning)

Wiesner, Thomas

Ein Vergleich von Lernalgorithmen für Parametersuche im hochdimensionalen Raum (Technical Report)

Technische Universität Graz B.Sc. Thesis, 2011.

(BibTeX | Tags: Reinforcement Learning, Simulation | Links: )

Ein Vergleich von Lernalgorithmen für Parametersuche im hochdimensionalen Raum