Vision based Robotics

Mission: To conduct bio-inspired research in the area of real-time image processing, pattern recognition, artificial intelligence, and dynamic system simulation with the aim of building service robots primarily applied to enhance quality of living and autonomous living for the elderly.

Research Area:  Robot Vision; Robot Intelligence; Social robots, companion robots and video surveillance for elderly care; Real-time vision systems for medical Instruments; Real-time embedded vision systems for Industrial Machines and Applications.

We aim at bridging the human-robot intelligence gap: We build humanoid robots with a human like body, however the intelligence we realize in robotic consumer products such as a vacuum cleaner hardly exceeds that of a Paramecium (Pantoffeldiertje), see Fig. 1.

As co-outcome we aim at obtaining insight in the architecture and functionality of the human body with respect to reflexes and behaviors based on the visual system, such as posture and balance.

Fig. 1 The intelligence gap between humans and humanoid robots

Research Lines:

• 3D and Active Vision Systems
• Declarative and Procedural Learning Systems
• Vision and AI for Companion Robots
• Instruments for Cure and Care

3D and Active Vision Systems

In this research line we investigate all possibilities - and their interplay - of the perception of the 6D pose of objects in a 3D scene and their 3D shape. We research Time of Flight cameras for industrial bin-picking, stereo vision for the mapping of the space around a robot , depth from vergence & focus for the perception and tracking of objects to grasp, and depth from shading for the mapping of the light distribution in operating rooms, for depth perception with endoscopic cameras and single lens 3D TV cameras. Active vision uses actuation, such as from robot-eyes, -head and -body to obtain a better insight in a scene and for visual servoing.

Fig. 2 Moveable robot eyes; Depth from Shading Camera; Visual Servoing; Grasp Point Detection

Declarative and Procedural Learning Systems

In this research line we investigate the abilities of robots to learn from scratch, to adapt to a changing environment. For declarative learning we research a “curious robot” that uses saliency detection to focus on objects, grasps it from various viewpoints to learn its appearance and object class. For procedural learning we develop simulated and real robots that learn to walk by trial and error driven by rewards and punishments (Reinforcement Learning)

Fig. 3 Pose detection of objects; Bin picking; Recognition learned objects; Robot that learns to walk

Vision and AI for Companion Robots

In this line of research we focus on the perception, cognition and intelligence of companion robots. Robots need perceive the scene, build up a map of the scene, need to know where they are in the scene, what the passive objects and the actors - like humans, animals and other robots - in the scene are and infer what they are up to. Then they need to perform their task in cooperation with other robots and humans, as good as possible without causing damage to themselves or their neighbourhood and possibly hindered by other actors. A toy application is robot soccer (ROBOCUP), the serious application is scout and household robots for elderly care (ROBOTS@HOME). The developed mapping and tracking algorithms for robots can be directly used for Augmented Reality. Hence we research AR tools for art and design, industrial design, architecture and medical applications.

Fig. 4 Keypoint detection for pose tracking; Map building; Delft Personal Robot; Augmented Reality

Instruments for Cure and Care

In this line of research we focus on the design of medical instruments based on computer vision, such as for strabismus measurements on young children and work flow tracking of surgeons in the operating room or for  detection of human behaviors such as collapsing and falling. Finally, we research the effectiveness of social robots for elderly with dementia.

Fig. 5 Surgical instruments tracking; Strabismus measuring; Fall detection; Effectiveness social robots