Researchers Help Robots Predict People’s Direction

Photo by Austin Distel on Unsplash

A group of MIT researchers collaborated with BMW in 2018 for a manufacturing project. They are creating a workplace where humans can safely work together with robots. The aim is to enhance productivity by rigging the robots on rails, enabling human workers to pass through. If robots can travel across stations, they can efficiently do precise but heavy tasks to assist humans. For the safety of employees, they programmed the robots to pause when someone passes by.

Robots work by using servo motors such as RDrive, where they only move after receiving instructions; this raises the problem of robots freezing at their positions since the program is making the robots wait for humans. If the waiting time of these robots is added together, the inefficiency will be significant. Having this implemented in a manufacturing shop floor is not feasible.

The challenge for the researchers

Usually, human movement prediction algorithms are borrowed from word and music processing programs. Researchers do it often, such as for typing prediction in suggesting search results. However, speech and music are structured with grammar and rhythm, respectively, unlike human movement, which is random. Even in routine actions like getting tools and even walking, people do it in a slightly different way the next time.

To make the researchers’ work more manageable, they created a data algorithm for streaming motion. It creates a motion diagram by using dots that represent positions of people during the time they are walking. The data is then compiled as a reference to compare every single thing to obtain typical trajectories in different scenarios. The findings show that a trajectory map can be created by the algorithm based on the position of the dots.

Lasota, lead of the project, mentioned that the algorithm is easy to be baffled. If people suddenly stopped while walking, the prediction would be disrupted. It will wait until people walk again so it can tell the robot to move, even the person has a safe distance from the robot. These temporary stops can confuse the distance-based algorithms because the dots will pile up in the same position until the individual continues to walk.

Project lead Lasota also said that a similar thing happened when the person moved back and forth. The first group of points will overlap the second group of positions. However, when the time is considered, the points will be regarded as different.

The unique and brand new algorithm

To solve the problem, the MIT researchers tried to enhance motion prediction and make the algorithm work simultaneous. The researchers tried a technique for robots to understand people better. The algorithm used created patterns which helps the robots recognize the stops and overlaps of the human motion. According to the previous records, this new program can precisely predict when the person will move,

After they tried to implement the modification to the BMW manufacturing robots, they behaved differently. The algorithm made the robot smarter, like a pedestrian crossing the street even without traffic lights. The previous reaction from walking people disappeared. However, the robots still passed by safely also when people walked back and forth. The changes made the robots avoid being stuck when people came across their path.

How does it work?

Before they tried the algorithm on manufacturing robots, the MIT team decided to test it first on datasets. They have two collected datasets to try on: the first one is the dataset of people passing by across the robots. The second is a pre-recorded set of hand movements of bolt installation and sealant brushing.

The solution involved collecting reference trajectories from human motions first. It will create a library where the algorithm can use as a reference. After that, it will observe a “partial trajectory” that compares the real-time motion segments to the library.

The real-time motion’s distance and timing will be compared to the library after that. The result is a predicted trajectory that will help the robot to predict if the person will stop or return. From the expected path, the robot can calculate and decide if it should cross or not.

It was also stated that while combining their previously-used algorithms, the improvements are seen on the robots. They can more reliably predict the motion and timing of a person, which makes the robots freeze less. The robots then resumed its path seamlessly after a human crossed his way in the factory floor robot dataset.

According to Shah, this algorithm can be used in future cases where human motions have patterns and are repetitive. For the robots to work along with humans safely and efficiently, robots must learn how humans move. The trends observed by the robots will become a basis for the algorithm, enabling the robots to learn human behavior.