Changes in Center of Mass during Preliminary Motion for Prediction of Direction Change

In recent years, the number of single elderly people has been increasing, and the needs of residents have been diversifying. Towards these backgrounds, we propose the concept of "Biofied bulding". The aim of Biofied Building is to create living spaces where residents can live safely, securely and comfortably. Small robots are used as an interface between residents and living space in Biofied Building. The aim of using robots is to sense the position and movement of residents in real time and providing feedback to them. However, the present control systems of the robot do not have enough functions to estimate the risk of accidents such as falls and choose the pathways which do not disturb residents. Therefore, the purpose of this research is to recognize and predict human behavior in a living space by using a robot to realize Biofied Building. In particular, we focus on the direction change motion, which is an important behavior in a living space, and extract the prediction parameters. In particular, it is reported that the direction change motion account for about 20% of gait during the daily life. Therefore, our research group decided to focus on direction change motion. In this study, we focused on the center of the head to extract parameters for prediction of the direction change motion. There are features in the velocity change of the center of the head compared with straight-line gait. There was a velocity amplification of the opposite direction of the direction change before the start of the motion. It is assumed that the shift of the center of mass make it to easier to step out to the direction of the turn. Introduction In recent years, the number of single households has been increasing and lifestyle has been diversified. Towards these backgrounds, we suggest the concept of “Biofied Building” in which a small sensor agent robot provides a safe, secure, and comfortable living space. The purpose of using the robot is sensing the positions and motions of residents in real-time. However, the present control systems of the robot do not have enough functions to estimate the risk of accidents such as falls and choose the pathways which do not disturb residents. A motion prediction will be a key function of the robot control system to realize more safe, secure, and comfortable living spaces. Many conventional motion prediction methods use RNN (Recurrent Neural Networks). These methods mainly have two problems. The one is that it is difficult to implement due to the necessity of the big data and the large calculation cost. The other is that the physical constraint of the body is not considered. Hence, our previous study proposed a simple motion prediction method of gait based on human dynamics. However, the method needs to expand to be applied to non-steady motion since its target assumed only steady motion. Structural Health Monitoring Materials Research Forum LLC Materials Research Proceedings 18 (2021) 287-293 https://doi.org/10.21741/9781644901311-35 288 Among the gait which is the most basic one of activities of daily living, the gait in which the paths are not straight account for over 40%, and direction change motion account for about 20%[1][2]. Therefore, our research group decided to focus on direction change motion. The motion prediction systems can be distinguished into three types: the optical motion capture system, the inertial motion capture system, and the skeleton tracking function by RGB-D sensors. Our research group has conducted an exhaustive extraction of predictive parameters for the direction change motion using an inertial motion capture system. However, this method has a problem that an inertial motion capture system is a contact sensor. Therefore, we aimed to establish a more practical system by using RGB-D sensors. As a first step, we investigated parameters of motion predictions by using the optical motion capture system and aimed to establish a motion prediction method. Device and Experiment Device: VICON (The optical motion capture system)[3] A motion capture system measures the body and joint position of time series. In recent years, motion capture systems have been used in a wide range of applications such as human motion analysis, rehabilitation, humanoid robotics, and residential and urban space evaluation. The optical motion capture system captures marker position which are attached body parts using multiple cameras to estimate the three-dimensional position. We have been developing a system to estimate the human body's position and joint angles by mapping the markers to a predefined articulated human body model. This is called inverse kinematics. There are two advantages of this system. First of all, it is small, lightweight and wireless. Therefore, subjects can move relatively freely. Secondary, the accuracy of absolute position is high. On the other hand, it needs a lot of technology and space to install and calibrate multiple cameras. Fig.1 VICON (The optical motion capture system)[3] Structural Health Monitoring Materials Research Forum LLC Materials Research Proceedings 18 (2021) 287-293 https://doi.org/10.21741/9781644901311-35 289 Experiment The experiment was conducted on the campus of Keio University. Subjects were 6 students in their twenties (males: 2 , females: 4). They were required to make a direction change motion voluntary within a specified range. They were not specified their own walking speed, they walked with their own comfortable walking speed. This experiment image is shown in Fig. 2. The number of trials was one each for straight-line and direction change, and the data of 12 trials were used. Fig.2 The condition of the experiment Definition of direction change period We define the initiation and termination of direction change motion. The definition was based on a previous study.[4] The angle of the lumbar region in steady gait before the direction change was a standard, and the relative angle in the direction of rotation was used for the definition. The relative angles are shown in Fig. 3. These values were calculated from the placement of lumbar taken by VICON. Fig. 3 shows that the subject walked in the straight-line from 0 to 2 seconds, and after 2 seconds, the subject's lumbar rotation increased and changed by almost 90°and subject walked in a steady again. The start time of the direction change was defined as the time when the rotation angle of the lumbar region exceeds a minimum value of one step before the end of the steady gait and the swing leg's heel contacts the ground. Likewise, the end time of the direction change was defined as the time when the rotation angle of the lumbar region exceeds the maximum value of rotation angle and the swing leg's heel contacts the ground. Structural Health Monitoring Materials Research Forum LLC Materials Research Proceedings 18 (2021) 287-293 https://doi.org/10.21741/9781644901311-35 290 Fig. 3 Definition of direction change period Parameter for prediction We consider the difference between the straight-line gait and direction change. As the direction change is a circular motion performed on the walking surface, it is assumed that the centrifugal force is added as an inertial force apart from the ground reaction force and gravity during the direction change. In order to resist the inertial force, it is considered to be efficient to change direction by shifting the center of gravity of the body by tilting the trunk in advance. In addition, the previous study [5] shows that the head movement is the most preceding movement in the direction change motion. Therefore, we focused on the movement of the head's center of mass to capture the cues for prediction of direction change motion. In Fig.4 shows that the 4 retro-reflective marker were placed at the following anatomical locations of head; the right forehead (RFHD), left forehead (LFHD), right back of head (RBHD), left back of head (LBHD). As the center of mass of the head, the values of the midpoint of these four points were used. In this study, we focused on the changes of the left and right sides of the head (x-y direction) during direction change. Therefore, we did not consider the height (z direction) of the head. Fig. 4 Marker placement of head 青:VICONデータ ピンク:モデル適用後 An gl e[ D eg ]


Introduction
In recent years, the number of single households has been increasing and lifestyle has been diversified. Towards these backgrounds, we suggest the concept of "Biofied Building" in which a small sensor agent robot provides a safe, secure, and comfortable living space. The purpose of using the robot is sensing the positions and motions of residents in real-time. However, the present control systems of the robot do not have enough functions to estimate the risk of accidents such as falls and choose the pathways which do not disturb residents. A motion prediction will be a key function of the robot control system to realize more safe, secure, and comfortable living spaces.
Many conventional motion prediction methods use RNN (Recurrent Neural Networks). These methods mainly have two problems. The one is that it is difficult to implement due to the necessity of the big data and the large calculation cost. The other is that the physical constraint of the body is not considered. Hence, our previous study proposed a simple motion prediction method of gait based on human dynamics. However, the method needs to expand to be applied to non-steady motion since its target assumed only steady motion. Among the gait which is the most basic one of activities of daily living, the gait in which the paths are not straight account for over 40%, and direction change motion account for about 20% [1] [2] . Therefore, our research group decided to focus on direction change motion.
The motion prediction systems can be distinguished into three types: the optical motion capture system, the inertial motion capture system, and the skeleton tracking function by RGB-D sensors. Our research group has conducted an exhaustive extraction of predictive parameters for the direction change motion using an inertial motion capture system. However, this method has a problem that an inertial motion capture system is a contact sensor. Therefore, we aimed to establish a more practical system by using RGB-D sensors. As a first step, we investigated parameters of motion predictions by using the optical motion capture system and aimed to establish a motion prediction method.

Device and Experiment
Device: VICON (The optical motion capture system) [3] A motion capture system measures the body and joint position of time series. In recent years, motion capture systems have been used in a wide range of applications such as human motion analysis, rehabilitation, humanoid robotics, and residential and urban space evaluation. The optical motion capture system captures marker position which are attached body parts using multiple cameras to estimate the three-dimensional position. We have been developing a system to estimate the human body's position and joint angles by mapping the markers to a predefined articulated human body model. This is called inverse kinematics. There are two advantages of this system. First of all, it is small, lightweight and wireless. Therefore, subjects can move relatively freely. Secondary, the accuracy of absolute position is high. On the other hand, it needs a lot of technology and space to install and calibrate multiple cameras.

Experiment
The experiment was conducted on the campus of Keio University. Subjects were 6 students in their twenties (males: 2 , females: 4). They were required to make a direction change motion voluntary within a specified range. They were not specified their own walking speed, they walked with their own comfortable walking speed. This experiment image is shown in Fig. 2. The number of trials was one each for straight-line and direction change, and the data of 12 trials were used.

Definition of direction change period
We define the initiation and termination of direction change motion. The definition was based on a previous study. [4] The angle of the lumbar region in steady gait before the direction change was a standard, and the relative angle in the direction of rotation was used for the definition. The relative angles are shown in Fig. 3. These values were calculated from the placement of lumbar taken by VICON. Fig. 3 shows that the subject walked in the straight-line from 0 to 2 seconds, and after 2 seconds, the subject's lumbar rotation increased and changed by almost 90°and subject walked in a steady again. The start time of the direction change was defined as the time when the rotation angle of the lumbar region exceeds a minimum value of one step before the end of the steady gait and the swing leg's heel contacts the ground. Likewise, the end time of the direction change was defined as the time when the rotation angle of the lumbar region exceeds the maximum value of rotation angle and the swing leg's heel contacts the ground.

Fig. 3 Definition of direction change period
Parameter for prediction We consider the difference between the straight-line gait and direction change. As the direction change is a circular motion performed on the walking surface, it is assumed that the centrifugal force is added as an inertial force apart from the ground reaction force and gravity during the direction change. In order to resist the inertial force, it is considered to be efficient to change direction by shifting the center of gravity of the body by tilting the trunk in advance. In addition, the previous study [5] shows that the head movement is the most preceding movement in the direction change motion. Therefore, we focused on the movement of the head's center of mass to capture the cues for prediction of direction change motion. In Fig.4 shows that the 4 retro-reflective marker were placed at the following anatomical locations of head; the right forehead (RFHD), left forehead (LFHD), right back of head (RBHD), left back of head (LBHD). As the center of mass of the head, the values of the midpoint of these four points were used. In this study, we focused on the changes of the left and right sides of the head (x-y direction) during direction change. Therefore, we did not consider the height (z direction) of the head.

Result
Position change of center of mass Figs. 5 and 6 show the trajectory of the center of gravity of the head and both ankles, in straight-line gait and direction change. In Fig. 5 (b), the trajectory of the center of the head before the change of direction is shown as a dashed line and the trajectory after the start of the direction change is shown as a solid line. In the straight-line gait in Fig. 5 (a), the center of the head is located between the left and right ankles and draws a straight trajectory, whereas in Fig. 5 (b), the center of head shifts to the right side immediately after the start of the direction change. However, there were no signs that were useful in predicting, therefore we focused on the changes of the velocity of the center of head and plotted it in Figure 6.
(a) the straight-line gait (b) the direction change Velocity change of center of mass The dashed red line in Fig. 6 is the start time of the direction change motion. Fig. 6 shows that the value of the x-velocity before the direction change motion is larger than the those in the straight-line gait. This result was shown in 5 of the 6 subjects and Table.1 shows the amplification of the velocity in the x-axis. The average value of the velocity in the straight-line gait were used. It shows that it is amplified by 20-80% compared to the straight-line gait. This amplification occurred around the time when the opposite foot of the initiation foot of the direction change contacted the ground. Therefore, the center of head shifted to the left side once before the direction change. That made it easier to step out to the right side. However, there are individual differences in this behavior and there is a bias in the amplification rate.
(a) the straight-line gait (b) the direction change

Conclusion and Future plan
In this study, we focused on the center of the head to extract parameters for prediction of the direction change motion. In the velocity change of the center of the head, compared with straight-line gait, there was an amplification of the velocity opposite direction of the direction change before the start of the motion. It is assumed that the shift of the center of mass make it to easier to step out to the direction of the turn.
In the future, it is necessary to compare the conditions of the direction change motion to make it more versatile. The conditions of the direction change motion mean "step turn" and "spin turn", change of direction angles [6]- [9] . In addition, there is a task to increase the number of subjects. Furthermore, we need to investigate the center of gravity of each segment and the relationship between segments during preliminary action of the direction change.