Extraction of Parameters for 90-degree Turn Prediction Using the IMU-based Motion Capture System

. Against the increasing number of single households, we have been proposing the “Biofied Building” that provides a safe, secure, and comfortable living space for a resident using a small home robot. The robot can be used for real-time sensing of the resident’s position and behavior. On the other hand, for further use of the robot, such as choosing a path that does not disturb the resident, a phase to predict the resident’s behavior is necessary. Walking, which is one of the most basic activities of daily living, is often targeted in studies of motion prediction. However, most of them deal with steady walking, even though walking in daily life includes unsteady walking such as the turning motion. Therefore, the purpose of this study was to extract the prediction parameters to construct a prediction method for the unsteady 90-degree turn. In this study, we explored the effective prediction parameters for 90-degree turns based on the measured data using the inertial measurement unit (IMU) based motion capture system aiming to introduce the prediction of unsteady walking to the “Biofied Building”.


Introduction
Because of the increase in the number of elderly single-person households and the diversification of lifestyles, the need for housing which is incorporating monitoring function is rising. Toward this need, we have been proposing the "Biofied Building", a building system that uses small home robots to provide a safe, secure, and comfortable space for each resident [1,2]. One of the aims of using robots is sensing the position and movement of the residents in real-time. The prediction of the resident's movement is effective to achieve more natural robot control, such as choosing paths of movement that does not interfere with the resident.
Recurrent Neural Networks (RNNs), a deep learning method with time-series elements, are widely used for motion prediction [3][4][5], but it is pointed out that physical constraints of the human body are not taken into account [1]. On the contrary, Kimura et al. proposed a simple method for predicting walking based on the dynamics of the human body [1]. However, the system only targeted steady walking motions even though the daily walking includes unsteady ones. Thus, a system which is adaptable for unsteady motions is needed.
Walking is known as the most basic motion among activities of daily living. It has been reported that more than 40% of the total number of walking is unsteady [6], and among them, the turn motion accounts for about 20% of the walking in daily life [7]. Thus, we targeted the 90-degree turn as one of the unsteady walking motions.
There are three main types of motion measurement systems: an optical motion capture system, which measures the motion of a subject who wears multiple reflective markers by surrounded infrared cameras; an IMU-based motion capture system, which measures the motion by attaches multiple IMU sensors; and a skeletal position tracking function using RGB-D sensors. Although optical motion capture systems are widely used in the medical field due to their high accuracy, they are not suitable for the measurement of turn motion that requires the long walking path due to their limited measurement range. The skeletal position tracking function of RGB-D sensors is not supposed to measure the turn motion, because the accuracy of the estimation depends on body orientation. On the other hand, the IMU is widely used for motion measurement in living spaces and urban spaces because they are relatively inexpensive and have no measurement environment restrictions. The methods for discriminating walking people and recognizing their motions by analyzing acceleration data has been proposed [8,9], and it can be applied to motion prediction by focusing on the transition state between motions. Besides, IMU has been widely used in clinical applications, which require relatively high accuracy, and their usefulness has been demonstrated in recent years [10,11]. Shigeta et al. used IMU sensors to extract gait features in the gait of ASD patients and compared them with those of healthy subjects [11].
The purpose of this study is to reveal the prediction parameters of 90-degree turns aiming to develop with a home robot in the "Biofied Building". In this paper, we extracted efficient prediction parameters of 90-degree turn based on experimental measurements using an IMU-based motion capture system to obtain acceleration and angular velocity data. This study is positioned as a basic study to introduce a prediction system of unsteady walking to the "Biofied Building".

Proposed method
The duration of the turn is very short and there is no linearity. Therefore, we tried to detect the transition to the 90-degree turn by comparing it with a highly reproducible steady walking.
A Perception Neuron [12], an IMU-based motion capture system, was used to measure the 90-degree turn ( Figure 1). An IMU called a Neuron, which consists of a triaxial acceleration sensor, a triaxial gyroscope, and a triaxial magnetometer is attached to each position to obtain the acceleration and angular velocity data at each sensor position. The 90-degree turn is caused by the change of acceleration and angular velocity of the body's center of gravity, which is caused by the chain of motion of each body part. Therefore, we focus on the variation of the acceleration and angular velocity of each part of the body. The frame rate was 120 fps, and the data was acquired by wireless communication. The accuracy of the measurements is considered to be within 5 degrees of upper body angular error [13]. MATLAB 2019a (MathWorks Inc., Natick, MA, USA) was used for data analysis. In this study, we use an Axis Neuron, a software for a Perception Neuron, to obtain acceleration and angular velocity data for 19 body parts, including estimates, from 17 sensors. Figure 2 shows the names and locations of each.
Since the accelerometer only detects the gravitational acceleration at resting, the theoretical value of the composite acceleration is 1 G. However, the actual composite acceleration may not be 1 G due to voltage fluctuations. Therefore, we normalized each axis' acceleration data by dividing the raw data by the mean square of acceleration data of each axis from the second to the 100 frames of resting. Besides, a 5th order Butter-worth low-pass filter [14] was applied to remove the noise caused by the rubbing between the sensor and the clothing, and the phase distortion was corrected by using the filtfilt function of MATLAB to deal with the delay. The cutoff frequency of the filter was determined based on the experimental results described in the next section. To remove the bias error in the gyroscope, the average of the angular velocity at resting was calculated and subtracted from the total angular velocity signal. The data from the second to 100 frames were used to calculate the mean value at resting. In this study, the coordinate system of the six axes is defined as shown in Figure 3.

Experiment for extraction of prediction parameters
We measured 90-degree turn and steady walking respectively and compared the acceleration data and angular velocity data. The blue area in Figure 4 is the walking path presented to the subject. Six students in 20s without any gait disorder (M: 3, F: 3) walked three trials in each motion at free walking speed.

Results & Discussions
Determination of the cutoff frequency. The cutoff frequency was determined based on the experimental results to take into account the peak frequencies of acceleration and angular velocity components caused by the walking motion. In general, it is said that the step frequency (fstep) appears in the X-axis, Z-axis, and pitch-axis, and the stride frequency (fstride) appears in the Y-axis, roll-axis, and yaw-axis in the walking motion. The step is the distance between the heels of both feet, and the stride is the combined length of the left and right steps ( Figure 5).
The fstride was about 0.9 Hz (Mean±SD: 0.919 ± 0.096) and the fstep was about 1.8 Hz (Mean±SD: 1.839 ± 0.192) according to the results. There was no significant sex difference in this value (p-value = 0.566). This result falls within the range of fstride and fstep values for a typical walk [15]. Therefore, the cutoff frequency was set to 1.2 Hz for the component with a peak at fstep and 2.5 Hz for the component with a peak at fstep.

Figure 5 Definition of step and stride
Definition of the section of 90-degree turn. We defined the start and end of 90-degree turns. The right one of Figure 6 shows a graph of the relative angles of the lumbar part to the initial angle in the yaw direction during a 90-degree turn including a steady walking. This can be obtained from the BVH file output on the Axis Neuron. This data is only used to define the turn section and is not used as a prediction parameter. The minimum and maximum points in the waveform, shown as blue and green points, represent the initial contact (IC) of the right and left feet respectively. IC is defined as the moment when the foot is contacted the floor as shown in the left one of Figure 6. According to the graph of Figure 6, the subject started walking from the right leg and performed a turn motion after the left IC at the 10th step. The subject performed a steady walking from 2 seconds to 6 seconds, and after 6 seconds, the lumbar rotation decreased almost 90° before entering a steady walking again. The time at which the lumbar rotation decreased as small as left IC which is the one step before was defined as the start of the turn. Similarly, the end time of the turn was defined as the time when the lumbar rotation decreases as small as the next right IC. The range between dashed lines in Figure 6 indicates the section of turn motion.

Figure 6 Definition of the section of 90-degree turn
Extraction of prediction parameters. According to the results, we found that the preliminary motion of the 90-degree turn was characterized by the asymmetry of the body's lateral sway and the kinematic parameters caused by the rotational motion. Figure 7 shows an example of the comparison between a steady walking and a 90-degree turn on the angular velocity data at the head. In this case, the yaw and roll components of the head are extracted as prediction parameters, since their waveform is changing from 6.8 s, while the turn motion is starting at 8.2 s which is shown by the dashed line in Figure 7 (b). In the same way, we extracted the roll and yaw components of angular velocity on the upper body, and Y-axis acceleration of left and right shoulders and shanks as parameters which shows the changes of the waveform before the turn started. The list of extracted predictive parameters is shown in Table 1.
The turn motion is accomplished by slowing down the speed of motion in the straight direction and shifting the center of gravity to the new direction [16]. The significant preliminary movements were observed particularly in the angular velocity yaw and roll components of the head and upper trunk in the results. These suggest that the center of gravity shift of the upper body, including the head, occurs at the earliest stage.  Study limitations. The experimental space has no walls or screens, however, it is assumed that many of the turn motions are performed at corners in the actual situations. Thus, additional experiments in the actual environment such as corners are needed.
The age of the subjects in this study was biased and may yield different results in other age groups such as the elderly. Also, the number of participants was small, and reproducibility should be examined.

Yaw
Pitch Roll

Conclusion
In this study, we extracted effective parameters for 90-degree turn prediction, which is a major unsteady motion, using an IMU-based motion capture system to implement the motion prediction phase to the home robot in the "Biofied Building". The results showed the possibility of the yaw and roll components of the head and upper trunk angular velocity, as well as Y-axis acceleration of shoulders and shanks to be used for the 90-degree turn prediction. Our findings can contribute to the further examination of turn motion prediction. The additional consideration for reproducibility confirmation with more participants including the other age groups will be our plan.