An Evolutionary Optimization Approach for Optimal Hopping of Humanoid Robots

Abstract

This paper proposes an evolutionary optimization approach for optimal hopping of humanoid robots. In the proposed approach, the hopping trajectory is generated by a central pattern generator (CPG). The CPG is one of the biologically inspired approaches, and it generates rhythmic signals by using neural oscillators. During the hopping motion, the disturbance caused by the ground reaction forces is compensated for by utilizing the sensory feedback in the CPG. Posture control is essential for a stable hopping motion. A posture controller is utilized to maintain the balance of the humanoid robot while hopping. In addition, a compliance controller using a virtual spring-damper model is applied for stable landing. For optimal hopping, the optimization of the hopping motion is formulated as a minimization problem with equality constraints. To solve this problem, two-phase evolutionary programming is employed. The proposed approach is verified through computer simulations using a simulated model of the small-sized humanoid robot platform DARwIn-OP.

1. Introduction

Research on humanoid robots has made rapid progress in dexterous motions along with hardware development [1-5]. Stable walking and running motions are especially representative research issues in humanoid robotics. The running motion is composed of flight phases and support phases, thus the jumping motion is a very important issue for the flight phase. In other words, the jumping motion is the prerequisite for the running motion. As research on jumping and hopping motions of the humanoid robot, running pattern generation based on a zero moment point has been studied by expanding the concept of the zero moment point to running control [6, 7]. To maintain dynamic stability while hopping or running, online balance controllers have been proposed [8]. Moreover, approaches have been proposed to generate a hopping trajectory for humanoid robots with passively compliant structures [9-11]. Although the jumping and hopping motions of the humanoid robot have been successfully implemented in the previous researches, it is necessary to formulate the hopping motion of the humanoid robot as an optimization problem for optimal hopping. The optimization problem can be solved by evolutionary algorithms [12, 13], which have been applied to walking [14, 15], footstep planning [16, 17], and posture control [18] of the humanoid robot.

For optimal hopping, there should be complex considerations, along with guaranteed stability, while hopping. While hopping, rocking back and forth of the humanoid robot occurs and should be minimized for stability. In addition, oscillation of the ground reaction forces (GRFs) on the feet should be minimized, and the actual flight time should be maximized for efficient hopping. This complicated optimization problem can be solved by the evolutionary algorithm. This paper proposes an evolutionary optimization approach for optimal hopping of humanoid robots. In the proposed approach, the hopping trajectory is generated by a central pattern generator (CPG). The CPG is one of the biologically inspired approaches, and it generates rhythmic signals by using neural oscillators [19, 20]. During the hopping motion, the disturbance caused by the GRFs is compensated for by utilizing the sensory feedback in the CPG. Posture control is essential for a stable hopping motion. Therefore, a posture controller is utilized to maintain the balance of the humanoid robot while hopping. In addition, a compliance controller, which utilizes a virtual spring-damper model for the pitch motion of the feet, is applied for stable landing. The optimization of the hopping motion is formulated as a minimization problem with equality constraints. To solve this problem, two-phase evolutionary programming (TPEP) is employed [21]. The proposed approach is verified through computer simulations with a simulated model of the small-sized humanoid robot platform DARwIn-OP.

This paper is organized as follows. Section 2 describes the hopping pattern generation. The CPG structure is reviewed and the hopping pattern generation using the CPG is explained, along with the sensory feedback of the CPG. Section 3 describes the posture controller and the compliance controller for hopping control. Section 4 describes the evolutionary optimization with the objective function design. The validity of the proposed approach is verified through computer simulations, as described in Section 5. Lastly, conclusions are addressed in Section 6.

 

2. Hopping Trajectory Generation

2.1 CPG structure

The sequence of extension and flexion of joints enables rhythmic biological locomotion. When one side of the body part extends, the other side flexes, and the extension and flexion continue alternately during rhythmic motion. For the modeling of this biological system, Taga devised the CPG structure [19]. In the CPG, multi-dimensional rhythmic signals are generated endogenously without a rhythmic sensory or central input, and these signals are adapted to environmental perturbations using sensory feedback. To generate the rhythmic signals, Matsuoka’s neural oscillator is utilized, the neuron of which is described as follows [20]:

with

[u]+ = max(0, u)

where the subscripts i and j denote the indexes of the neurons, ui and vi are the inner state and self-inhibition state, respectively, and c is the external input signal that affects the output amplitude. 𝛽 and wij denote respectively the self-inhibition weight and the connecting weight which decides the phase difference between the i-th and j-th neurons. fi is the sensory feedback signal. τ and τ ' denote the time constants that have an effect on the shape and period of the output signal, and n is the number of neurons. Fig. 1 shows the neuron structure. The detailed description of the parameters of the neuron is provided in [20]. In [20], the rhythmic control in several neural oscillators consisting of two to six neurons was discussed and by the computer simulation, the mathematical conditions for the neural oscillators to produce a stable rhythmic signal were obtained.

Fig. 1.Neuron structure.

In the CPG structure, rhythmic signals are generated by Matsuoka’s neural oscillators (in (1)), each of which consists of an extensor neuron (EN) and a flexor neuron (FN). These are interconnected in the same neural oscillator and produce flexion and extension signals, as shown in Fig. 2. Through the effects of this relationship, rhythmic signals are obtained as follows [19]:

Fig. 2.Neural oscillator structure for CPG.

where the subscripts e and f denote the EN and FN, respectively. wef and wfe are the connecting weights between the EN and the FN, and ocpg is the output signal.

2.2 Hopping trajectory generation using CPG

For a hopping motion, the vertical center of mass (COM) trajectory of the humanoid robot is generated using the CPG, as shown in Fig. 3. The COM trajectory zcom is computed as follows:

Fig. 3.Hopping trajectory generation using CPG.

with

where Zc is the initial COM height. Aflight and Asupport denote the amplitude scaling factors for the flight phase and the support phase, respectively. In the neural oscillator, when the magnitude of the output signal is negative, ατ and ατ ' are used instead of τ and τ ' , respectively, to adjust the period of the output signals, where α is the scaling factor.

To compensate for the disturbance from the ground during the hopping motion, sensory feedback of the neural oscillator is utilized. If the robot stands motionlessly, the sum of the GRFs on the feet is equal to the robot’s weight. On the other hand, the sum of the GRFs oscillates around the value of the weight of the robot during the hopping motion. Thus, the following sensory feedback is designed to improve the robot’s stability during the hopping motion by minimizing the oscillation of the GRFs on the feet:

where kf denotes the scaling factor, Fl and Fr indicate the GRFs on the left and right feet, respectively, and M is the mass of the humanoid robot.

 

3. Hopping Control

3.1 Posture controller

A posture controller is utilized to maintain the balance of the humanoid robot while hopping. For the posture controller, the humanoid robot is simplified as shown in Fig. 4, and the ankle pitch angle is controlled as follows:

Fig. 4.Structure for posture controller.

with

where is the reference ankle pitch trajectory by the hopping pattern generation and denotes the modified reference ankle trajectory by the posture control. 𝜽desired is the desired body angle of the robot and 𝜽body is the actual body angle of the robot, which is measured by the posture sensor. The posture controller uses a proportional-derivative (PD) controller. kp and kd are the coefficients of the PD controller.

3.2 Compliance controller

When the humanoid robot is hopping, either the toe or the heel of the robot could contact the ground sooner while landing, which would cause the robot to become unstable. To solve this problem, a virtual spring-damper model is used, as shown in Fig. 5. A rotational virtual spring-damper model is utilized for the pitch motion of the foot, and the rotational foot displacement Δ𝜽ankle is governed by the following equation:

Fig. 5.Virtual spring-damper model for foot.

where kvsd and bvsd are the spring and damper coefficients, respectively. τankle is the pitch torque on the foot, and is obtained as follows:

where lf and lb denote the front and back foot lengths from the center of the foot, respectively. fi (i=1, 2, 3, and 4) are the GRFs measured by the force sensing resistors (FSRs). The sole of each foot is equipped with four FSRs.

 

4. Evolutionary Optimization

Optimization of the hopping motion is formulated as a constrained optimization problem to generate the desired output signal from the CPG, and ensure a stable and efficient hopping motion of the humanoid robot. To solve this problem, the objective function is designed, along with the equality constraints. To generate the desired output signal from the CPG, the time period of the trajectory generated by the evolutionary optimized CPG should satisfy the equality constraints on the flight and support times while hopping. Note that the trajectory generated by the evolutionary optimized CPG oscillates around zero. When the magnitude of the trajectory generated by the evolutionary optimized CPG is positive, the corresponding time Tp should be equal to the flight time of the hopping, Tflight. In contrast, when the magnitude is negative, the time Tn should be equal to the support time Tsupport. This is because the vertical COM trajectory should move up during the flight phase and down during the support phase. These constraints make the magnitude of the trajectory positive and negative during the flight and support phases, respectively. When the magnitude of the trajectory reaches the maximum value, the corresponding time Tmax should be equal to half of Tflight. Similarly, when it reaches the minimum value, the time Tmin should be equal to half of Tsupport. The amplitude of the trajectory obtained by the evolutionary optimized CPG at each period either increases, decreases, or remains constant with respect to time, according to τ and τ ' . Since the hopping of the humanoid robot repeats the same motion, the amplitudes of the trajectory at every period should be equal to each other. If the amplitudes of the trajectory in the initial and final periods, Ai and Af, are equivalent, the amplitudes of the trajectory at every period are equal. In addition, for normalization, the amplitude of the trajectory should equal one. For stable hopping, the rocking back and forth of the humanoid robot while hopping should be minimized. In addition, the oscillation of the GRFs on the feet while hopping should be minimized. For efficient hopping, the actual flight time should be maximized.

Considering the objectives and constraints mentioned above, c, τ , τ ' , and α in the neural oscillator, Aflight and Asupport in the hopping trajectory generator, kf in the sensory feedback of the neural oscillator, kp and kd in the posture controller, and kvsd and bvsd in the compliance controller are optimized by TPEP to minimize the following objective function, while satisfying the equality constraints:

with

subject to constraints

where kpos, kgrf, kflight,1, and kflight,2 are scaling factors. In the objective function f, the first term on the right-hand side is the sum of the posture errors between the desired and actual body angles of the robot while hopping. This minimizes the rocking back and forth of the robot while hopping. The second term is the sum of the difference between the GRFs and the robot’s weight while hopping. This minimizes the oscillation of the GRFs on the feet while hopping. The third term is the difference between the scaling factor and the sum of the actual flight time while hopping. This makes the robot hop with the maximum flight time. The fourth term P is a penalty value, which is assigned when the robot falls down while walking, because the constrained optimization problem is a minimization problem. To solve the above constrained optimization problem, TPEP is employed. TPEP is highly suitable for problems with diverse types of constraints and gives a better result with respect to the solution accuracy, convergence stability, and computation time [21].

 

5. Computer Simulations

5.1 Robot model

To verify the performance of the proposed method, we used a simulated model of the small-sized humanoid platform DARwIn-OP, depicted in Fig. 6(a). The height and mass of this platform are 45.5 cm and 2.8 kg, respectively. The front and back foot lengths from the center of the foot are 5.2 cm. The simulated model of DARwIn-OP was produced using Webots, which is a 3-D robotics simulation software that enables users to perform physical and dynamical simulations [22]. In other words, the physical and dynamical characteristics of the real DARwIn-OP were reflected in the evolutionary optimization of the hopping motion using the simulated model of DARwIn-OP. The model is depicted in Fig. 6(b). To measure the GRFs on the feet, the sole of each foot was equipped with four FSRs. A posture sensor, which is composed of an accelerometer, a gyro, and a complementary filter, was installed on the robot’s torso to measure the body angle of the robot.

Fig. 6.(a) DARwIn-OP. (b) Simulation model.

5.2 Evolutionary optimization results

The initial inner states and self-inhibition states ue, uf, ve, and vf were obtained from computer simulation as 0.484, 0.508, 0.240, and 0.752, respectively, in order to force the initial value of zcpg to zero. The connecting weights and self-inhibition weight wef, wfe, and 𝛽 were set as 1.5, 1.5, and 2.5, respectively, to cause the phase difference between [ue]+ and [uf]+ to equal π [20]. The initial COM height Zc was set to 18.35 cm and the desired body angle of the robot was set to 5.0° at which the front and back GRFs on the foot were equal to each other. The flight time and support time were set to 0.1 s and 0.2 s, respectively. The scaling factors kpos, kgrf, kflight,1, and kflight,2 were set as 10.0, 0.007, 1000.0 and 100.0, respectively, to make the scales of the terms in the objective function similar, which ensures similar priorities for the three objectives in the hopping motion. That is, the hopping motion was optimized with similar priorities for the minimization of the rocking back and forth of the robot, the minimization of the oscillation of the GRFs on the feet, and the maximization of the actual flight time while hopping. P was set to +∞. The population size and the number of generations for TPEP were set to 25 and 200, respectively.

Fig. 7 shows the variation of the objective function value with respect to the number of generations during the evolutionary optimization using the simulation model of the DARwIn-OP. The evolutionary optimization was executed on PC with 4.0 GHz quad-core processor and it took about 7.5 hours. The optimized parameters afforded by TPEP are given in Table 1. Consequently, Tp and Tn were obtained as 0.1 s and 0.2 s, respectively, and were equal to the flight and support times Tflight and Tsupport, respectively. Tmax and Tmin were obtained as 0.05 s and 0.1 s, respectively, and were equal to half of Tflight and Tsupport, respectively. Ai and Af had the same value of 1.0 cm. Fig. 8 shows the vertical COM trajectory of the humanoid robot for the hopping motion generated by the evolutionary optimized CPG.

Fig. 7.Variation of objective function value with respect to the number of generations.

Table 1.Optimized parameters generated by TPEP

Fig. 8.Vertical COM trajectory of humanoid robot for hopping motion, generated by evolutionary optimized CPG.

5.3 Hopping simulation results

Dynamic hopping simulations were performed to verify the proposed approach. Fig. 9 shows the posture error trajectory between the desired and actual body angles of the robot while hopping without and with the posture controller. At this time, the sensory feedback in the CPG and the compliance controller were working. As seen in the figure, the posture error gradually increased without the posture controller, and the robot eventually fell down. On the other hand, with the posture controller, the posture error was minimized by the evolutionary optimization and it varied between -2.0° and 2.0° while hopping. Thus, the robot was able to maintain its balance while hopping because of the posture controller.

Fig. 9.Posture error trajectory between desired and actual body angles of robot while hopping: (a) without and (b) with posture controller.

The measured GRF trajectory while hopping without and with sensory feedback in the CPG can be examined in Fig. 10. During this simulation, the posture and compliance controllers were working. As shown in the figure, without sensory feedback in the CPG, there were two large peaks at the beginning and end of the support phase. The first peak was caused by the impact at touch-down, and the second peak came from the impact at lift-off. Because of these peaks, the GRF oscillated during the support phase, and the sum of the differences between the GRFs and the robot’s weight while hopping, fgrf, was 86628.45 N for fifty hopping steps. On the other hand, with sensory feedback in the CPG, the peak at the beginning of the support phase was relatively reduced compared with the result without sensory feedback in the CPG. In addition, the peak at the end of the support phase was almost entirely removed, and consequently, the oscillation of the GRFs on the feet while hopping was minimized (fgrf =69347.14 N for fifty hopping steps).

Fig. 10.Measured GRF trajectory while hopping: (a) without and (b) with sensory feedback in CPG.

Fig. 11 shows snapshots of the hopping simulation. As shown in the figure, the humanoid robot was able to hop successfully, with an actual flight time of 0.095 s. The actual flight time was maximized and was almost equal to the desired flight time of 0.1 s. In addition, the hopping simulations on the inclined terrain were performed. As a result, the humanoid robot was able to hop on the inclined terrain and the allowable inclination was from -3.0° to 6.0°. Because the hopping motion was optimized on the flat terrain, the allowable inclination was limited.

Fig. 11.Snapshots of hopping simulation: (a) Beginning of support phase; (b) Support phase; (c) Lift-off; (d) Flight phase; (e) Touch-down.

 

6. Conclusion

The main contribution of this paper is a novel evolutionary optimization approach that can be applied to the optimal hopping of humanoid robots. In the proposed approach, a hopping trajectory generation method by the CPG with sensory feedback was devised. Moreover, a hopping controller composed of a posture controller and a compliance controller was utilized for stable hopping. The proposed approach formulated the hopping motion as a constrained optimization problem, which was successfully solved by TPEP. The effectiveness of the proposed approach was demonstrated through computer simulations. The simulation results indicate that the proposed approach enables a humanoid robot to hop with minimum posture error, minimum oscillation of the GRFs on the feet, and maximum flight time.