Hierarchical Age Estimation based on Dynamic Grouping and OHRank

Abstract

This paper describes a hierarchical method for image-based age estimation that combines age group classification and age value estimation. The proposed method uses a coarse-to-fine strategy with different appearance features to describe facial shape and texture. Considering the damage to continuity between neighboring groups caused by fixed divisions during age group classification, a dynamic grouping technique is employed to allow non-fixed groups. Based on the given group, an ordinal hyperplane ranking (OHRank) model is employed to transform age estimation into a series of binary enquiry problems that can take advantage of the intrinsic correlation and ordinal information of age. A set of experiments on FG-NET are presented and the results demonstrate the validity of our solution.

1. Introduction

Human age has long been thought to be an important attribute, revealing an individual’s personal condition, anthropometric information, and social background. As a “window to the soul,” the human face can display various individual characteristics, including age. Therefore, age estimation via faces is feasible. Further, because of its many potential applications, age estimation has become a hot topic in the fields of human-computer interaction and computer vision [1]. It is well known that face aging is an unavoidable natural process. It is affected not only by internal factors, but also by external factors [2]. Thus far, age estimation using facial images remains a considerably difficult and challenging task.

Age estimation is considered a task of multi-classification [3-5], regression [6-9] or a combination of both [10, 11]. Guo et al. [3] studied age estimation across expression changes. They learnt the correlations between neutral and other expressions using a partial least squares algorithm. Then, a marginal Fisher analysis was used for discriminative mapping of aging patterns. Finally, age was estimated using support vector machine (SVM) classifier. Yan et al. [4] reported an age estimation algorithm based on a Gaussian mixture model that maximized the sum of likelihoods from spatially flexible patch features that simultaneously encoded local appearance and position information. Lu and Tan [7] devised cost-sensitive methods to project high-dimensional face and gait samples into low-dimensional subspaces. To uncover the relation between the projected features and ground-truth age values, they employed a multiple linear regression function with a quadratic model for age estimation. Suo et al. [8] also reported a novel regression formulation for mapping feature vectors to an age label. Specifically, they decomposed a facial image into a coarse-to-fine graph model. Tao et al. [10] emphasized facial shapes, which were modeled as landmarks on a Grassmann manifold. These points were then projected onto a tangent space and age was estimated by a tangent-space regression. Kohli et al. [11] used an active appearance model to describe the aging process. A test sample was first grouped into childhood or adulthood by a global classifier, following which the final age was estimated using regression functions.

Although significant progress has been achieved, the following two challenging problems need to be studied further: (1) Ignorance of the ordinal order between ages: Samples are dependent and there exists relative ordinal information between them that is considerably important to age estimation. Some work regards samples as individuals, leading to the loss of the intrinsic age correlation. Therefore, how this ordinal information may be effectively used needs to be studied. (2) Fixed age group division: The existing work divides the overall age range into several fixed age groups, such as 0 - 20, over 20 [6] or under 15, 16 - 30, 31 - 50, and over 51 years [5]. Fixed division destroys the continuity or intrinsic relationship of the aging process and in particular has a negative influence on the samples that are near the group borders. For example, if we use the group span shown in [5], the image of a 17-year-old’s face will be grouped into the second group (16 - 30), if it is classified correctly. However, compared to the span of 16 - 30, the age span of 10 - 24 may hold more similar samples. Therefore, fixing the group division beforehand may disrupt this close relation.

To utilize ordinal information between samples, some researches preserve ordinal information in feature space [12-13] and some regard age estimation as a ranking problem (also called OHRank) in decision space [14-16]. Lu and Tan [12] proposed an ordinary preserving manifold analysis method to build a low-dimensional subspace. Subsequently, a multiple linear regression model was learned to gain the relation between low-dimensional features and sample ages. Chen and Hsu [13] proposed a subspace learning method where both an aging manifold and age ranks were embedded to obtain more discriminative ranking features. Further, support vector regression (SVR) was adopted to train an age estimator. Chang et al. [14, 15] utilized an ordinal hyperplane ranking algorithm, called OHRank. Every ordinal hyperplane separated all samples into two groups according to age order. Based on the classification costs, better hyperplanes could be found. By aggregating the preference information from hyperplanes, sample ages were predicted.

Comparatively few researchers have mentioned the fixed age group problem; consequently, hardly any efforts have been taken to solve it.

We draw on the idea of OHRank and find new methods to solve this problem. In this paper, we propose a hierarchical solution consisting of age group classification and age value estimation that is also a coarse-to-fine process. The estimation of age for an unknown facial image is determined first by utilizing weighted k-NN to determine a coarse age value y, followed by the construction of a dynamic age group centering on y. Further, using an OHRank framework, we estimate the ranking order from the ordinal hyperplanes in the particular given group. By deciding whether a test sample is older than a series of ages or not, the relative ordinal information can be used efficiently. Moreover, because age value estimation is limited within a special age span, the age pairs involved in a binary inquiry will be reduced dramatically, compared to the global rank model [14].

The rest of this paper is organized as follows. Section 2 briefly overviews our proposed method. Section 3 presents our feature extraction method that combines shape and texture features. Section 4 elaborates age estimation including age group classification and age value estimation. Section 5 reports the experimental results on FG-NET, and Section 6 summarizes our work.

 

2. The Proposed Method

The framework of our proposed hierarchical method is showed in Fig. 1, which consists of a dynamic grouping-based age group classification and OHRank-based age value estimation. It infers age value using a coarse-to-fine strategy. There are four main modules in our solution: preprocessing, feature extraction, age group classification, and age value estimation.

Fig. 1.Framework of the proposed method

Preprocessing. Facial region partitioning is widely used to extract local texture features for age estimation [16]. To obtain accurate local regions and robust features, a forward-facing pose is ideal for age estimation. However, profile and other poses are unavoidable when capturing facial images. Hence, we adopt pose correction for those images with deflection angles larger than 5 degrees, by rotating them in a plane. Furthermore, a histogram normalization algorithm is applied to improve the image quality.

Feature extraction. Both shape and texture have indispensable roles in depicting the appearance of aging. In this paper, ratio features, wrinkle density (WD) features, and uniform local binary pattern (ULBP) [17] features are extracted to describe facial traits. As shown in Fig. 1, during age group classification, ratio and WD features are fused to obtain a coarse age label. During age value estimation, we combine ratio features with ULBP texture to arrive at a more exact age.

Age group classification (coarse estimation). In this step, each age label is viewed as a class, and a weighted k-NN is employed because of its simplicity and effectiveness. Then, based on the coarse estimation, we divide the test sample into a dynamic rather than fixed group [5, 6]. In this manner, the effect of the discontinuity of samples at the group border can be decreased.

Age value estimation (fine estimation). This process is based on OHRank model with a k-NN as the base classifier that predicts the age of the test sample within its specific age group. In other words, this process acts as a minor adjustment within a small age span. In addition, estimation within a particular group can also boost enquiry speed.

 

3. Feature Extraction

Studies have revealed that from infancy to adulthood, the greatest change is in craniofacial growth [1]. For example, the position of an adult’s eyes is generally higher than that of infants. In this growth period, skin wrinkles change little. Conversely, from adulthood to senior citizen, the most obvious variation is skin aging, leading to a difference in wrinkles and other skin artifacts [2, 18]. Hence, facial aging can be described as a function of facial shape and texture.

3.1 Shape features

Biologically, facial shapes change dramatically because of bone movements during the period from birth to adulthood. This change can effectively help distinguish immature faces from others [5, 16]. To reduce the influence of sample size, we utilize ratios of distances instead of distances between facial points. Fifteen facial landmarks are selected to calculate eight distances (shown in Fig. 2). Subsequently, six ratios r0~r5 are generated as follows, based on the eight distances.

Fig. 2.Eight facial distances

3.2 LBP texture features

Because of its low computational complexity and rotational invariance, LBP has become a very popular approach for extracting local texture. A traditional LBP operator works over a 3 × 3 neighborhood, using the center pixel value as a threshold [19]. Given a pixel gc and its p neighbors, the LBP descriptor is formed by following equation.

A further extension of the basic LBP operator is ULBP. ULBP introduces bit transitions (from one to zero or vice versa), and a LBP pattern is called uniform if it contains two transitions at most. Compared to the original LBP, ULBP is more robust and stable. A one-dimensional histogram feature is formed by counting the numbers of each ULBP pattern. In this paper, we set p = 8 and the dimension of the ULBP feature vector to 59.

3.3 Wrinkle density

For adults, the change in wrinkle intensity is obvious [5]. In fact, with increasing age, more intense wrinkles appear on human faces. To calculate WD, we use a Canny descriptor to obtain the edges and assume the edge pixels are wrinkle pixels.

Let I be a local M × N facial region, WDI be the WD of region I, EI be the edge image of I obtained by the Canny operator, EI(i,j) be a pixel of image EI, and SI = M × N be the area of region I. Then, the WD is given by

3.4 Feature combination

Human skin varies with aging and wrinkles usually appear around facial features such as regions of the forehead, the corners and eye bags [20]. We define five major wrinkle regions, including forehead, cheek, eye corner, and mouth corner (shown in Fig. 3). Considering the influence of out-plane deflection, only larger regions are selected from the left and right cheeks and corner regions. Thus, we finally obtain five local regions from which texture features can be extracted.

Fig. 3.Key facial regions

In age group classification, the combined shape features {r0,r1,r2,r3,r4,r5} and WD features {d0,d1,d2,d3,d4} are used, because shape features effectively distinguish juveniles from adults, and WD can roughly represent wrinkle distribution. In age value estimation, because the LBP descriptor provides more detailed information for texture description, we combine shape and ULBP texture features replacing WD with ULBP features, Hence, the feature dimensions of the coarse and fine estimation stages are 11 and 301, respectively.

 

4. Hierarchical Age Estimation

A hierarchical model is a powerful framework for age estimation that can greatly improve results. In this paper, we treat age estimation as a classification problem First, the overall age space is roughly divided into several flexible age ranges, and then a serial of binary classifiers are combined to locally approximate the exact age under an OHRank framework.

4.1 Age group classification

Age group classification is a coarse decision process that aims to group a test sample into a local age range. First, a weighted k-NN is employed to predict a possible age value. Then we use the coarse age values to build a specific age group for the test sample by a dynamic grouping method.

4.1.1 Weighted k-NN

The collection of an aging database with a wide span of ages from very young to very old and many samples for each age value is one of the most difficult tasks in age estimation. The reality is that the sample distribution for all ages in existing public aging databases is uneven. For example, the Morphy database does not include the facial images of young children under 15 years old. In the FG-NET database [21], more than half of the subjects are between ages 0 and 13. Table 1 provides the age distribution of FG-NET. Clearly, more than 85% of the samples are under 30 years old. In addition, no samples exist for some ages. This weakness can lead to the “classification preference” problem, meaning that samples with less “friend-generation” (samples belong to similar ages) are more likely to be classified into other types with more “friend-generation.”

Table 1.Age distribution of FG-NET

To handle the uneven distribution of aging databases, we employ a weighted k-NN [22] to assign higher weights to samples with smaller distances to the test sample. In this manner, the influence of sample distribution is lowered and the roles of the nearer neighbors are further emphasized.

For the selected K nearest neighbors of a sample x, K distances are calculated. Further, the distances are listed in ascending order and the ordered distances are recorded as dj(1,2,…,K) . Hence, dK is the largest distance and d1 is the smallest. The weights Wj(j = 1,…,K) are the weights of each neighbor pair and are evaluated by

Assume{xj,yj}(j = 1,…,K) are theKnearest neighbors, xj is a sample, yj is its age value. By sorting Y = {y1,y2,…,yK} in ascending order, we can obtain a non-repeating age label vector Y' = {y'1,y'2,…,y'C}. Given a new test sample x, the weight that belongs to its age label y'j is

Clearly, if we assign Wj = 1 in (4), (5) is equivalent to traditional k-NN. We then sort S = (s1,s2,…,sC) in descending order to obtain a new vector S' = {s'1,s'2,…,s'C}. If K' is the number of top weights in S, the coarse age label yw is calculated as

Clearly, if K'=1, (6) can be written as

In other words, this weighted k-NN decision is equivalent to the maximum weight decision.

4.1.2 Dynamic grouping

Fixed-group divisions, such as grouping samples in FG-NET into five age groups (0 - 9, 10 - 19, 20 - 29, 30 - 39, and 40 - 69), is apt to cause “mis-classification” for those samples near the border of an age group. For instance, a 19 year-old sample is very likely to be classified into a 20 - 29 group if its features are more similar to those of 20-year old samples. We instead suppose that it is better to build an estimator for 19-year-old samples using samples within the age region of 16 - 22, instead of 10 - 19. Overall, the fixed-grouping technique by artificial segmentation beforehand may destroy the continuity or intrinsic relationships of the aging process.

To handle the negative influence of fixed grouping, we set an individual age group for each test sample based on the output yw of the weighted k-NN. We build a fixed-length local range for any yw. Let LL and UL be the ages of the youngest and oldest samples in the aging database respectively, and 2Lg + 1 be the length of a local region. The local range R (shown in Fig. 4) for a yw-year-old sample is calculated as

Fig. 4.Fixed span for local Range R

The optimal value of Lg is determined by experiments in Section 5.2.

4.2 Age value estimation based on OHRank

Human aging is a typical ordinal process. The purpose of age estimation is to label a face image automatically with an exact age value or age group. In fact, traditional solutions regard age estimation as a classification or regression problem. The former treats each an age label as a class. Given a new sample, the age estimation algorithm decides to which class it belongs. Obviously, classification approaches are not concerned with the inherent relationship between samples of different labels. Regression methods infer age label using a function that is learnt from a training set. Because of the insufficient number of training samples, an overfitting problem exists in the learning process. In addition to this, regression is toward the mean property of least squares estimation (LSE) [23].

To avoid overfitting and preserve the relative order of human ages, we regard age estimation as a classification problem and utilize the OHRank framework [15] to predict the age label. A series of k-NN classifiers are used to enquiry if a sample belongs to a labeled class or not. To train each k-NN classifier, the overall training set is divided into two subsets. Compared with traditional classification problems, each classifier has more data available for training. This also means that the OHRank can reduce the influence of an uneven distribution of the training set.

Assume is a set containing all training images within group R, denotes a subset with age labels more than c, represents the subset with age labels no more than c and are the non-repeating age labels in ℜ. Our ranking k-NN algorithm is described in Table 2.

Table 2.Algorithm for ranking k-NN

To reduce the influence of age distribution, the neighbor number K in Step 1.2 is not a fixed value. We assign it according to the age label yw, which decides the local region R as well as its sample distribution. Table 3 shows the values of K in our proposed OHRank method, whose performance has been tested experimentally.

Table 3.Values of K in OHRank method

 

5. Experimental Performance

As a public facial aging database with 68 landmarks for each image, the FG-NET aging database has been widely used in age estimation research. It contains 1,002 samples of 82 people and covers an age range of 0 to 69. The Mean Absolute Error (MAE) [24, 25] and Cumulative Score (CS) [26] are adopted to test our solution under the Leave-One-Person-Out (LOPO) strategy.

5.1 Parameter evaluation of weighted k-NN

We performed the following experiments (E1 and E2) to compare the performance of weighted k-NN with different parameters of Wj and K'.

E1 (non-weighted k-NN): weights to all neighbors were fixed (Wj = 1) and then MAE was tested with different K'.

E2 (weighted k-NN): different weights were set using Equation (5) for different neighbors, then MAE was tested by varying K'.

Because weighted k-NN is used in age group classification, MAE was tested only in terms of the output yω of weighted k-NN, and not the final ranking k-NN output. Table 4 shows how the MAEs vary with K'.

Table 4.MAEs of weighted and non-weighted k-NN

Table 4 reveals that: (1) MAE varies with different values of K' both for traditional k-NN (non-weighted k-NN) and weighted k-NN. When K' is set between three and six, the MAEs are lower. For both E1 and E2, the best results are achieved when K' = 4. (2) Weighted k-NN performs better than traditional non-weighted k-NN.

Because the best performance can be obtained by K' = 4 with a weighted k-NN, we used this parameter in the following experiments.

5.2 Age span evaluation of dynamic group

Here, we compare the effect of different ages from 1 - 12 on MAE. The results are shown in Fig. 5.

Fig. 5.MAEs of yw under different lg

Fig. 5 shows that different spans generate different MAEs. When lg＜ 6，MAEs vary only slightly. In contrast, when lg ≥ 6, MAE increases. The lowest MAE is obtained when lg = 4.

In addition, we attempted varying lg in the dynamic grouping process; however, the performance is not as good as for a fixed lg.

5.3 Contribution of different components

Four components contribute to the final performance weighted k-NN, dynamic grouping, ranking k-NN, and hierarchical structure. They work together to improve the performance of our method. To evaluate their contribution to age estimation, we designed a set of experiments to test the MAEs by removing or changing one component individually. The results are shown in Table 5 and Fig. 6, and the methods were as follows.

Fig. 6.CS curves of the algorithms with different components

The whole method: This method includes all four components detailed in Section 4.

Method with traditional k-NN replacing weighted k-NN (method 1): The weights for all neighbors are set to one. This method is used to test the effect of weighted k-NN on the final performance.

Method with k-NN replacing ranking k-NN (method 2): This experiment is used to test the effect of ranking k-NN on the final performance.

Method with fixed grouping (method 3): Dynamic grouping is replaced by fixed groups (0 . 4, 5 . 9, 10 . 14,…, 40 . 44, 45 . 50, 50 . 69). This experiment is used to test the effect of dynamic grouping.

Method with weighted k-NN only (non-hierarchical, method 4): The fine estimation step is removed from the hierarchical framework. Hence, the remaining procedure only estimates coarse age. This experiment is designed to test the role of fine estimation (age value estimation within a local age group).

Method with ranking k-NN only (non- hierarchical, method 5): The coarse estimation step is removed from the hierarchical framework. This experiment is designed to test the role of the coarse estimation step (age group classification).

As shown in Table 5, compared with the whole method, the MAEs of non-hierarchical methods increase quickly, from 4.73 to 7.57 and 7.29. Even the performances of the other hierarchical methods without some component are better than the performance of the non-hierarchical methods. These results strongly indicate that the hierarchical framework can greatly improve the accuracy of age estimation because the different steps have distinctive roles. The first step (coarse estimation based on weighted k-NN and dynamic grouping) can help decrease the number of “confused” samples, and the second step (age value estimation) helps to adjust the age value to be more accurate within a given age span.

Table 5.MAEs of algorithms with different components

Among the hierarchical methods, the methods with k-NN replacing weighted k-NN and ranking k-NN show an increase in the MAEs. This proves that weighted k-NN and ranking k-NN are better than traditional k-NN for age estimation, because weighted k-NN considers the roles of different neighbors and the OHRank framework takes advantage of the ordinal information of ages.

The method with fixed grouping also performs a little worse than our system. This is because a fixed-group system is more likely to classify a sample into a “confused” adjacent class. Moreover, it also loses some continuous information for those samples near the group border.

From the above experiments, we can conclude that all components, especially the hierarchical structure, have an irreplaceable impacts on the final performance. Only by combining all of them can the best performance be achieved.

Fig. 6 displays the CS curves for different error levels on the FG-NET database for the six methods tested. The non-hierarchical structure methods yield the lowest accuracy among the methods compared. The curves of methods 2 and 3 are very similar. As the error level increases, our method is more efficient than all the others, yielding a higher CS.

5. 4 Performance comparisons with state-of-the-art algorithms

Table 6 shows the MAE comparisons of our work with some state-of-the-art methods.

Table 6.MAE Comparison for different methods

From Table 6, we observe that our algorithm performs better than most state-of-the-art approaches. However, it is slightly inferior to OHRank-AAM [15] (MAE = 4.48) and PAR [13] (MAE = 4.56) and not as good as MFOR [29] (MAE = 4.25).

We also performed experiments to test the performance of our system on different age ranges. Table 7 shows the results.

Table 7.Results on different age ranges

Table 7 clearly shows that the volume of samples within an age span affects the final predicted results. As the sample number of subsets decreases, MAE increases and estimation accuracy declines accordingly. This shows that the quality of the aging database affects performance significantly. A good aging database should not only have good quality pictures, it should also have many samples of each age.

 

6. Conclusions

In this paper, we used a hierarchical method to estimate age from facial images. In contrast to previous work with fixed grouping, we introduced a dynamic grouping technique, in order to avoid discontinuity between different groups. In addition, a kNN-based OHRank model (ranking k-NN) was used to conduct age-value estimation. It used intrinsic ordinal information between samples and boosted the estimation performance. Moreover, weights were assigned for different neighboring samples to strengthen their roles. Our method achieved a MAE of 4.73 when it was tested on FG-NET; this is better than most of the state-of-the-art approaches.

With the rapid development in mobile visual researches [41, 42], for future work, we will consider the significant influences of facial occlusion, expression, and pose, especially toward mobile applications. Simultaneously, we will attempt to refine our algorithm for age group estimation, because its result has a crucial effect on the overall system performance. Moreover, extracting more discriminative visual features is also an important goal, particularly taking into account the idea of discriminative vocabulary coding technique [43, 44] to transform the feature set into a concise visual codebook.