A Study on Overcoming Disturbance Light using Polarization Filter and Performance Improvement of Face Recognition System

Abstract

The performance of the facial recognition system is determined by many technical factors. Further, most of the technical factors have been realized or are still in continued research. The recognition rate has a great influence on performance not only by technical factors but also by other factors. However, researchers are trying to improve the recognition rate by focusing only on technical factors. The mechanism of recognizing is to compare a face image obtained by photography to an already stored face image and determine the score of the similarity. However, if the photographed image is damaged by external light, even a system with a good algorithm will fail to recognize it. Therefore, it is important to prevent the disturbance of light entering from the outside, so it should be blocked, but the camera will not work without light. Thus, it is proposed that a method to secure the external light but block the disturbance of light that affects photography. A method of blocking disturbance light is to use a polarization filter. There are three polarization methods: circular polarization, linear polarization, and elliptical polarization. In this paper, an experiment was performed to overcome disturbance of light using only a circularly polarized filter. In addition, a lighting system that reproduces disturbance light was provided for the experiment, and light of varying intensities and angles was installed to affect the face recognition camera. As a result of actual application, it was determined that a very improved recognition performance in various disturbance light environments.

I. INTRODUCTION

The first step of face recognizing is to find a face in an image. If there is a face in the image, it finds the position and size of the face and determines whether it is in recognizable or not. If recognizable, finding the 'eye' on the face takes priority over everything else. This is the beginning of face recognition. Face recognition methods can be classified into a knowledge-based method, a feature- point based method, a template-matching method, and an appearance-based method [5], [10], [11], [12], [13].

Because face recognition technology has been developed for a long time, its performance is very efficient. However, the light disturbance is a typical external factor that significantly reduces the performance of the face recognition system.

The final goal of this study is to make it possible to recognize faces by blocking or alleviating various types of light disturbance entering from outside of the recognition camera. Light disturbance refers to light that damages the image taken by the face recognition camera. There are a variety of forms, specifically reflective light, direct sunlight, scattered light, etc. that are classified as light disturbance. As shown in Fig. 1, if light disturbance enters the face recognition camera, it is not able to recognize and fails. Therefore, the blocking of light disturbance enables the face recognition system to be installed in various environments, thereby improving usability.

Fig. 1. Original image and damaged image by disturbance.

In order to achieve the goal, an experiment was conducted using a polarization filter. The polarization filter is attached to the lens part of the recognition camera to filter out the light disturbance to receive only the desired light. Therefore, the polarization filter is used to obtain polarization in the desired oscillation direction.

II. GOAL AND ANALYSIS

2.1. Face Recognition System

In this study, face recognition technology is used as a feature-point extraction method. This method is the most widely known and used technology among facial recognition technologies. It is also well-known to have the highest recognition rate and accuracy.

This method captures the feature points distributed on the face image for recognition and compares it with the feature points of the stored face image in advance. At this time, about 120 to 180 number of feature points are extracted.

A certain threshold is used to determine the recognition acceptance. In this study, the threshold value is 65, which means the match rate. The larger the number, the higher the accuracy but the lower the recognition rate, and the smaller the number, the higher the recognition rate but the lower the accuracy. Set to the threshold of 65, the face recognition rate is 99.9996%, which is the optimal boundary between the recognition rate and accuracy.

‘OpenFace’ (NeoSecu, Korea) was used as a face recognition system. This product was developed in Korean technology and as already described, feature point extraction, which is a state-of-the-art face recognition method, has been applied and is already widely used commercially, such as being applied to various banking applications. It also has world-top-class accuracy.

Fig. 2. Feature points extraction method.

2.2. Obstruction of light

Light disturbance is a headache when acquiring an image for face recognition. Light scattering occurs due to light disturbance which frequently occurs when the light source is directly photographed, or strong incident light flows into the lens. Light scattering makes the acquired image messy and damages the original form, eventually leading to a failure of recognition [2].

Light scattering is called by various names; depending on its shape, it is called Flare, Halation, Glow, Haze, Starbuster et al. The main cause of light scattering is related to the lenses, but it is caused by reflection and diffusion [4].

Flare often occurs in light passing through small slots. Accurately, flare does not occur in small slots or holes, but in all boundary surfaces where light is blocked. Therefore, flare occurs in cameras with small lens diameters such as face recognition cameras. In addition, the shape of the aperture is not completely circular, or starbuster occurs due to the problem of the apertures blade [1].

Fig. 3 shows the damaged image when flare of light occurs.

Fig. 3. Damaged image state when flare of light occurs.

2.3. Purse of Study

Light disturbance has a fatal effect on the face recognition system. Until now, many face recognition technology researchers have solved the problem of light disturbance by manipulating the acquired image or improving the environment of the device installation location. This method is not a direct and fundamental problem-solving method.

Therefore, three solutions were proposed to solve the fundamental problem. The method using a polarization filter is one of the three solutions.

Polarization technology has been applied to various industries so far with LCD as one of the typical applications. It can therefore be surmised that LCD technology would not have existed without polarization technology [3].

Thus, the purpose of this study is to suggest a fundamental method to improve the performance of face recognition using polarizing film.

2.4. Concept of polarization

The phenomenon that some energy is propagated to another place is called 'wave'. Because light is also a wave, light energy is carried in waves and moves to other places. Light is a transverse wave with a direction of progress and a direction of vibration perpendicular to each other. Fig. 4 illustrates the direction of light vibration and the direction of propagation.

Fig. 4. Direction of light vibration and the direction of propagation.

Light from light sources such as sunlight, lamps and candles is a package of light that vibrates in all directions. Only the components of light that vibrate in one direction can be filtered out among the light component vibrating in many directions. This filtered light is called polarized light or polarization.

In the polarizing filter, pillars for blocking light are closely arranged in one direction like a comb. Fig 5 is an illustration of vertical and horizontal pillars of a polarization filter. There are two photographs in Fig. 6. Fig. 6(A) is seen without polarization; Fig. 6(B) is seen through polarization. As shown in Fig. 6(B), the light is not completely blocked, but the intensity of light is reduced compared to Fig. 6(A). This is because only the light vibrating in a parallel direction to the pillar of the polarization filter has passed.

Fig. 5. Vertical and horizontal pillars of a polarization filter.

Fig. 6. Without polarization filter and with polarization filter.

The light blocking state by angle was illustrated in Fig. 7 while rotating one film in two overlapping polarization films. Fig. 7(A) shows that two polarization films overlap in the same direction, and Fig. 7(B) shows a 45-degree rotation of a film. Fig. 7(C) shows a film rotated 90 degrees. If Fig. 7(A) illustrates that only the vertical vibrating light has passed, Fig. 7(C) shows that all vertical and horizontal vibrating light could not pass.

Fig. 7. Light blocking by angle.

There are three types of polarization: linear polarization, circular polarization and elliptical polarization. The above description is related to liner polarization because it is a prototype of polarization technology.

In this study, the experiment is performed with circular polarization instead of linear polarization. The circular polarization means that the wave of light is twisted in the clockwise or counterclockwise direction, as shown in Fig. 8, and is mainly used in OLEDs.

Fig. 8. Circular polarization.

2.5. Prediction of problem solving

It is well known to professional photographers to solve the light disturbance problem using polarization filters. However, there have been no cases of studies or experiments that have solved the light disturbance problem in the face recognition system using polarization filters.

There are many reasons, but the law of demand and supply seems to have worked because facial recognition has not yet become commonplace. Prediction by the theoretical basis of polarization and the empirical basis of professional photographers is very likely to succeed in the experiment.

In the case of linear polarization, many parameters were obtained by experimenting with two filters according to the angle, but it is difficult to apply them easily and simply. However, circular polarization has a complex function different from linear polarization, so it is expected to solve the light disturbance problem.

Ⅲ. PLANNING OF EXPERIMENT

3.1. Experiment space planning

Polarization experiments to improve the performance of the face recognition system must have lighting conditions that can reproduce disturbed light. Therefore, it is a requirement to prepare the lighting as shown in Fig. 6.

Furthermore, by setting the face recognition terminal, the recognition threshold score can be displayed on the screen in real time.

The preparation of circular polarization and linear polarization filters is made when each polarizing film is prepared in various specifications and attached to the slide glass. It is essential to secure a space for the polarization experiment and mark the distance on the floor. A computer must be prepared so that it can perform face recognition in real time and record its score. The experiment space configuration is illustrated in Fig. 9.

Fig. 9. Experiment space plan.

3.2. Lighting system

Light disturbance refers to light incident from the light source to the camera in the forward direction, light from the light source reflected by the reflector, and light incident from the reflector and scattered from the reflector.

Specific examples of the light disturbance are classified into DC light disturbance such as sunlight and incandescent light, and AC disturbance light such as an inverter fluorescent light and an LED light. In particular, the AC light source can have a similar effect to sunlight because the photoelectric sensor has a frequency component similar to the self-emission period required for detection. Disturbance light that affects the facial recognition system is often sunlight. Accordingly, it is effective to bring the experimental conditions most similar to sunlight, so the AC light source is most suitable.

Therefore, as shown in Fig 6, two sets of reflective continuous light are prepared and each of them is equipped with two light sources to secure a total of four light sources.

It should be also in the form of an umbrella that can be adjusted in height, angle of up and down and rotation angle. Each umbrella has two light sources and each light source is numbered, as shown in Fig. 10, to distinguish the light sources and form the conditions for the light disturbance.

Fig. 10. Lighting system.

3.3. Polarization filter

The polarizing film for fabricating the polarizing filter should be an anti-reflective adhesive type. As shown in Fig. 11, the basic structure of the polarizing film is a six-layer structure with a polarization layer (PVA: Poly Vinyl Alcohol) at the center and a protective layer (TAC: Tri Acetyl Cellulose) on the top and bottom. Then, an adhesive and surface protection film are added.

For the experiment, prepare a circular polarization filter and a linear polarization filter respectively. Circular polarization does not have directionality, but linear polarization has directionality, so it is necessary to distinguish and attach the film's attachment surface [6], [8].

Prepare a laboratory glass slide to attach the film. Glass slides are positively charged and uncoated, so choose a product with improved adhesion.

Fig. 7. Polarization film.

Fig. 8. Polarization filter.

If the slide thickness is not constant, polarization errors may occur due to chromatic aberration of the glass thus the thickness should be about 0.1 to 0.2㎜. Fig. 12 shows the polarizing film attached to the slide. The slide is equipped with linear polarization and circular polarization according to the specifications of each, and the circular polarization filter is selected from 20 to 50 ㎛ and the linear polarization filter is selected from 20 to 50㎛ [7], [9].

Ⅳ. EXPERIMENT

4.1. Illuminance measurement by lighting conditions

Before the experiment, the illuminance was measured for fifteen different lighting conditions. There are four types of lighting, including experimental space lighting. Fifteen lighting conditions were created with the lighting in the experiment space turned on. Table 1 shows the measured illuminance values for each lighting condition.

Table 1. Illuminance values for each lighting condition.

In Table 1, ‘LgC’ refers to 15 lighting conditions, ‘ExS’ is the lighting of the experimental space, and ‘Lg’ is the lighting fixture number. ‘1’ means light on, ‘0’ means light off.

4.2. Experimental setup

The experimental space was already constructed as designed in Fig. 9 and a face recognition system and lighting devices were installed, and each standing distance was marked on the floor. Fig. 13 shows the actual installed experimental system.

Fig. 13. Experiment space and face recognition device.

The experiment is to be performed with the lighting turn on. In addition, a computer was installed for real-time recording of experimental data. The recognition score is displayed on the screen of the face recognition system every time it is recognized.

4.3. Experiment method

As shown in Fig. 13, the experiment method is to measure the recognition rate with the face recognition camera being first 200㎝ apart and gradually getting closer by 50㎝. Face recognition is performed three times in each condition, and the score of the face recognition for each implementation is recorded. Therefore, the recognition experiment is performed three times without the polarization filter and the recognition experiment is performed three times on mounted the polarization filter.

For the experiment, the light disturbance is reproduced, and the illuminance value for each disturbance light is measured and recorded.

4.4. Experiment progress.

Reproduce the light disturbance for the experiment as shown in Fig. 14(A), and measure and record the illuminance value for each disturbance light. Disturbance light reproduction lighting conditions are 15 as shown in Table 1.

Fig. 14. Experiment.

The experiment method is to conduct a polarization filter experiment on each face recognition camera under 15 lighting conditions at a distance of 200cm. Then to repeat the experiment at a distance of 150cm and 100cm.

As shown in Fig. 14(B), face recognition experiments were conducted three times without polarization filter (N/F: No Filter) and 3 times with polarization filter (F/M: Filter mounted) under 15 lighting conditions. A total of 270 experiments were conducted, and each face recognition points were recorded for experiment as shown in Table 2.

Table 2. Experimental data.

Fig. 15(A) shows the failure of recognition due to the strong disturbance light. Fig. 15(B) is what is recognized when a circularly polarized filter is mounted under the same lighting conditions. When the polarizing filter was mounted at Fig. 15(A), recognition was successful as shown in Fig. 15(B).

Fig. 15. Difference in filter mounting.

If the external light is severe as shown in Fig. 15(A), the photoelectric sensor rises to the saturation voltage level by the disturbance light which reduces the sharpness of the image. Therefore, the sharpness is improved by filtering out the light disturbance by using a circular polarization filter. The image with higher sharpness enables recognition.

4.5. Data analysis

The experimental data are shown in Table 2. ‘LgC’ means 15 lighting conditions. ‘100 N/F’ means a distance of 100㎝(1meter) & No Filter, that is, a polarization filter is not mounted. And ‘100 F/M’ means 100㎝(1meter) & Filter Mounted, that is, a polarization filter is mounted.

Numbers are attempts to recognize and express recognition scores. Since the recognition threshold is 65 points, a recognition score of less than 65 is considered a failure.

Recognition experiments were performed three times under 15 lighting conditions in Table 1 at each distance and whether or not the filter was mounted.

For example, 100 N/F & LgC-1: 94, 92, 91 are the score recognized three times. At this time, recognition was successful because they all exceeded 65 points, but 100 N/F & LgC-15: 53, 57, 55 were less than 65 points in all three cases, so they were all failed.

The 100 N/F & LgC-15, i.e., under No. 15 lighting conditions without a polarization filter at a distance of 1 meter, and all three times they failed. However, 100 F/M & LgC-15, under the same lighting and distance conditions with polarization filter mounting all succeeded the three times.

At the of 150 N/F & LgC-10/12/13/15, that is, without a polarizing filter at a distance of 1.5 meters, under the lighting conditions No. 10, 12, 13, and 15, each condition failed to produce recognition.

There were three recognition failures in a total of twelve attempts in the case of LgC-11. An analysis of the recognition failure shows that it was found to be due to light reflection from the left wall. The experiment was conducted by the experiment conductor and the test subject. Then the recognition failure occurred due to diffuse reflection caused by the movement of the experiment conductor. This was found in the additional verification experiments to analyze the data after a total of twelve attempts were completed. But, any diffused reflection could be filtered through a polarization filter as a result of obtained data analysis and verification.

At the 150 F/M & LgC-10/12/13/15, a polarization filter was mounted under the same conditions as the previous case and all recognition was successful. Besides, it was effective to install a polarization filter under any diffused reflection condition.

At the 200 N/F & LgC-10/12/13/15, that is, without a polarizing filter at 2 meters and under the lighting conditions No. 10, 12, 13, and 15, each condition failed to recognize. LgC-4/5/7/8/9/11/14 intermittently failed to recognize one to three times. The recognition failure was caused by the diffused reflection of light which is occurring by the movement of the experiment conductor.

At 200 F/S, 2 meters apart, in the experiment under this condition, diffuse reflections of light occurred in various lighting conditions. The common point is that diffused reflection occurred when the wall surface was used as a reflector. Of course, the condition of LgC-10/12/13/15 is a case of failure of recognition regardless of diffuse reflection. All failures were solved by installing a polarization filter despite various causes.

The data analysis concludes that recognition problems arise when there are strong light or multiple light sources. However, there is no problem with recognition work without polarization filter in case of the incident of not strong light or only one light source. Most cameras have an automatic light control function. However, when the amount of light exceeds the threshold or various light sources are incident, it will not be prevented.

This is because the camera's automatic light control function has slowed down. The slowed function can recover, but it takes more than a second. Diffuse reflected light caused such a problem. The polarization filter is very effective in blocking diffused reflection regardless of the light control function of the slowing camera.

Table 2 is illustrated in Fig. 16, 17, and 18 respectively. The red circle is the N/F condition and is expressed in different shapes according to twelve attempts. The blue square is the F/M condition and is expressed in different shapes according to twelve attempts.

Fig. 16. 100 (1 meter) experiment.

Fig. 17. 150 (1.5 meter) experiment.

Fig. 18. 200 (2 meter) experiment.

And the pink block under the red dotted line indicates the data that has failed to recognize. The absence of a blue graph in the pink block proves that the mounting of the polarization filter is very effective.

Ⅴ. CONCLUSION

Comparing the scores from the experimental result, F/M condition does not guarantee better performance under any conditions. The F/M condition can achieve better performance in a light disturbance environment, but N/F condition secures better performance under no disturbance light.

As shown in Fig. 16, N/F condition can achieve better performance than F/M condition at a distance of one meter. However, F/M condition have achieved better performance under severe disturbance environment like LgC-15.

Separately, under F/M condition, the farther the face from the recognition device, the higher the recognition score. Because of the camera's light control function is set to short-range priority. Thus, when the face is far from the recognition device, it can't be controlled the amount of light due to the camera's light control function slows down. Therefore, as the distance between the recognition device and the face is farther away, the brightness of the image taken becomes brighter, making it difficult to recognize. Based on the conclusion of the experiment, if the face recognition system is installed in a place with plenty of disturbance light, a light-sensitive automatic polarization filter or a constant device is recommended. However, if the face recognition system is more than 4 meters away from the face and remote recognition such as walk-through is desired, a polarization filter is not recommended. Nevertheless, if there is a lot of disturbance light, a light- sensitive automatic polarization filter is recommended.

The continuous inflow of light slows down the camera's light control function and the response speed. As a result, it was also analyzed that the reflected light could affect the automatic light control function of the slowed camera and cause temporary recognition failure.

This problem was also solved using polarized filters, so the problem caused by diffused reflection was also proved to be effective. However, if the face recognition system is more than 4 meters away from the face and remote recognition such as walk-through is desired, a polarization filter is not recommended. Nevertheless, if there is a lot of disturbance light, a light-sensitive automatic polarization filter is recommended.

The continuous inflow of light slows down the camera's light control function and the response speed. As a result, it was also analyzed that the reflected light could affect the automatic light control function of the slowed camera and cause temporary recognition failure. This problem was also solved using polarized filters, so the problem caused by diffused reflection was also proved to be effective.

Through this study, the final goal was achieved, which is in the blocking of the disturbance light and the improvement of the face recognition system. However, the circular polarization filter used in this study has limitations in coping with various light disturbances. In conclusion, it seems that it is possible to find a method to cope with various light disturbances by using a linear polarization filter’s fine adjustment function.