Chip-based adaptive skin color detection using trajectory constraints on hue

Abstract

Background

In a construction site, the safety of workers can be better secured if vehicle or robot can be properly maneuvered by using image-based gesture guidance. This kind of semi-automatic motion control is based on the features extracted in colors, outlines, textures, and motion intentions. Among these features, detecting worker’s skin color on face and hands could serve as an essential method for enabling robot to determine the region of interest (ROI). However, the performance of skin detection is usually unreliable due to interferences from reflections and shadows. Although many efficient skin color detection methods had been developed in past years, those are mostly based on complicated learning and statistical processes and are unsuitable for embedded systems. The exploration of a concise and efficient skin color detector, therefore, becomes a challenge for applications in mobile robots.

Method

In this paper, we propose a novel adaptive skin detector on face and hands as a fundamental capability of a gesture tracking system. This approach enhances the detection performance of traditional HSV color space but only requires a low computing power. For the design criteria of small size, low-power, low-cost, and minimum computing resource usages on mobile robots, the entire detecting system is built on single field-programmable-gate-array (FPGA) chip. Meanwhile, besides the contributions of adaptive algorithms and FPGA chip designs, the proposed skin detector also employs a touch screen to designate expected skin color of worker as the human-robot interaction (HRI) in real-time.

Results

The chip design of the FPGA is based on hardware circuits in register-transfer-level (RTL) for real-time image processing. A reasonable amount of hardware resource usages of FPGA have consumed 8% of logic elements (LEs) and 1.4% for embedded random access memory (RAM). Demonstrations with various pictures had indicated that sufficient ROI can be efficiently identified by using the proposed adaptive skin color detector.

Conclusions

According to our experimental results, the proposed adaptive skin detection can work well for non-ideal illumination. It has demonstrated the reliability and feasibility on supporting the embedded robotic control in the future.

Background

In past years, skin color detection has been regarded as an essential component in human-robot interaction systems for analyzing human intentions (Kim et al. [2006]; Luo et al. [2011]). This methodology had become popular because it requires a lower computing effort than other image processing approaches even though skin color analysis is a fundamental area of pattern recognition. Recently, we have also developed a chip-based gesture tracking system by cooperating with motion detection to control mobile robots on construction sites (Yu et al. [2014]). Our system simulated a robot to receive worker’s instructions by detecting a traffic light baton on his hand. Once our gesture tracking system confronts the situation of similar lighting sources on the same field, localizing traffic light baton involving regional information of worker’s face and hands will be an efficient and feasible scheme.

In this paper, we propose an adaptive skin color detection strategy by using single FPGA chip. We replaced image collection and training processes with a touch screen as the man–machine interface after considering the characteristics of skin colors of different human races and working environments. The worker can designate his skin color as a target through the interface, then the proposed adaptive skin color detection system, based on HSV color space, can define a preliminary threshold as a skin map. With the input of images, a novel adaptive algorithm named trajectory constraints on hue (TCH) is proposed to dynamically adjust the thresholds of skin color detection.

The contributions of our work rest on the economic circuit resource usages on chip, satisfactory real-time performance, high detection rate under non-ideal illumination, and convenience in installation. Comparing with the state-of-the-art technologies either based on PC simulations and FPGA chip designs, our skin color detection system can achieve superior detecting rate under severe tests. While nowadays efficient skin color detection mostly relies on the complicated algorithms, intelligent learning, skin texture analysis, working on high-end computers or a large amount of FPGA gates count; our concise skin color detection provides the answer to improve detection rates on embedded systems and miniature robots.

In the rest of this paper, the background of skin color detection will be first reviewed. Next, the relationship between HSV and RGB color spaces and the mechanism of the proposed skin color detection by using TCH will be presented. The configuration of chips, hardware resource usages, and the performance of the proposed skin color detection will be described in fourth section. The final sections contain the discussion and the conclusion.

Background

In designing a skin color detection system, there are many color spaces that can be chosen to obtain the components of skin color. The initial idea comes from the use of thresholds to examine human skin color with the red (R), green (G), and blue (B) color space. Human skin color can be determined by the specific magnitudes of R, G, and B. This kind of RGB thresholding method has been recognized as unreliable in that the imaged skin color is highly dependent on the combination of luminance on the skin and results in poor resistance to non-ideal illumination, e.g., shadows and reflections (Yang et al. [2010]). Although the design by using multi-intensity thresholds can mitigate the interferences of shadows and reflections, complicated calibrating processes for different intensity thresholds become another problem for implementation (Yu et al. [2011]). Consequently, modern color detection systems prefer to use color attributes for detection, which are improved from the use of RGB color space for independent color components and discarding intensity, e.g., Normalized RGB, YCbCr, YUV, YIQ, and HSV (Chaves-González et al. [2010]; Liu et al. [2009]). With these kinds of color spaces, it is expected to detect skin without the being affected by intensity. Better detecting performance can be achieved by combining different color spaces in order to support each other (Kumar [2014]; Liu and Shi, [2011]).

However, the hue of skin color may vary in real-life operations. The interferences from shadows and reflections usually shift skin color beyond thresholds and degrade detection performance if intensity-independent color spaces were used (Zhu and Cai, [2011]). This leads to skin color detection methods developed on the basis of adaptive or learning methodologies. For example, Liu employed a variable as the learning rate of histogram to adapt to different environments and then improved skin color detection by using morphological operations (Liu and Peng [2010]). Similar work can be also found on (Li et al. [2013]); where authors simply adopted a ratio to evaluate the deviation distance from centroid value of skin color. This proportion played the role as a threshold to refine their skin color maps in the database. Moreover, Khan et al. ([2011]) designed a skin detector by using RGB color space and Bayesian-based classifiers training, which contained about 5000 samples of skin samples. Chenaoua and Bouridane ([2006]) employed the Markov Random Field to estimate the distribution of skin pixels in images. Fotouhi et al. ([2009]) combined with YC_bC_r, texture extraction, Gaussian mixture models, multi-layer perceptron, and artificial neural networks to perform a comprehensive skin detector. In summary, adaptive skin color detection usually has higher detection rate than methodologies based on traditional color spaces. Particularly for the system designed with learning capabilities, the regions of skin color can be reliably detected except small areas on chin, neck, and fingers.

Although a sufficient skin detection rate can be achieved by probability and learning, the real-time detection performance while running with limited computing power is also another critical issue in robotic applications. Intensive-computing algorithms may achieve a superior detection rate for a static picture, but it is unsuitable for a high resolution and operating at video frame rate. In the works of Toledo et al. ([2006]), Zheng et al. ([2010]), and Liu and Shi ([2011]), the detection of skin color was obtained from rules table, probability, normalized RGB, YCbCr and I₁I₂I₃ color spaces for low computing effort. However, the shadow regions on skin could not be detected efficiently or they could only work under ideal illumination by lacking of adaptive capability. Zafarifar proposed a skin detector by using YUV color space, texture-adaptive analysis, and DSP slices on chip (Zafarifar et al. [2012]). Their works featured additional capability of background eliminations, which were colored as skin. However, a large amount of hardware circuits on chip were required and detection performance in non-ideal illuminating environments was difficult to verify.

In general, skin color detection based on high-level statistics and machine learning usually requires huge computing resources. In spite of the sufficient performance with static image analysis, these methodologies usually result in low frame rate and therefore are unsuitable for tracking with embedded video systems without any modification (Fotouhi et al. [2009]). On the other hand, traditional skin color detection working on basic color space thresholds cannot provide enough quantity of detection for real-life applications. These design dilemmas had motivated our work both on computer- and embedded-based systems, which provides excellent detection rate to provide ROI of targets for further pattern recognition processes.

Methods

Relationship between RGB and HSV

Many intensity independent color spaces can be utilized for skin color detection. Here we have adopted HSV color space after considering the advantage of separating skin color in hue, saturation, and intensity. The distribution of hue and saturation on a circular plan is convenient to develop an algorithm for examining the trajectory variation of skin color.

The HSV color space is known as an improvement from traditional RBG color space. The term of H denotes the hue (or color-depth), and S is for saturation (or color-purity). Intensity is represented as V (or brightness). The algorithm for converting RGB into HSV color space can be expressed as the follows (Chaves-González et al. [2010]):

$$H=\left\{\begin{array}{ccc}\hfill \mathit{Undefined},\hfill & \hfill \mathit{if}\hfill & \hfill \mathit{MAX}=MIN\hfill \\ \hfill \frac{60\left(G-B\right)}{\mathit{MAX}-MIN},\hfill & \hfill \mathit{if}\hfill & \hfill \mathit{MAX}=R\hfill & \hfill \mathit{and}\hfill & \hfill G\ge B\hfill \\ \hfill \frac{60\left(G-B\right)}{\mathit{MAX}-MIN}+360,\hfill & \hfill \mathit{if}\hfill & \hfill \mathit{MAX}=R\hfill & \hfill \mathit{and}\hfill & \hfill G<B\hfill \\ \hfill \frac{60\left(B-R\right)}{\mathit{MAX}-MIN}+120,\hfill & \hfill \mathit{if}\hfill & \hfill \mathit{MAX}=G\hfill \\ \hfill \frac{60\left(R-G\right)}{\mathit{MAX}-MIN}+240,\hfill & \hfill \mathit{if}\hfill & \hfill \mathit{MAX}=B\hfill \end{array}\right.$$

(1)

and

$$S=\left\{\begin{array}{ccc}\hfill 0,\hfill & \hfill \mathit{if}\hfill & \hfill \mathit{MAX}=0\hfill \\ \hfill \frac{\mathit{MAX}-MIN}{\mathit{MAX}}\hfill & \hfill \mathit{Otherwise}\hfill \end{array}\right.$$

(2)

$$V=\mathit{MAX}$$

(3)

where MIN and MAX in equations (1) to (3) denote the RGB-intensity bound in [0,1]. The distribution of hue is usually plotted as a circular plan in four quadrants, and the plan radius is equivalent to the saturation value S. Hue is composed of red, green, and blue colors changing by every 120^∘. When the HSV color space is plotted as a 3D model, the part right under the H-S circular plan is the intensity V of color space, which is in an inverted conical shape.

As shown in Figure 1, by changing the magnitudes of RGB from the scale of 0 to 255 with constraints for R ≥ G ≥ B, the variation of blue are approximately the S, and adjusting green will obtain H. On the other hand, the trajectory of red color combines with the features of H and S at the same time. According to the studies of Vezhnevets et al. ([2003]) and Lin et al. ([2011]), human skin color can be roughly classified as R > G > B, and different skin colors can be derived by adjusting R, G, or B magnitude arbitrarily within the constraints. By avoiding the use of intensity value, the variation of skin color is the same as the changes of H and S.

Figure 1

The trajectories in HSV color space by adjusting R-G-B color individually; (a) R = 255,G = 250,B = 0 ~ 255, (b) R = 146,G = 115,B = 0 ~ 115, (c) R = 240,G = 20 ~ 240,B = 20, (d) R = 170,G = 100 ~ 170,B = 100, (e) R = 52 ~ 255,G = 52,B = 12, (f) R = 92 ~ 255,G = 92,B = 60.

Based on above discussion, most colors can be simulated by altering the positions of colors in constraints. For example, the relationship of R > B > G results in a pink color and the B trajectory becomes H instead. If we continually increase the magnitude of B, the output of color will be closer to vivid blue in a clockwise direction for B > R > G towards the plan boundary. Since the behavior of color between upper and lower constraints is similar to hue, here we define the color in the middle as the dominant color. Therefore, it can be seen that human skin color is dominated by green.

The definition of H-S plan

According to the discussion in the last sub-section, the intensity V could be neglected in skin color detection. Although skin color may look different under dim environments, one can achieve skin color discrimination by only examining the variations on hue and saturation. When detecting skin color by using HSV color space, the H-S plan can be plotted as in Figure 2(a). The coordinate of centroid T of skin color map can be expressed as:

$$T_x=ScosH$$

(4)

$$T_y=SsinH$$

(5)

Figure 2

The definition of H-S plan; (a) the H-S plan, (b) the initial region of skin color map.

thus

$$S=\sqrt{T_x^2+T_y^2}$$

(6)

where the hue H in equations (4) and (5) is equivalent to the angle suspended by vector T, and S is the radius of the circular plan. Based on the coordinate of T, an initial skin color map can be defined by four points of as:

$$T_A\left(H,S\right)=\left(H_T+E_H,S_T-E_S\right)$$

(7)

$$T_B\left(H,S\right)=\left(H_T-E_H,S_T-E_S\right)$$

(8)

$$T_C\left(H,S\right)=\left(H_T-E_H,S_T+E_S\right)$$

(9)

$$T_D\left(H,S\right)=\left(H_T+E_H,S_T+E_S\right)$$

(10)

where H _T and S _T denote the H and S of T, and the variables E _H and E _S both are tolerances in determining the range of initial skin color map F, as shown in Figure 2(b). The intensity scale of H-S plan is [0, 255] for an 8-bits digital system. Thus the coordinates of initial skin color map are:

$$T_{A\left(x,y\right)}=\left(S_{T_A}cos{H}_{T_A},S_{T_A}sin{H}_{T_A}\right)$$

(11)

$$T_{B\left(x,y\right)}=\left(S_{T_B}cos{H}_{T_B},S_{T_B}sin{H}_{T_B}\right)$$

(12)

$$T_{C\left(x,y\right)}=\left(S_{T_C}cos{H}_{T_C},S_{T_C}sin{H}_{T_C}\right)$$

(13)

$$T_{D\left(x,y\right)}=\left(S_{T_D}cos{H}_{T_D},S_{T_D}sin{H}_{T_D}\right)$$

(14)

According to equations (7) to (10), various skin color can then be obtained by adjusting the range of H and S. Chen et al. ([2012]) had defined humans skin colors by using constraints of 0 ≤ H ≤ 50 and 51 ≤ S ≤ 173. Before this, Yang et al. ([2011]) had also defined the skin color for the east Asian races by using the constraints of 14.4 ≤ H ≤ 46.8 and 69.615 ≤ S ≤ 149.55. Here we define the centroid point T by using 3 ≤ H ≤ 18 and leave S as undefined in that value is varied to different human races and can be dynamically determined by touch screen. Finally, the tolerance of skin hue is set for E _H  = ± 25 and the range of E _S  = ± 30 is for saturation testing based on our experiments. This arrangement leads to TCH’s adaptability on H which will mostly stop around yellow and only a small amount of sampled greenish color will be passed at low saturation.

Skin color detection by using TCH algorithm

After described the H-S plan characteristics, next we can determine an adaptive skin color threshold by using the TCH algorithm, which can be discussed as in the following cases.

• 1. |H _i  − H _T | ≤ 25 and |S _i  − S _T | ≤ 30:

If the input of sampled image pixel color (H _i , S _i ) always falls into the region of initial skin color map F as shown in Figure 2(b), this pixels is defined as corresponding to human skin.

• 2. |H _i  − H _T | ≤ 25 and |S _i  − S _T | > 30:

These kinds of input colors indicate that pixels in image have exceeded saturation values of the initial skin color map F but are still with acceptable hue. Since the saturation value is impossible for human skin color, the locations of these pixels in picture are defined as the non-skin areas.

• 3. |H _i  − H _T | > 25 and |S _i  − S _T | ≤ 30:

Skin color may vary slowly in hue by reflections and illumination. It makes it difficult to determine skin color without learning capability, as shown in Figure 3(a). Once any sampled image pixel color first conforms to this case, the adaptive algorithm using TCH is triggered and then proceeds in the following steps:

• a. Record the pixel saturation value S as a constant, which is from the last color T ₀ before exceeding the initial skin color map F, as shown in Figure 3(a) and (b). This procedure ignores the effects from saturation besides F. In other words, S will be unvalued if H is too far away from centroid T.

• b. If incoming image pixels colors are all outside the region of F, the proposed TCH will continually replace these image pixels’ saturations from T ₀ —otherwise the system resets its operation to the initial condition. Such arrangement leads to varying trajectories of image pixels colors beyond both sides of F and will all follow a concentric circle, as shown in Figure 3(b). This results in a simple and adaptive skin color detection process which only needs to examine the variances on the pixel’s hue.

• c. Examining the slope of line $\overline{T_n{T}_{n+1}}$ after obtained the coordinates of points T _A to T _D , as shown in Figure 3(c); T _n , and T _{n+ 1} are derived based on equations (4) to (14), and the corrected hue trajectory of line $\overline{T_n{T}_{n+1}}$ will run on a circle as tangent. Once the slope m _T of tangent $\overline{T_n{T}_{n+1}}$ exceeds the slopes constraints of four lines $\overline{T_{n+1}{T}_A}$, $\overline{T_{n+1}{T}_B}$, $\overline{T_{n+1}{T}_C}$, and $\overline{T_{n+1}{T}_D}$, the pixel color at point T _n + 1 is defined as a non-skin color and then the system is reset to the initial condition. The examination mechanism can be represented in a decision tree shown in Figure 4. The m _min and m _max in the tree denote the upper and lower constraints of four lines’ slopes respectively. The slopes of $\overline{T_{n+1}{T}_A}$, $\overline{T_{n+1}{T}_B}$, $\overline{T_{n+1}{T}_C}$, and $\overline{T_{n+1}{T}_D}$ on both sides of skin color map F can be summarized for three conditions at H _i  ≥ 0:

Figure 4

The decision tree to judge skin color.

State_1: One positive slope (×1) and three negative slopes (×3).

State_2: Two positive slopes (×2) and two negative slopes (×2).

State_3: The slopes of the four lines are all in negative (×4).

Figure 3

The operations of TCH algorithm; (a) incoming image pixels’ colors exceeding out of skin region F , (b) updating colors’ saturations with homogenous value of T ₀ , (c) examining slope using line $\overline{{\mathit{T}}_{\mathit{n}}{\mathit{T}}_{\mathit{n}+\mathbf{1}}}$ .

The detected skin color below line $\overline{T_B{T}_D}$ represent lip and blood vessel colors which can be judged as a part of the skin.

Results

Figure 5(a) shows the hardware setup of the proposed skin color detection system. A touch screen is employed to designate expected skin color of worker and examines detection performance. The digital camera module is vertically installed on a slot of the FPGA platform. Synthesis of FPGA circuits and summary of chip resources usages are based on the Quartus II from Altera software. The block diagram of FPGA chip is shown in Figure 5(b). Raw video images from the digital camera are converted into full color images by the real-time de-mosaicking of our previous works (Yu et al. [2011]). Full color images from de-mosaicking are then sent to the touch screen for user to designate expected skin color. Once any skin region has been designated (touched) by the user’s finger, the information of skin color and screen coordinate of the touched point are sent to the TCH algorithm together for skin detection. As shown in Figure 5(c), output images can be controlled by the signal denoted as Image Selection. A full color was first chosen to observe the region of skin color. After the user touching screen, a touched mark is generated and shown on the screen to indicate the location of the touching point. This action also determines the initial skin color map F in HSV color space, which is transferred from RGB. Next the user switch image output to detection mode. Skin color map coordinates T _A to T _D are then derived and examined for slopes constraints as aforementioned. In the end, the skin color detected by TCH is represented in a binary image and displayed on the touch screen.

Figure 5

The proposed chip-based skin detection; (a) System setup, (b) FPGA block diagram, (c) TCH block diagram.

Resource usages of the proposed FPGA chip designs are shown in Table 1. Consumption of logic elements (LEs) are 9,216 (8%), which involves 7,646 LEs for TCH. The FPGA chip is the Altera Cyclone IV EP4CE115F29C7 N. The proposed TCH was designed in RTL circuits in order to achieve real-time image processing. It can be seen that the TCH algorithm consumed the largest amount of LEs in the proposed skin color detection system that the algorithm required 21 9-bit embedded multipliers for RGB-HSV conversion and slope derivation with trigonometric functions. The embedded memory consumption was 57,400 bits (1.4%). This consumption was for the image buffer of touch screen and full color de-mosaicking.

Table 1 FPGA resource usages of proposed skin color detection

The digital camera adopted in our design has a maximum of 5 M pixels resolution, but it was set for 800 × 480 pixels for use with the touch screen at a frame rate of 15fps and 30 MHz pixel clock. Meanwhile, the proposed TCH required 300 MHz clock speed and 37 clocks to run TCH during image pixel scans. Such arrangement leads to about four pixels delay showing on the touch screen.

Snapshots of real-life operations for proposed skin color detection are shown in Figure 6 without a video noise filter. Five severe test scenarios were designed to demonstrate the reliability of the proposed adaptive skin color detection system. We focused on the resistances to shadows and reflections at forehead, chin, hands, and the part of the face under the brim of a cap. Figure 6 demonstrates excellent skin color detection of TCH, which provides satisfactory ROI area for pattern recognition, object tracking, and image processing purposes. In these test scenarios, the skin color detections by using traditional RGB and HSV thresholds were also conducted to compare with our TCH algorithm using constant skin color map. We could see that the skin color detection using RGB threshold in these tests exhibited the worst detection rate while the proposed TCH algorithm had the best detection performance in tests.

Figure 6

Comparisons among traditional RGB and HSV thresholds and proposed TCH algorithm; (a)(e)(i)(m)(q) the test scenarios, (b)(f)(j)(n)(r) tested with RGB threshold, (c)(g)(k)(o)(s) tested with HSV threshold, (d)(h)(l)(p)(t) tested with proposed TCH.

In the first test scenario of Figure 6, see Figures 6(a) to (d), the interferences of shadows and reflections existed at the same time on tester’s face. It can be seen that the proposed TCH skin color detection system demonstrated excellent adaptive capability to detect skin areas with shadows and reflections. For the test result using HSV threshold, although it has been a popular method on embedded systems, the detection rates at testers’ chin and neck were still deteriorated by shadows. The detection performances between RGB and HSV thresholds had not shown much difference in this test. In the second test scenario, we tested TCH algorithm with extreme interference of shadow by huddling up a palm. The RGB and HSV thresholds both still had low detection rate in test. Comparatively, proposed TCH algorithm in this test could detect the skin color reaching the inner part of palm perfectly. In the third test scenario, we tested our skin detector by using green color to shift skin color moderately. The tester covered his palm with a green paper to obtain some green reflection on palm, as shown in Figure 6(i). It can be seen that the proposed TCH algorithm operating in such environment could still resist to green shadow and reflection. Again, RGB and HSV constant thresholds in this test still had low detection rate at fingers, where was covered by green shadow. After illustrated the success in resistance to shadows and reflections, the fourth test scenario simulated the condition of wearing a cap, as shown in Figure 6(m). In this test, the tester’s face was covered by shadow and some yellow reflections from inner brim of cap. The RGB threshold with this test scenario nearly failed to detect the tester’s face, and even the HSV threshold could only improve little. Nevertheless, our TCH algorithm could still work quite well in this test except a small part right under the brim of cap. In the last test scenario of Figure 6, the capability of skin detection under a dark environment was also tested, as shown in Figure 6(q). The lights in laboratory were nearly turned off in this test scenario. It can be seen the RGB and HSV thresholds could surprisingly detect skin region of the subject, and the silhouette of HSV threshold was sufficient for recognition on the face and palm. The proposed TCH skin color detection system under such severe environment was apparently superior to traditional HSV threshold for full scope detection rate of the subject’s face and palm.

For quantitative analysis, we tested TCH with different human races and illumination conditions. Tested images were all downloaded from the internet (Eastday [2013]; Gamebase [2008]; Gigacircle [2014]; Nextmedia [2013]; Sina house [2011]; Sohu sport [2010]; Vanity fair [2014]; 10202blog [2013]) and references and then printed by a color printer so that digital camera could capture these pictures into FPGA. However, it should be noted that this test method also incurred chromatic distortions due to imperfect reproduction in paper and from the printer. Figure 7 indicates that TCH could detect these candidate faces efficiently even in outdoor environments. With the same test method, we also compared with state-of-the-art methodologies given in Table 2, and experimental results are shown in Figure 8. The images for comparison were copied from articles and enlarged so that the TCH system was fed with images of larger chromatic distortions than that given in Figure 7. It can be seen that TCH with such severe test condition can also provide sufficient detection accuracy for ROI.

Figure 7

Proposed TCH without noise filter for working on various human races and illumination conditions with imperfect reproduction of pictures; (a)-(d) are from Vanity fair ([2014]), (e) is from Gamebase ([2008]), (f) and (g) are from Sina house ([2011]), (h) is from Gigacircle ([2014]), (i) is from 10202blog ([2013]), (j) is from Sohu sport ([2010]), (k) and (l) are from Eastday ([2013]), and (y) is from Nextmedia[2013].

Table 2 Comparisons of resource usages among different skin detector designs

Figure 8

The comparison with skin detection methodologies from references; (a)-(e) are test scenarios from Table 2 , (1)-(5) with their results in (f)-(j), and (k)-(o) are for pixel-based TCH tested without noise filter.

Discussions

The salient features of the proposed skin color detection system involve simple calibration processes, moderate hardware resource usage on chips, and efficient adaptability without complicated training algorithms. Comparing with the state-of-the-art technologies listed in Table 2, the proposed TCH adaptive algorithm cooperating with a touch screen is comparable to the methodologies using statistical and training classifier for high detection rate and low implementation effort. Besides, the combination of different color thresholds may achieve sufficient detection rate on embedded system by using a small amount of hardware resources (Liu and Shi, [2011]), but such system can only work under adequate illumination and tedious calibration is still unavoidable because of the lack of adaptive mechanisms. After inspecting the skin detector results in Figure 8, the best skin color detection design for embedded system is the texture-based skin color analysis; see the performance in Figure 8(j). This kind of system is not only applicable for skin color detection but also provides additional functionality on background elimination, where it is colored with skin color. However, a large amount of hardware resources usages is required for complicated texture analysis. Consequently, it dramatically increases design difficulty on real-time processing speed and the implementation in a small chip.

In addition, the adaptability of TCH’s skin color detection is determined by the pixel saturation before exceeding the initial skin color map F. As shown in Figure 9, the pixel color before exceeding the initial skin color map at a high saturation value will result in a low adaptability to variation in skin color, and a null adaptability will be set if the exceeded color is at the upper saturation boundary of the initial skin color map, as shown in the Figure 9(a). In contrast, TCH’s adaptability will be increased if the exceeded color is at a low saturation value, and it results in maximum adaptability at the lower saturation boundary of initial skin color map, as shown in the Figure 9(b). The proposed non-linear adaptive mechanism is similar to human’s vision in that people can distinguish different colors easily between high saturation colors but not the low saturation ones.

Figure 9

Operations of TCH at upper and lower boundaries of adaptability; (a) upper saturation boundary, (b) lower saturation boundary.

Finally, the minor drawback of TCH arises from the delay of image processing and incurs coarse binary picture quality, as shown in Figures 6 to 8. With the pixel-based skin color detection, the image quality is sensitive to random noise in the camera, then the noise is enlarged horizontally due to the late update of pixels information during image acquisition. Although the coarse binary picture quality seems not affecting detection rate, the authors will determine how to improve processing speed with refined hardware circuits in the future, and a real-time and simplified pixel-based noise filter from our previous work may be also implemented (Yu et al. [2011]).

Conclusion

In this paper, we have proposed an efficient adaptive skin color detection system. The proposed algorithm can derive an initial skin color map using traditional HSV color space after captured images from touch screen. Once incoming pixel colors exceed the region of default skin color map, the non-linear adaptive algorithm of trajectory constraints on hue (TCH) can dynamically adjust the range of thresholds for detection. Various severe test scenarios had been arranged to verify the capability of proposed skin detection. Our experimental results indicate that the proposed adaptive skin color detection is efficient and applicable to various effects from illumination conditions. A reasonable amount of hardware resource usages in the FPGA chip can allow the TCH skin color detection to be implemented on single FPGA chip and suitable for miniature electronic devices. The future works involve the refinement of circuits in economical gates count and processing delay improvement.