Open Access

Chip-based adaptive skin color detection using trajectory constraints on hue

Visualization in Engineering20142:11

https://doi.org/10.1186/s40327-014-0011-1

Received: 16 July 2014

Accepted: 24 October 2014

Published: 12 November 2014

Abstract

Background

In a construction site, the safety of workers can be better secured if vehicle or robot can be properly maneuvered by using image-based gesture guidance. This kind of semi-automatic motion control is based on the features extracted in colors, outlines, textures, and motion intentions. Among these features, detecting worker’s skin color on face and hands could serve as an essential method for enabling robot to determine the region of interest (ROI). However, the performance of skin detection is usually unreliable due to interferences from reflections and shadows. Although many efficient skin color detection methods had been developed in past years, those are mostly based on complicated learning and statistical processes and are unsuitable for embedded systems. The exploration of a concise and efficient skin color detector, therefore, becomes a challenge for applications in mobile robots.

Method

In this paper, we propose a novel adaptive skin detector on face and hands as a fundamental capability of a gesture tracking system. This approach enhances the detection performance of traditional HSV color space but only requires a low computing power. For the design criteria of small size, low-power, low-cost, and minimum computing resource usages on mobile robots, the entire detecting system is built on single field-programmable-gate-array (FPGA) chip. Meanwhile, besides the contributions of adaptive algorithms and FPGA chip designs, the proposed skin detector also employs a touch screen to designate expected skin color of worker as the human-robot interaction (HRI) in real-time.

Results

The chip design of the FPGA is based on hardware circuits in register-transfer-level (RTL) for real-time image processing. A reasonable amount of hardware resource usages of FPGA have consumed 8% of logic elements (LEs) and 1.4% for embedded random access memory (RAM). Demonstrations with various pictures had indicated that sufficient ROI can be efficiently identified by using the proposed adaptive skin color detector.

Conclusions

According to our experimental results, the proposed adaptive skin detection can work well for non-ideal illumination. It has demonstrated the reliability and feasibility on supporting the embedded robotic control in the future.

Keywords

Gesture trackingSkin color detectionColor spaceField-programmable-gate-array (FPGA)Trajectory constraints on hue (TCH)

Background

In past years, skin color detection has been regarded as an essential component in human-robot interaction systems for analyzing human intentions (Kim et al. [2006]; Luo et al. [2011]). This methodology had become popular because it requires a lower computing effort than other image processing approaches even though skin color analysis is a fundamental area of pattern recognition. Recently, we have also developed a chip-based gesture tracking system by cooperating with motion detection to control mobile robots on construction sites (Yu et al. [2014]). Our system simulated a robot to receive worker’s instructions by detecting a traffic light baton on his hand. Once our gesture tracking system confronts the situation of similar lighting sources on the same field, localizing traffic light baton involving regional information of worker’s face and hands will be an efficient and feasible scheme.

In this paper, we propose an adaptive skin color detection strategy by using single FPGA chip. We replaced image collection and training processes with a touch screen as the man–machine interface after considering the characteristics of skin colors of different human races and working environments. The worker can designate his skin color as a target through the interface, then the proposed adaptive skin color detection system, based on HSV color space, can define a preliminary threshold as a skin map. With the input of images, a novel adaptive algorithm named trajectory constraints on hue (TCH) is proposed to dynamically adjust the thresholds of skin color detection.

The contributions of our work rest on the economic circuit resource usages on chip, satisfactory real-time performance, high detection rate under non-ideal illumination, and convenience in installation. Comparing with the state-of-the-art technologies either based on PC simulations and FPGA chip designs, our skin color detection system can achieve superior detecting rate under severe tests. While nowadays efficient skin color detection mostly relies on the complicated algorithms, intelligent learning, skin texture analysis, working on high-end computers or a large amount of FPGA gates count; our concise skin color detection provides the answer to improve detection rates on embedded systems and miniature robots.

In the rest of this paper, the background of skin color detection will be first reviewed. Next, the relationship between HSV and RGB color spaces and the mechanism of the proposed skin color detection by using TCH will be presented. The configuration of chips, hardware resource usages, and the performance of the proposed skin color detection will be described in fourth section. The final sections contain the discussion and the conclusion.

Background

In designing a skin color detection system, there are many color spaces that can be chosen to obtain the components of skin color. The initial idea comes from the use of thresholds to examine human skin color with the red (R), green (G), and blue (B) color space. Human skin color can be determined by the specific magnitudes of R, G, and B. This kind of RGB thresholding method has been recognized as unreliable in that the imaged skin color is highly dependent on the combination of luminance on the skin and results in poor resistance to non-ideal illumination, e.g., shadows and reflections (Yang et al. [2010]). Although the design by using multi-intensity thresholds can mitigate the interferences of shadows and reflections, complicated calibrating processes for different intensity thresholds become another problem for implementation (Yu et al. [2011]). Consequently, modern color detection systems prefer to use color attributes for detection, which are improved from the use of RGB color space for independent color components and discarding intensity, e.g., Normalized RGB, YCbCr, YUV, YIQ, and HSV (Chaves-González et al. [2010]; Liu et al. [2009]). With these kinds of color spaces, it is expected to detect skin without the being affected by intensity. Better detecting performance can be achieved by combining different color spaces in order to support each other (Kumar [2014]; Liu and Shi, [2011]).

However, the hue of skin color may vary in real-life operations. The interferences from shadows and reflections usually shift skin color beyond thresholds and degrade detection performance if intensity-independent color spaces were used (Zhu and Cai, [2011]). This leads to skin color detection methods developed on the basis of adaptive or learning methodologies. For example, Liu employed a variable as the learning rate of histogram to adapt to different environments and then improved skin color detection by using morphological operations (Liu and Peng [2010]). Similar work can be also found on (Li et al. [2013]); where authors simply adopted a ratio to evaluate the deviation distance from centroid value of skin color. This proportion played the role as a threshold to refine their skin color maps in the database. Moreover, Khan et al. ([2011]) designed a skin detector by using RGB color space and Bayesian-based classifiers training, which contained about 5000 samples of skin samples. Chenaoua and Bouridane ([2006]) employed the Markov Random Field to estimate the distribution of skin pixels in images. Fotouhi et al. ([2009]) combined with YCbCr, texture extraction, Gaussian mixture models, multi-layer perceptron, and artificial neural networks to perform a comprehensive skin detector. In summary, adaptive skin color detection usually has higher detection rate than methodologies based on traditional color spaces. Particularly for the system designed with learning capabilities, the regions of skin color can be reliably detected except small areas on chin, neck, and fingers.

Although a sufficient skin detection rate can be achieved by probability and learning, the real-time detection performance while running with limited computing power is also another critical issue in robotic applications. Intensive-computing algorithms may achieve a superior detection rate for a static picture, but it is unsuitable for a high resolution and operating at video frame rate. In the works of Toledo et al. ([2006]), Zheng et al. ([2010]), and Liu and Shi ([2011]), the detection of skin color was obtained from rules table, probability, normalized RGB, YCbCr and I1I2I3 color spaces for low computing effort. However, the shadow regions on skin could not be detected efficiently or they could only work under ideal illumination by lacking of adaptive capability. Zafarifar proposed a skin detector by using YUV color space, texture-adaptive analysis, and DSP slices on chip (Zafarifar et al. [2012]). Their works featured additional capability of background eliminations, which were colored as skin. However, a large amount of hardware circuits on chip were required and detection performance in non-ideal illuminating environments was difficult to verify.

In general, skin color detection based on high-level statistics and machine learning usually requires huge computing resources. In spite of the sufficient performance with static image analysis, these methodologies usually result in low frame rate and therefore are unsuitable for tracking with embedded video systems without any modification (Fotouhi et al. [2009]). On the other hand, traditional skin color detection working on basic color space thresholds cannot provide enough quantity of detection for real-life applications. These design dilemmas had motivated our work both on computer- and embedded-based systems, which provides excellent detection rate to provide ROI of targets for further pattern recognition processes.

Methods

Relationship between RGB and HSV

Many intensity independent color spaces can be utilized for skin color detection. Here we have adopted HSV color space after considering the advantage of separating skin color in hue, saturation, and intensity. The distribution of hue and saturation on a circular plan is convenient to develop an algorithm for examining the trajectory variation of skin color.

The HSV color space is known as an improvement from traditional RBG color space. The term of H denotes the hue (or color-depth), and S is for saturation (or color-purity). Intensity is represented as V (or brightness). The algorithm for converting RGB into HSV color space can be expressed as the follows (Chaves-González et al. [2010]):
H = Undefined , if MAX = M I N 60 G B MAX M I N , if MAX = R and G B 60 G B MAX M I N + 360 , if MAX = R and G < B 60 B R MAX M I N + 120 , if MAX = G 60 R G MAX M I N + 240 , if MAX = B
(1)
and
S = 0 , if MAX = 0 MAX M I N MAX Otherwise
(2)
V = MAX
(3)

where MIN and MAX in equations (1) to (3) denote the RGB-intensity bound in [0,1]. The distribution of hue is usually plotted as a circular plan in four quadrants, and the plan radius is equivalent to the saturation value S. Hue is composed of red, green, and blue colors changing by every 120. When the HSV color space is plotted as a 3D model, the part right under the H-S circular plan is the intensity V of color space, which is in an inverted conical shape.

As shown in Figure 1, by changing the magnitudes of RGB from the scale of 0 to 255 with constraints for R ≥ G ≥ B, the variation of blue are approximately the S, and adjusting green will obtain H. On the other hand, the trajectory of red color combines with the features of H and S at the same time. According to the studies of Vezhnevets et al. ([2003]) and Lin et al. ([2011]), human skin color can be roughly classified as R > G > B, and different skin colors can be derived by adjusting R, G, or B magnitude arbitrarily within the constraints. By avoiding the use of intensity value, the variation of skin color is the same as the changes of H and S.
Figure 1

The trajectories in HSV color space by adjusting R-G-B color individually; (a) R = 255,G = 250,B = 0 ~ 255, (b) R = 146,G = 115,B = 0 ~ 115, (c) R = 240,G = 20 ~ 240,B = 20, (d) R = 170,G = 100 ~ 170,B = 100, (e) R = 52 ~ 255,G = 52,B = 12, (f) R = 92 ~ 255,G = 92,B = 60.

Based on above discussion, most colors can be simulated by altering the positions of colors in constraints. For example, the relationship of R > B > G results in a pink color and the B trajectory becomes H instead. If we continually increase the magnitude of B, the output of color will be closer to vivid blue in a clockwise direction for B > R > G towards the plan boundary. Since the behavior of color between upper and lower constraints is similar to hue, here we define the color in the middle as the dominant color. Therefore, it can be seen that human skin color is dominated by green.

The definition of H-S plan

According to the discussion in the last sub-section, the intensity V could be neglected in skin color detection. Although skin color may look different under dim environments, one can achieve skin color discrimination by only examining the variations on hue and saturation. When detecting skin color by using HSV color space, the H-S plan can be plotted as in Figure 2(a). The coordinate of centroid T of skin color map can be expressed as:
T x = S cos H
(4)
T y = S sin H
(5)
Figure 2

The definition of H-S plan; (a) the H-S plan, (b) the initial region of skin color map.

thus
S = T x 2 + T y 2
(6)
where the hue H in equations (4) and (5) is equivalent to the angle suspended by vector T, and S is the radius of the circular plan. Based on the coordinate of T, an initial skin color map can be defined by four points of as:
T A H , S = H T + E H , S T E S
(7)
T B H , S = H T E H , S T E S
(8)
T C H , S = H T E H , S T + E S
(9)
T D H , S = H T + E H , S T + E S
(10)
where H T and S T denote the H and S of T, and the variables E H and E S both are tolerances in determining the range of initial skin color map F, as shown in Figure 2(b). The intensity scale of H-S plan is [0, 255] for an 8-bits digital system. Thus the coordinates of initial skin color map are:
T A x , y = S T A cos H T A , S T A sin H T A
(11)
T B x , y = S T B cos H T B , S T B sin H T B
(12)
T C x , y = S T C cos H T C , S T C sin H T C
(13)
T D x , y = S T D cos H T D , S T D sin H T D
(14)

According to equations (7) to (10), various skin color can then be obtained by adjusting the range of H and S. Chen et al. ([2012]) had defined humans skin colors by using constraints of 0 ≤ H ≤ 50 and 51 ≤ S ≤ 173. Before this, Yang et al. ([2011]) had also defined the skin color for the east Asian races by using the constraints of 14.4 ≤ H ≤ 46.8 and 69.615 ≤ S ≤ 149.55. Here we define the centroid point T by using 3 ≤ H ≤ 18 and leave S as undefined in that value is varied to different human races and can be dynamically determined by touch screen. Finally, the tolerance of skin hue is set for E H  = ± 25 and the range of E S  = ± 30 is for saturation testing based on our experiments. This arrangement leads to TCH’s adaptability on H which will mostly stop around yellow and only a small amount of sampled greenish color will be passed at low saturation.

Skin color detection by using TCH algorithm

After described the H-S plan characteristics, next we can determine an adaptive skin color threshold by using the TCH algorithm, which can be discussed as in the following cases.
  1. 1.

    |H i  − H T | ≤ 25 and |S i  − S T | ≤ 30:

     
If the input of sampled image pixel color (H i , S i ) always falls into the region of initial skin color map F as shown in Figure 2(b), this pixels is defined as corresponding to human skin.
  1. 2.

    |H i  − H T | ≤ 25 and |S i  − S T | > 30:

     
These kinds of input colors indicate that pixels in image have exceeded saturation values of the initial skin color map F but are still with acceptable hue. Since the saturation value is impossible for human skin color, the locations of these pixels in picture are defined as the non-skin areas.
  1. 3.

    |H i  − H T | > 25 and |S i  − S T | ≤ 30:

     
Skin color may vary slowly in hue by reflections and illumination. It makes it difficult to determine skin color without learning capability, as shown in Figure 3(a). Once any sampled image pixel color first conforms to this case, the adaptive algorithm using TCH is triggered and then proceeds in the following steps:
  1. a.

    Record the pixel saturation value S as a constant, which is from the last color T 0 before exceeding the initial skin color map F, as shown in Figure 3(a) and (b). This procedure ignores the effects from saturation besides F. In other words, S will be unvalued if H is too far away from centroid T.

     
  2. b.

    If incoming image pixels colors are all outside the region of F, the proposed TCH will continually replace these image pixels’ saturations from T 0 —otherwise the system resets its operation to the initial condition. Such arrangement leads to varying trajectories of image pixels colors beyond both sides of F and will all follow a concentric circle, as shown in Figure 3(b). This results in a simple and adaptive skin color detection process which only needs to examine the variances on the pixel’s hue.

     
  3. c.

    Examining the slope of line T n T n + 1 ¯ after obtained the coordinates of points T A to T D , as shown in Figure 3(c); T n , and T n+ 1 are derived based on equations (4) to (14), and the corrected hue trajectory of line T n T n + 1 ¯ will run on a circle as tangent. Once the slope m T of tangent T n T n + 1 ¯ exceeds the slopes constraints of four lines T n + 1 T A ¯ , T n + 1 T B ¯ , T n + 1 T C ¯ , and T n + 1 T D ¯ , the pixel color at point T n + 1 is defined as a non-skin color and then the system is reset to the initial condition. The examination mechanism can be represented in a decision tree shown in Figure 4. The m min and m max in the tree denote the upper and lower constraints of four lines’ slopes respectively. The slopes of T n + 1 T A ¯ , T n + 1 T B ¯ , T n + 1 T C ¯ , and T n + 1 T D ¯ on both sides of skin color map F can be summarized for three conditions at H i  ≥ 0:

     
Figure 4

The decision tree to judge skin color.

State_1: One positive slope (×1) and three negative slopes (×3).

State_2: Two positive slopes (×2) and two negative slopes (×2).

State_3: The slopes of the four lines are all in negative (×4).
Figure 3

The operations of TCH algorithm; (a) incoming image pixels’ colors exceeding out of skin region F , (b) updating colors’ saturations with homogenous value of T 0 , (c) examining slope using line T n T n + 1 ¯ .

The detected skin color below line T B T D ¯ represent lip and blood vessel colors which can be judged as a part of the skin.

Results

Figure 5(a) shows the hardware setup of the proposed skin color detection system. A touch screen is employed to designate expected skin color of worker and examines detection performance. The digital camera module is vertically installed on a slot of the FPGA platform. Synthesis of FPGA circuits and summary of chip resources usages are based on the Quartus II from Altera software. The block diagram of FPGA chip is shown in Figure 5(b). Raw video images from the digital camera are converted into full color images by the real-time de-mosaicking of our previous works (Yu et al. [2011]). Full color images from de-mosaicking are then sent to the touch screen for user to designate expected skin color. Once any skin region has been designated (touched) by the user’s finger, the information of skin color and screen coordinate of the touched point are sent to the TCH algorithm together for skin detection. As shown in Figure 5(c), output images can be controlled by the signal denoted as Image Selection. A full color was first chosen to observe the region of skin color. After the user touching screen, a touched mark is generated and shown on the screen to indicate the location of the touching point. This action also determines the initial skin color map F in HSV color space, which is transferred from RGB. Next the user switch image output to detection mode. Skin color map coordinates T A to T D are then derived and examined for slopes constraints as aforementioned. In the end, the skin color detected by TCH is represented in a binary image and displayed on the touch screen.
Figure 5

The proposed chip-based skin detection; (a) System setup, (b) FPGA block diagram, (c) TCH block diagram.

Resource usages of the proposed FPGA chip designs are shown in Table 1. Consumption of logic elements (LEs) are 9,216 (8%), which involves 7,646 LEs for TCH. The FPGA chip is the Altera Cyclone IV EP4CE115F29C7 N. The proposed TCH was designed in RTL circuits in order to achieve real-time image processing. It can be seen that the TCH algorithm consumed the largest amount of LEs in the proposed skin color detection system that the algorithm required 21 9-bit embedded multipliers for RGB-HSV conversion and slope derivation with trigonometric functions. The embedded memory consumption was 57,400 bits (1.4%). This consumption was for the image buffer of touch screen and full color de-mosaicking.
Table 1

FPGA resource usages of proposed skin color detection

Chip family

Cyclone IV E

Device

EP4CE115F29C7 N

Total logic elements

9,216 LEs/114,480 (8%)

De-mosaicking and camera controller

1,275 (1.1%)

TC algorithm

7,646 (6.7%)

Touch screen control

295 (0.2%)

Embedded multiplier 9-bit elements

21/532 (3.9%)

Total embedded memory bits

57,400 bits/3,981,312 (1.4%)

Total PLLs

2/4 (50%)

The digital camera adopted in our design has a maximum of 5 M pixels resolution, but it was set for 800 × 480 pixels for use with the touch screen at a frame rate of 15fps and 30 MHz pixel clock. Meanwhile, the proposed TCH required 300 MHz clock speed and 37 clocks to run TCH during image pixel scans. Such arrangement leads to about four pixels delay showing on the touch screen.

Snapshots of real-life operations for proposed skin color detection are shown in Figure 6 without a video noise filter. Five severe test scenarios were designed to demonstrate the reliability of the proposed adaptive skin color detection system. We focused on the resistances to shadows and reflections at forehead, chin, hands, and the part of the face under the brim of a cap. Figure 6 demonstrates excellent skin color detection of TCH, which provides satisfactory ROI area for pattern recognition, object tracking, and image processing purposes. In these test scenarios, the skin color detections by using traditional RGB and HSV thresholds were also conducted to compare with our TCH algorithm using constant skin color map. We could see that the skin color detection using RGB threshold in these tests exhibited the worst detection rate while the proposed TCH algorithm had the best detection performance in tests.
Figure 6

Comparisons among traditional RGB and HSV thresholds and proposed TCH algorithm; (a)(e)(i)(m)(q) the test scenarios, (b)(f)(j)(n)(r) tested with RGB threshold, (c)(g)(k)(o)(s) tested with HSV threshold, (d)(h)(l)(p)(t) tested with proposed TCH.

In the first test scenario of Figure 6, see Figures 6(a) to (d), the interferences of shadows and reflections existed at the same time on tester’s face. It can be seen that the proposed TCH skin color detection system demonstrated excellent adaptive capability to detect skin areas with shadows and reflections. For the test result using HSV threshold, although it has been a popular method on embedded systems, the detection rates at testers’ chin and neck were still deteriorated by shadows. The detection performances between RGB and HSV thresholds had not shown much difference in this test. In the second test scenario, we tested TCH algorithm with extreme interference of shadow by huddling up a palm. The RGB and HSV thresholds both still had low detection rate in test. Comparatively, proposed TCH algorithm in this test could detect the skin color reaching the inner part of palm perfectly. In the third test scenario, we tested our skin detector by using green color to shift skin color moderately. The tester covered his palm with a green paper to obtain some green reflection on palm, as shown in Figure 6(i). It can be seen that the proposed TCH algorithm operating in such environment could still resist to green shadow and reflection. Again, RGB and HSV constant thresholds in this test still had low detection rate at fingers, where was covered by green shadow. After illustrated the success in resistance to shadows and reflections, the fourth test scenario simulated the condition of wearing a cap, as shown in Figure 6(m). In this test, the tester’s face was covered by shadow and some yellow reflections from inner brim of cap. The RGB threshold with this test scenario nearly failed to detect the tester’s face, and even the HSV threshold could only improve little. Nevertheless, our TCH algorithm could still work quite well in this test except a small part right under the brim of cap. In the last test scenario of Figure 6, the capability of skin detection under a dark environment was also tested, as shown in Figure 6(q). The lights in laboratory were nearly turned off in this test scenario. It can be seen the RGB and HSV thresholds could surprisingly detect skin region of the subject, and the silhouette of HSV threshold was sufficient for recognition on the face and palm. The proposed TCH skin color detection system under such severe environment was apparently superior to traditional HSV threshold for full scope detection rate of the subject’s face and palm.

For quantitative analysis, we tested TCH with different human races and illumination conditions. Tested images were all downloaded from the internet (Eastday [2013]; Gamebase [2008]; Gigacircle [2014]; Nextmedia [2013]; Sina house [2011]; Sohu sport [2010]; Vanity fair [2014]; 10202blog [2013]) and references and then printed by a color printer so that digital camera could capture these pictures into FPGA. However, it should be noted that this test method also incurred chromatic distortions due to imperfect reproduction in paper and from the printer. Figure 7 indicates that TCH could detect these candidate faces efficiently even in outdoor environments. With the same test method, we also compared with state-of-the-art methodologies given in Table 2, and experimental results are shown in Figure 8. The images for comparison were copied from articles and enlarged so that the TCH system was fed with images of larger chromatic distortions than that given in Figure 7. It can be seen that TCH with such severe test condition can also provide sufficient detection accuracy for ROI.
Figure 7

Proposed TCH without noise filter for working on various human races and illumination conditions with imperfect reproduction of pictures; (a)-(d) are from Vanity fair ([2014]), (e) is from Gamebase ([2008]), (f) and (g) are from Sina house ([2011]), (h) is from Gigacircle ([2014]), (i) is from 10202blog ([2013]), (j) is from Sohu sport ([2010]), (k) and (l) are from Eastday ([2013]), and (y) is from Nextmedia[2013].

Table 2

Comparisons of resource usages among different skin detector designs

Items

(1)

(2)

(3)

(4)

(5)

(6)

Resources

Chenaoua and Bouridane ([2006])

Fotouhi et al. ([2009])

Khan et al. ([2011])

Liu and Shi ([2011])

Zafarifar et al. ([2012])

Proposed TC

Platforms

PC

PC

PC

FPGA

FPGA

FPGA

EP2C20

Xilinx

Cyclone IV E

EP4CE115F29C

7 N

Methods

Markov random field

Texture analysis & ANN classifier

Cascaded skin training & classifier

YCbCr & I1I2I3 Classifiers

YUV & Adaptive texture analysis

Adaptive HSV constraints

Tested images

Static picture

Static picture

Static picture

Video

Video

Video

Adaptability

Yes

Yes

Yes

No

Yes

Yes

Resource usages (LEs/LUTs)

-

-

-

4,091

30,720

7,646

DSPs

-

-

-

No

192

No

Embedded memory (bits)

-

-

-

36,512

3,456,000

57,400

Figure 8

The comparison with skin detection methodologies from references; (a)-(e) are test scenarios from Table 2 , (1)-(5) with their results in (f)-(j), and (k)-(o) are for pixel-based TCH tested without noise filter.

Discussions

The salient features of the proposed skin color detection system involve simple calibration processes, moderate hardware resource usage on chips, and efficient adaptability without complicated training algorithms. Comparing with the state-of-the-art technologies listed in Table 2, the proposed TCH adaptive algorithm cooperating with a touch screen is comparable to the methodologies using statistical and training classifier for high detection rate and low implementation effort. Besides, the combination of different color thresholds may achieve sufficient detection rate on embedded system by using a small amount of hardware resources (Liu and Shi, [2011]), but such system can only work under adequate illumination and tedious calibration is still unavoidable because of the lack of adaptive mechanisms. After inspecting the skin detector results in Figure 8, the best skin color detection design for embedded system is the texture-based skin color analysis; see the performance in Figure 8(j). This kind of system is not only applicable for skin color detection but also provides additional functionality on background elimination, where it is colored with skin color. However, a large amount of hardware resources usages is required for complicated texture analysis. Consequently, it dramatically increases design difficulty on real-time processing speed and the implementation in a small chip.

In addition, the adaptability of TCH’s skin color detection is determined by the pixel saturation before exceeding the initial skin color map F. As shown in Figure 9, the pixel color before exceeding the initial skin color map at a high saturation value will result in a low adaptability to variation in skin color, and a null adaptability will be set if the exceeded color is at the upper saturation boundary of the initial skin color map, as shown in the Figure 9(a). In contrast, TCH’s adaptability will be increased if the exceeded color is at a low saturation value, and it results in maximum adaptability at the lower saturation boundary of initial skin color map, as shown in the Figure 9(b). The proposed non-linear adaptive mechanism is similar to human’s vision in that people can distinguish different colors easily between high saturation colors but not the low saturation ones.
Figure 9

Operations of TCH at upper and lower boundaries of adaptability; (a) upper saturation boundary, (b) lower saturation boundary.

Finally, the minor drawback of TCH arises from the delay of image processing and incurs coarse binary picture quality, as shown in Figures 6 to 8. With the pixel-based skin color detection, the image quality is sensitive to random noise in the camera, then the noise is enlarged horizontally due to the late update of pixels information during image acquisition. Although the coarse binary picture quality seems not affecting detection rate, the authors will determine how to improve processing speed with refined hardware circuits in the future, and a real-time and simplified pixel-based noise filter from our previous work may be also implemented (Yu et al. [2011]).

Conclusion

In this paper, we have proposed an efficient adaptive skin color detection system. The proposed algorithm can derive an initial skin color map using traditional HSV color space after captured images from touch screen. Once incoming pixel colors exceed the region of default skin color map, the non-linear adaptive algorithm of trajectory constraints on hue (TCH) can dynamically adjust the range of thresholds for detection. Various severe test scenarios had been arranged to verify the capability of proposed skin detection. Our experimental results indicate that the proposed adaptive skin color detection is efficient and applicable to various effects from illumination conditions. A reasonable amount of hardware resource usages in the FPGA chip can allow the TCH skin color detection to be implemented on single FPGA chip and suitable for miniature electronic devices. The future works involve the refinement of circuits in economical gates count and processing delay improvement.

Declarations

Acknowledgments

Supports from Ministry of Science and Technology, funded by the government of Taiwan under Grant NSC 99-2221-E-027-057-MY3, Ministry of Education Taiwan for the Top University Project to the National Cheng Kung University (NCKU), and Mr. En-Shi Lin are gratefully acknowledged.

Authors’ Affiliations

(1)
Department of Electrical Engineering, National Chung Cheng University
(2)
Department of Electrical Engineering, Tamkang University
(3)
School of Mechanical and Manufacturing Engineering, The University of New South Wales
(4)
Department of Computer Science and Information Engineering, National Cheng Kung University
(5)
Department of Electrical Engineering, Chung Yuan Christian University

References

  1. 10202blog:Photos collection. 2013. http://swjhgrade10202.blogspot.tw/2013_10_01_archive.html. Accessed 10. July. 2014.Google Scholar
  2. Chaves-González JM, Vega-Rodríguez MA, Gómez-Pulido JA, Sánchez-Pérez JM: Detecting skin in face recognition systems: a colour spaces study. Digital Image Processing 2010, 20: 806–823. 10.1016/j.dsp.2009.10.008View ArticleGoogle Scholar
  3. Chen YH, Hu KT, Ruan SJ: Statistical skin color detection method without color transformation for real-time surveillance systems. Engineering Application of Artificial Intelligence 2012, 25(7):1331–1337. 10.1016/j.engappai.2012.02.019View ArticleGoogle Scholar
  4. Chenaoua K, Bouridane A: Skin Detection Using a Markov Random Field and a new Color Space (IEEE international conference on image processing (ICIP 2006)). Georgia Institute of Technology, Atlanta, USA; 2006.Google Scholar
  5. Eastday (2013) Photos collection. http://big5.eastday.com:82/gate/big5/msnphoto.eastday.com/hdqxb2013/20130717_9/index.html. Accessed 10. July. 2014.Google Scholar
  6. Fotouhi M, Rohban MH, Kasaei S: Skin Detection Using Contourlet-Based Texture Analysis (4th international conference on digital telecommunication (ICDT 2009)). University of Haute Alsace, Colmar, France; 2009.Google Scholar
  7. Gamebase (2008) Photos collection. http://www.gamebase.com.tw/forum/5351/topic/83232878/1. Accessed 10. July. 2014.Google Scholar
  8. Gigacircle (2014) Photo collection. http://tw.gigacircle.com/1039866-1. Accessed 10. July. 2014.Google Scholar
  9. Khan S, Gupta GS, Demidenko S: Adaptive Classifier for Robust Detection of Signing Articulators Based on Skin Colour (6th IEEE international symposium on electronic design, test and application (DELTA 2011)). Institute of Electrical and Electronics Engineers Incorporated, Queenstown, New Zealand; 2011.Google Scholar
  10. Kim KK, Kwak KC, Chi SU: Gesture Analysis for Human-Robot Interaction (8th international conference on advanced communication technology (ICACT 2006)). Institute of Electrical and Electronics Engineers Incorporated, Gangwon-Do, Korea; 2006.Google Scholar
  11. Kumar A: An Empirical Study of Selection of the Appropriate Color Space for Skin Detection: a Case of Face Detection in Color Images (2nd International conference on issues and challenge in intelligent computing techniques (ICICT 2014)). Krishna Institute of Engineering and Technology, Ghaziabad, India; 2014.Google Scholar
  12. Li C, Liu YH, Shih HC: Adaptive Skin Color Tone Detection With Morphology-Based Model Refinement (9th international conference on information, communication, and signal processing (ICICS 2013)). National Cheng Kung University, Tainan, Taiwan; 2013.Google Scholar
  13. Lin C, Su CH, Huang HS: Colour image segmentation using relative values of RGB in various illumination circumstances. Int J Comput 2011, 5(2):252–261.MathSciNetGoogle Scholar
  14. Liu Q, Peng GZ: A Robust Skin Color Based Face Detection Algorithm (2nd international conference on information in control, automation, and robotics (CAR 2010)). Hubei University of Education, Wuhan, China; 2010.Google Scholar
  15. Liu G, Shi Z: Embedded Implementation of Real-Time Skin Detection System (International conference on transportation, mechanicl, and electrical engineering (TMEE 2011)). Jilin University, Changchun, China; 2011.Google Scholar
  16. Liu L, Sang N, Yang S, Hunag R: Real-time skin color detection under rapidly changing illumination conditions. IEEE Trans Consum Electron 2009, 57(3):1295–1302. 10.1109/TCE.2011.6018887View ArticleGoogle Scholar
  17. Luo RC, Chang SR, Yang YP: Tracking With Pointing Gesture Recognition for Human-Robot Interaction (IEEE/SICE international symposium on system integration (SII-2011)). Kyoto University, Kyoto, Japan; 2011.Google Scholar
  18. Nextmedia (2013). Photos collection. http://hk.apple.nextmedia.com/realtime/news/20130826/51675805. Accessed 10. July. 2014.Google Scholar
  19. Sagheer, A, & Aly, S. (2012). An Effective Face Detection Algorithm based on Skin Color Information (8th international conference on signal image technology and internet based systems (SITIS 2012)). Naples, Italy.Google Scholar
  20. Sina house (2011). Photos collection. http://sx.bbs.house.sina.com.cn/thread-851846640456301-2.html. Accessed 10. July. 2014.Google Scholar
  21. Sohu sport (2010). Photos collection. http://sports.sohu.com/20100320/n270968863.shtml. Accessed 10. July. 2014.Google Scholar
  22. Toledo FJ, Martínez JJ, Garrigós J, Ferrández J, Rodellar V: Sink Color Detection for Real Time Mobile Applications (16th international conference on field programmable logic and applications (FPL 2006)). Universidad Autónoma de Madrid, Madrid, Spain; 2006.Google Scholar
  23. Vanity fair (2014). Photo: The 2014 Vanity Fair Hollywood Portfolio. http://www.vanityfair.com/hollywood/photos/2014/03/chuck-close-hollywoodportfolio#6. Accessed 10. July. 2014.Google Scholar
  24. Vezhnevets V, Sazonov V, Andreeva A: A Survey on Pixel-Based Skin Color Detection Techniques (13th international conference on computer graphics (GraphiCon 2003)). Moscow State University, Moscow, Russian Federation; 2003.Google Scholar
  25. Yang G, Ren J, Huang C, Li H: Research on a Skin Color Detection Algorithm Based on Hybrid Color Model (3rd international congress on image signal processing (CISP 2010)). Yantai University, Yantai, China; 2010.Google Scholar
  26. Yang L, Li H, Wu XY, Zhao D: An Algorithm of Skin Detection Based on Texture (IEEE international congress on image and signal processing (CISP 2011)). Donghua University, Shanghai, China; 2011.Google Scholar
  27. Yu YH, Kwok NM, Ha QP: Color tracking for multiple robot control using a system-on-programmable-chip. Autom Constr 2011, 20: 669–676. 10.1016/j.autcon.2011.04.013View ArticleGoogle Scholar
  28. Yu YH, Yeh CH, Lee TT, Chen PY, Shiau YH: Chip-based Real-time Gesture Tracking for Construction Robot’s Guidance (31st international symposium on automation and robotics in construction and mining (ISARC 2014)). The University of Technology Sydney, Sydney, Australia; 2014.Google Scholar
  29. Zafarifar B, van den Kerkhof T, de With PHN: Texture-adaptive skin detection for TV and its real-time implementation on DSP and FPGA. IEEE Trans Consum Electron 2012, 58(1):161–169. 10.1109/TCE.2012.6170069View ArticleGoogle Scholar
  30. Zheng Y, Yang G, Wu L: Fast Face Detection in Field Programmable Gate Array (International conference on digital manufacturing and automation (ICDMA 2010)). Institute of Electrical and Electronics Engineers Incorporated, ChangSha, China; 2010.Google Scholar
  31. Zhu JQ, Cai CH: Region Growing Based High Brightness Skin Detection (10th international symposium on signals, circuits, and systems (ISSCS 2011)). Gheorghe Asachi Technical University, Lasi, Romania; 2011.Google Scholar

Copyright

© Yu et al.; licensee Springer. 2014

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.