CS585:Image and Video Computing
Homework 1: face detection
Problem Definition:
We were asked to create a program in Java or C to detect a face in an image. The method we were asked to use was to detect the color of the subject's skin, and turn all non-skin pixels black. This would be very useful in any application involving the detecting people. For example (and this is currently implemented) a digital camera can use this feature to find people in its field of view and make sure they are in focus. Or this could be used in countingg people passing by a camera, or many other scenarios.
Method & Implementation:
I set out to solve the problem by first calculating a skincolor from the image, and then scanning through it to turn all pixels outside a certain tolerance of this color to black. To do this I used two main functions, with a nested call. The functions were called by:
img1=colorImage(20,getSkinColor(30,img),img);
the functions are as follows:
public Color getSkinColor(int thresh, BufferedImage image): scan through a 5x5 square in the dead center of the image and average the RGB values of all pixels that do not stray from the maximum values by more than a certain threshold. This is implemented by first scanning through them (with two nested loops) to determine the brightest pixel (if red, green, and blue values are greater than the previous max), then scanning through and summing each pixel and an average value, and dividng by two. Pixels whose difference with max is greather than the threshold are excluded.
public BufferedImage colorImage(int thresh1, Color skin, BufferedImage image): given the skin color and a second threshold value, set the color of all pixels who do not equal the skin color +/- the threshold values (must pass this test for each color value) to the color black.
Experimental Results:
I found that the first threshold did not make much of a difference except at very low values (about 5 and below). In that case it would return black or a very light pixel. I found that a threshold range of 20-30
This changes the threshold in the colorImage method. Don't scroll too much as the grapics are pretty buggy.
| Source Image |
Threshold=20 |
Threshold=25 |
Threshold=30 |
 |
 |
 |
 |
Discussion and Conclusion:
- Were you able to find a large portion of the face ("true positive
detection")?
-I was able to find the majority of the face, though some was obscured by my beard.
-
Which pixels are missing ("false negative detection")?
-The pixels in my beard were gone, and some of those on my forehead(shiny because I had just been in a snowball fight and was wet) and my beard (black and red-ish). For very narrow tolerances, my flushed cheeks also were excluded
-
Did your method falsely label pixels in the background as faces
("false positive detection")?
-Yes, apparently I am very white, because part of the background showed up for larger thresholds. I think I cast a shadow on it well.
-
-What are the limitations of your current skin-based face detection
method? What would you do to try to overcome these difficulties and
develop a reliable face detector (if you had a lot more time...)?
This method only detects pixels of a certain color. If I happened to be in front of a pink or beige background, it would probably be too close to my skincolor, and a tolerance low enough to exclude it would exclude most of my face. Also, as is clear in my picture, facial features like rosy cheeks (which i had) and a beard (which i have) will exclude parts of the face. Also people with skin diseases or birthmakrs which create splotches would throw off the detector, and stage make-up or similar face paint (like a clown with a red nose) would cause the face to be unable to be detected. Also, a larger flaw than all those above is that the skincolor detecion requires skin to be centered on the image, which is hard to guarantee for anything but a mug shot. To improve this, I would probably include a library of skincolor values to compare this to, so that I can be sure that what I detected is in fact skin, and I would include shape in my criteria for what a face is. Also, if I stuck with color, I would try to find the center of the face and work outward until I hit a region with no skin. This way even if the background was close to skin color, I could detect the jump from skin to not-skin and that might help to avoid including such a region. I've found from other images as well that hair forms a border which would contain this detection nicely.
-
How can you describe the location of a face in an image?
-To describe the location, one could keep track of a bounding box, a traced outline, a central point, or a binary matrix which flagged each pixel of the image as including the face. Each of the preceeding methods are increasingly costly in terms of the data needed to represent them, but given that we are using image analysis, our data set is large to begin with.
Link to code