Computer Vision and Projection Mapping in Python

Identify and Track Faces

Ok, now that we've identified our project-able region, it's time to watch for and identify faces! We don't care about the regions of the image that the camera can see but can't interact with, so we only need to work with the corrected and cropped portion that is our project-able region, which we found in the last section.

Detecting faces is where dlib really excels. In the past I've used openCV for face detection (Harr cascades) and while it gets the job done, the detected faces from dlib are far more flexible, and give information about facial features, among other things.

Setting up our face detector is as follows:

language:python
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor(args.get('predictor'))

You can use a variety of face predictors, but I've used the generic predictor available directly from dlib here.

Don't forget to extract your download:

language:python
> bzip2 -d shape_predictor_68_face_landmarks.dat.bz2

Here's an example of getting our faces from our image:

language:python
rects = detector(image, 0)
for mark in rects:
    shape = predictor(image, mark)
    shape = face_utils.shape_to_np(shape)

This is the basic usage for finding faces. You'll see when we put this all together how we can use this data to overlay our sprites.