CS180 Project 3

Goal

The goal of this assignment is to experiment with face morphing techniques. Face morphing can be performed by defining key points, creating a triangular mesh for the reference images, and applying affine transformations to turn one image into another.

Part 1

This part of the project involved defining correspondences between two reference images. The correspondences would be useful for defining the Delaunay triangulation (triangular mesh) for the reference images. The Delaunay triangulation divides each reference image into local segments that are used to provide information on pixel values for the morphed image. For this part, I used the provided online tool to define 37 key point correspondences (excluding the four image corner key points). I defined the Delaunay triangulation for the average of the two point sets (midway points), since that would lead to less warping in further steps. The results are as follows:

Part 2

For this part, I was tasked with computing a “midway face” between Image A and Image B. I implemented code to find an affine transformation between each triangle in the “midway image” and each triangle in the reference images. Using the skimage.draw.polygon function, I was able to get all the pixels in the midway image triangle and find the corresponding pixels in each reference image. I would take the values of the corresponding pixels in each reference image and apply them to the midway image. For points that fell “in between pixels” in the reference images (i.e. non-integer coordinates), I used nearest-neighbor interpolation via spicy.interpolate.griddata to estimate values for the corresponding pixels in the midway image. I averaged the values obtained from the reference images to get the values of the midway image. After iterating through all the triangles in the midway image mesh, I obtained the following results:

Part 3

For this part, I used my findings from Part 2 to generate a gif/morph sequence from Image A to Image B. For the gif/morph sequence, I generated 45 frames. For each frame, I used one weight in the interval [0, 1] to modulate both the warping (affects triangulation, affine transformations) and the cross-dissolving (affects pixel value) of the reference images.

Part 4

For this part, I was tasked with computing the mean face of a population of images. For this and subsequent parts, I used the Danes Dataset. The version of the Danes Dataset I used for this project includes 240 portraits of Danes. The portraits contain various poses and facial expressions. Each portrait has a corresponding .asf file, which contains 58 relevant key point information for facial features. I parsed the .asf files to gain a numpy array of key points for each image. I computed the mean shape of the faces in the Danes Dataset, and morphed each image in the dataset onto the mean geometry. Here are a few examples:

After computing all images warped onto the mean geometry, I simply averaged the values of all warped images to obtain the average Dane face of the Danes Dataset:

I also calculated my face warped onto the Average Face geometry:

…as well as the Average Face warped onto my face geometry:

Part 5

For this part, I used the Danes Dataset again to produce a caricature of myself. This time, I calculated the deviation of my face from the average male Dane face, multiplied this deviation by alpha=1.5, and added the result to the average male Dane face. This way, I emphasized the differences between my face and the average male Dane face:

The above caricature shows that at least from ratios present in each image, my eyes are bigger and set further apart compared to the average male Dane face.

Bells and Whistles

For this component of the project, I experimented further with the average male Dane face by attempting to modify the ethnicity of my face. For this part, I first experimented with morphing my face onto the geometry of the average male Dane face:

Next, I experimented with morphing just my appearance into that of the average male Dane face:

And for the last step, I morphed both my appearance and shape onto the average male Dane face: