In this project, we write functions that allow us to morph faces, create a mean face of a population, and more.

Part 1: Defining Correspondences

First, to be able to morph two faces, we first need to definte corresponding points between them. For example, we can label where the eyes of each person are, the mouth of each person, and so on. The important part is that the points are chosen in the same order for each face so that the correct features are corresponding. To do this, I used the tool created by a previous year’s student that was provided to us. Then, once we have the corresponding points, we need to compute a triangulation of the point sets. This is so that we can morph between the corresponding triangles from each image. I used Delaunay triangulation on the average point set. This means that I just took the average of the two point sets, and did a triangulation of this since it tends to prevent us from getting deformed triangles. Below is the triangulation and points of both images.

Above we see the triangulation and points for image A (Christian Bale) and B (Kanye West).

Part 2: Computing the Mid-way face

Next, we had to compute the mid-way face, meaning we want to warp halfway between image A and image B. To do this, I implemented two primary functions, computeAffine, and morph. computeAffine takes the points of two triangles, and then returns the transformation matrix used to transform the first triangle into the second. Then, the morph function takes in two images, their point sets, the triangulation, and warp and dissolve fractions. These fractions determine how much we use the shape and colors of each face for our resulting morphed image. For example if, both are .5, we just warp to the average point set and use half of the color intensity for each image. Specifically, to do this midway face, I used my functions to first warp each image to the average point set, and then I averaged the pixel color values. I will explain these two functions more below, but first here is the resulting midway face of Christian and Kanye.

computeAffine and Morph:

Like I mentioned, the computeAffine function takes in the corner points of two triangles. Then it solves a system of equations to determine the matrix M that can be used to transform the first triangle to the second. This is very useful because we can then call this function in morph, when we looping over each triangle and morphing the corresponding pixels within those triangles in each image. In this implementation, we had to do an inverse warping, meaning that we consider each triangle in the destination image, and then compute the transformation between this triangle and the two source triangles. Then we transform all of the points in the destination triangle using the transformation matrices, and this gives us the pixel values we need from the source images. However, one issue that comes up with inverse warping is that we can land in between pixels when we transform to the source images. Therefore, I used bilinear interpolation, which essentially interpolates between the surrounding pixel values if we land in between pixels.

Part 3: The Morph Sequence

Now that we have showed that the functions work, we can perform a full morph sequence between the two faces. This can be done by just calling the morph function many times, each time passing in different warp fractions and dissolve fractions. With a warp or dissolve fraction of 1 (in my implementation) we use Christian Bale’s points or colors, and we use Kanye’s for values of 0. I created a gif showing the transformation between the two by calling the function 45 times with decreasing fractions, so that we get 45 frames showing the transformation. Then, I display each frame for 1/30 of a second.

Part 4: The “mean face” of a Population

In this part, we compute the mean face of a population. I chose to use the FEI dataset, was created at a lab in Brazil, and contains photos of many different people taken in the same conditions. I specifically used a subset of 200 images that were head-on, and where the people had a neutral face. The dataset also comes with key point annotations for each of the photos. First, I created the mean face shape by just averaging the point sets for all 200 images. Then, I warped each face to this mean shape of the population, meaning I just warped but did not do any cross-dissolving. Here are some examples of this.

Left is the original and right is warped to the average shape.

Finally for this part, we actually compute the mean face. We already had the mean face shape, and have each face warped to the mean face shape. To get the mean face, we just have to average all of the images that have already been warped to the mean shape. Here is the result.

Then, here is my face , my face warped to the average geometry, and the mean face warped to my geometry:

I think different focal lengths possibly contributed to the results looking a little strange, particularly around the eyes and nose. It’s kind of interesting that the average face looks more feminine when warped to my geometry.

Part 5: Caricatures Extrapolated from the Mean

Now that we have the population mean, we can actually make a caricature by extrapolating from the mean. Specifically, we similar to morphing between faces, we can use some value alpha, where we take the average of the geometry for my face weighted by alpha, and the geometry of the mean face weighted by 1 - alpha, and then warp my face to that. However, if we now use alpha values greater than 1, we will be exaggerating my features, and if we use values less than 0 we will be exaggerating the mean features.

Left is original me (alpha=1), middle is positive alpha, and right is negative alpha

Part 6: Bells and Whistles

For the Bells and Whistles, I chose to do the option of changing the gender/ethnicity of my face. I thought it would be interesting to try to do both at the same time, so I found this image of the mean Chinese woman and used that for morphing. First, I just warped my face to her geometry. The results weren’t great and I tried using multiple different sets of key points but just couldn’t get a good warp. Again, I think this might have been caused partially by different focal lengths.

Then I warped her face to my geometry:

Finally, I warped my face to her geometry, and then set the dissolve fraction to .5, so that I was using the color from both my face and hers. I think this one had the best result.