Haunted mirrors is quite common in horror movies. An (incomplete) list of movies which plots rotates around a mirror are
- Mirrors (2008)
- Bloody Mary (2006)
- Dark Mirror (2007)
- Look Away (2018)
- The Broken (2008)
- The Hole in the Ground (2019)
- Oculius (2013)
- Candyman (1992)
- Poltergeist (1982), which, even after 43 years, remains one of the most revered horror films
In particular, I got inspiration for this project from this clip from Candyman
While pondering about this project, I ended up asking myself "why mirrors are so often used in horror movies?". So I did what every person does when in doubt: I asked Bard. Here is its response
Quite exhaustive I would say...
Back to the project, basically what I want to build is a smart mirror that can detect when a person is looking at himself in the mirror. When a face is detected, some spooky clips are played exactly where the face is detected. This gives the impression the ghost or the hideous animal is walking just on your face. As an alternative, ghosts can be added to the reflected image so that they looks are they are just behind you.
But let's get started. What I need is
- a monitor. I have a leftover HD monitor. It's working but it is quite old and has a form factor of 4:3. This causes all the recent desktop environment to be truncated. So it's the perfect victim for this project
- a computer module that will run the application that generates the special effects. I have a LattePanda 3 Delta board that Element14 kindly sent me to be roadtested. This is a better choice than a Raspberry Pi or any other ARM-based board because its computational power will allow me to play with quite sophisticated image processing algorithms
- a webcam, that will be connected to the computer module and will frame the area just in front of the mirror
- a two-way film, to make the false mirror. The film will be glued to a sheet of synthetic glass, that will be then mounted just in front of the monitor
- some wood, to build a frame a make the mirror more realistic
1. The frame
The frame is made from wood. A front frame slides just on the existing monitor and is kept in place by some holders (see pictures).
| {gallery}Haunter Mirror |
|---|
|
|
|
|
 |
|
|
 |
 |
2. The sofware
The software is written in Pyhton3 and will use OpenCV2 for the image processing. There are basically two main functions the software does
- detect when a person is in front of the mirror. This is achieved by looking for faces in the image captured by the camera
- when a face is detected, a scary movie is overlaid on the webcam stream to reproduce a "haunted mirror" effect
2.1 Software installation
Before proceeding, let's install all the necessary software and libraries. I started from a clean Ubuntu 22.04 distribution.
- install OpenCV
sudo apt install python3-opencv - Install CVzone
pip3 install cvzone - install numpy (cvzone requires a specific version of numpy, so this component must be updated manually)
pip install numpy=1.26.1 - download Haar cascade https://github.com/kipr/opencv/blob/master/data/haarcascades/haarcascade_frontalface_default.xml and save it to the folder where your source code is
- (still WIP) install VLC
sudo snap install vlv
pip3 install python-vlc
2.1 Face detection
Implementing face detection with OpenCV is just a matter of a few lines of code. First you need to instantiate the
# Load the cascade
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
then call the detectMultiScale method for each frame captured by the webcam
# Detect the faces
faces = face_cascade.detectMultiScale(gray, 1.1, minSize, 1000)
minSize is the minimum size of the detected faces. Faces smaller that this threshold are ignored. This methods returns an array of "rectangles". Every "rectangle" is an array of four values, represeting respectivaly the x and y coordinates of the position of the area where the face has been detected, and the width and height of such an area
So, to determine if a face has been found, simply check if the size of the returned array is greater than zero
if (len(faces) > 0):
# a face has been detected
2.2 Background removal
To remove the background, there is again a convenient function in CVZone: the SelfiSegmentation class
from cvzone.SelfiSegmentationModule import SelfiSegmentation
segmentor = SelfiSegmentation()
To create an image where the background has been removed, simply call
imgNoBg = segmentor.removeBG(camImg, green)
where camImg is the image captured by the webcam and imgNoBg is the image where the background has been replaced by the color passed as the second parameter of the removeBG function. In this case, the background is filled with a green color, which is perfect for later processing that involves chroma keying.
2.3 Playing the spooky clip
To make the spooky more realistic, I added some metadata for each clip. Metadata includes the following information
- whether the clip should overlap the detected face or be placed in a free region (i.e. part of the video frame which is not occupied by the face)
- the preferred position (top-right, top-center, top-left, bottom-right or bottom-left)
When the clip has to be played in a free region, some masks operations are performed over the spooky clip to give the impression that the ghost is just behind you. This basically means that the part of the spooky clip that overlaps the face, is masked out. To achieve this results, several steps are required, which I am trying to explain in more details now
This is the original image
Background is removed
imgNoBg = segmentor.removeBG(camImg, green)
Then I extracted mask of my region on interest (i.e. the area where the overlay will be shown
imgMask = cv2.inRange(imgNoBg, green, green)
imgMask = imgMask[startRow:endRow, startCol:endCol]
The overlay image is loaded
and combined with the mask extracted from the image captured by the camera
imgMaskedGhost = cv2.bitwise_and(overlayImg, overlayImg, mask=imgMask)
Black pixels are converted to the chroma-key color to obtain the final overlay image
# get (i, j) positions of all RGB pixels that are black (i.e. [0, 0, 0])
black_pixels = np.where(
(imgMaskedGhost[:, :, 0] == 0) &
(imgMaskedGhost[:, :, 1] == 0) &
(imgMaskedGhost[:, :, 2] == 0)
# set those pixels to green
imgMaskedGhost[black_pixels] = green
In order to eliminate the artifacts introduced by the removeBG functions, we are now using the masked overlay image to create a mask that will be applied to the webcam image.
The masked overlay image is converted to black-and-white
imgGhostMask = cv2.inRange(imgMaskedGhostHSV, l_green, u_green)
and then inverted
imgGhostMaskInv = cv2.bitwise_not(imgGhostMask)
The mask is the applied to the ROI on the original image
roi = camImg[startRow:endRow, startCol:endCol]
camImgBg = cv2.bitwise_and(roi, roi, mask = imgGhostMask)
Finally we get the final images, where we take the original overlay image, mask it with the imgGhostMaskInv mask to obtain the image that will be superimposed to the original image
overlayImgFg = cv2.bitwise_and(overlayImg, overlayImg, mask = imgGhostMaskInv)
This image is now added to the original image (only the ROI is involved in this operation)
final = cv2.add(camImgBg, overlayImgFg)
Finally, the combined image is copied into the original image
camImg[startRow:endRow, startCol:endCol] = final
For the haunted mirror, we are using a two-way reflective film. So we want to show only the masked ghost and leave the rest of the scene black. To achieve this, we copy the masked ghost into a black image
fullscreenImg = np.zeros((1080, 1920, 3), dtype = np.uint8)
fullscreenImg[startRow:endRow, startCol:endCol] = overlayImgFg
3. Final demo
Python source code of the application available here
