Blog Post Actions
Group Actions
Engagement
  • Author Author: amgalbu
  • Date Created: Date Created
  • Views 104 views
  • Likes 8 likes
  • Comments 2 comments
Related
Recommended

Haunted Mirror

amgalbu

Haunted mirrors is quite common in horror movies. An (incomplete) list of movies which plots rotates around a mirror are

  • Mirrors (2008)
  • Bloody Mary (2006)
  • Dark Mirror (2007)
  • Look Away (2018)
  • The Broken (2008)
  • The Hole in the Ground (2019)
  • Oculius (2013)
  • Candyman (1992)
  • Poltergeist (1982), which, even after 43 years, remains one of the most revered horror films

In particular, I got inspiration for this project from this clip from Candyman

While pondering about this project, I ended up asking myself "why mirrors are so often used in horror movies?". So I did what every person does when in doubt: I asked Bard. Here is its response

image

Quite exhaustive I would say...

Back to the project, basically what I want to build is a smart mirror that can detect when a person is looking at himself in the mirror. When a face is detected, some spooky clips are played exactly where the face is detected. This gives the impression the ghost or the hideous animal is walking just on your face. As an alternative, ghosts can be added to the reflected image so that they looks are they are just behind you.

But let's get started. What I need is

  1. a monitor. I have a leftover HD monitor. It's working but it is quite old and has a form factor of 4:3. This causes all the recent desktop environment to be truncated. So it's the perfect victim for this project
  2. a computer module that will run the application that generates the special effects. I have a LattePanda 3 Delta board that Element14 kindly sent me to be roadtested. This is a better choice than a Raspberry Pi or any other ARM-based board because its computational power will allow me to play with quite sophisticated image processing algorithms
  3. a webcam, that will be connected to the computer module and will frame the area just in front of the mirror
  4. a two-way film, to make the false mirror. The film will be glued to a sheet of synthetic glass, that will be then mounted just in front of the monitor
  5. some wood, to build a frame a make the mirror more realistic

1. The frame

The frame is made from wood. A front frame slides just on the existing monitor and is kept in place by some holders (see pictures).

{gallery}Haunter Mirror

image

image

image

image

image
image

2. The sofware

The software is written in Pyhton3 and will use OpenCV2 for the image processing. There are basically two main functions the software does

  • detect when a person is in front of the mirror. This is achieved by looking for faces in the image captured by the camera  
  • when a face is detected, a scary movie is overlaid on the webcam stream to reproduce a "haunted mirror" effect

2.1 Software installation

Before proceeding, let's install all the necessary software and libraries. I started from a clean Ubuntu 22.04 distribution. 

2.1 Face detection

Implementing face detection with OpenCV is just a matter of a few lines of code. First you need to instantiate the

# Load the cascade
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')

then call the detectMultiScale method for each frame captured by the webcam

# Detect the faces
faces = face_cascade.detectMultiScale(gray, 1.1, minSize, 1000)

minSize is the minimum size of the detected faces. Faces smaller that this threshold are ignored. This methods returns an array of "rectangles". Every "rectangle" is an array of four values, represeting respectivaly the x and y coordinates of the position of the area where the face has been detected, and the width and height of such an area

So, to determine if a face has been found, simply check if the size of the returned array is greater than zero

if (len(faces) > 0):
     # a face has been detected


2.2 Background removal

To remove the background, there is again a convenient function in CVZone: the SelfiSegmentation class

from cvzone.SelfiSegmentationModule import SelfiSegmentation
segmentor = SelfiSegmentation()

To create an image where the background has been removed, simply call

imgNoBg = segmentor.removeBG(camImg, green)

where camImg is the image captured by the webcam and imgNoBg is the image where the background has been replaced by the color passed as the second parameter of the removeBG function. In this case, the background is filled with a green color, which is perfect for later processing that involves chroma keying.

2.3 Playing the spooky clip

To make the spooky more realistic, I added some metadata for each clip. Metadata includes the following information

  • whether the clip should overlap the detected face or be placed in a free region (i.e. part of the video frame which is not occupied by the face)
  • the preferred position (top-right, top-center, top-left, bottom-right or bottom-left)

When the clip has to be played in a free region, some masks operations are performed over the spooky clip to give the impression that the ghost is just behind you. This basically means that the part of the spooky clip that overlaps the face, is masked out. To achieve this results, several steps are required, which I am trying to explain in more details now

This is the original image

image

Background is removed

imgNoBg = segmentor.removeBG(camImg, green)

image

Then I extracted mask of my region on interest (i.e. the area where the overlay will be shown

imgMask = cv2.inRange(imgNoBg, green, green)
imgMask = imgMask[startRow:endRow, startCol:endCol]

image 

The overlay image is loaded

image

and combined with the mask extracted from the image captured by the camera

imgMaskedGhost = cv2.bitwise_and(overlayImg, overlayImg, mask=imgMask)

image

Black pixels are converted to the chroma-key color to obtain the final overlay image

# get (i, j) positions of all RGB pixels that are black (i.e. [0, 0, 0])
black_pixels = np.where(
  (imgMaskedGhost[:, :, 0] == 0) &
  (imgMaskedGhost[:, :, 1] == 0) &
  (imgMaskedGhost[:, :, 2] == 0)

# set those pixels to green
imgMaskedGhost[black_pixels] = green

image

In order to eliminate the artifacts introduced by the removeBG functions, we are now using the masked overlay image to create a mask that will be applied to the webcam image. 

The masked overlay image is converted to black-and-white

imgGhostMask = cv2.inRange(imgMaskedGhostHSV, l_green, u_green)

image

and then inverted

imgGhostMaskInv = cv2.bitwise_not(imgGhostMask)

image

The mask is the applied to the ROI on the original image

roi = camImg[startRow:endRow, startCol:endCol]
camImgBg = cv2.bitwise_and(roi, roi, mask = imgGhostMask)

image

Finally we get the final images, where we take the original overlay image, mask it with the imgGhostMaskInv  mask to obtain the image that will be superimposed to the original image

overlayImgFg = cv2.bitwise_and(overlayImg, overlayImg, mask = imgGhostMaskInv)

This image is now added to the original image (only the ROI is involved in this operation)

final = cv2.add(camImgBg, overlayImgFg)

Finally, the combined image is copied into the original image

camImg[startRow:endRow, startCol:endCol] = final

image

For the haunted mirror, we are using a two-way reflective film. So we want to show only the masked ghost and leave the rest of the scene black. To achieve this, we copy the masked ghost into a black image

fullscreenImg = np.zeros((1080, 1920, 3), dtype = np.uint8)
fullscreenImg[startRow:endRow, startCol:endCol] = overlayImgFg

image

3. Final demo

Python source code of the application available here