When challenges are challenging, it may happen to fail. And this is one of those cases...

Downsizing the neural network

I tried to run the neural network on the Arduino Nano 33 IoT board and... the board became unusable (I suspect because of a failure in a memory allocation operation). Luckily, I have some experience on these scenarios, as you can read here.

So my first attempt was to reduce the number of parameters in the neural network, by reducing both the number of inputs and the number of nodes in the hidden layers

The only neural network I was able to run on the Arduino Nano 33 IoT board was a neural network with just 4 feature and 10 nodes in the hidden layer. These are the metrics for such a neural network

Accuracy - Train: 0.772000

Loss - Train: 0.697, Test: 0.694

Precision: 0.6816326530612244

Recall: 0.4575342465753425

F1-score: 0.5475409836065573

AUC: 0.6794613632478063

What I can figure out here is that a coin toss can predict the fall better than the neural network.

I have to options at this point

• Option 1: run the neural network in one of the many cloud services available out there
• Option 2: implement a classic threshold-based algorithm

Because of considerations about battery consumption for option 1 (you need to keep WiFi constantly on to transfer data to the cloud), I choose to follow the second option and implement a threshold-based fall detection algorithm

Fall data analysis

To implement our algorithm, let's start with the analysis of the acceleration data from the IMU

This is what the accelerometer readings when a person stands up

And this is the accelerations in case of a fall

As one can see, they are completely different

There are some notable characteristics that can be used to detect a fall

1. start of the fall: the phenomenon of weightlessness will always occur at the start of a fall. It will become more significant during free fall, and the vector sum of acceleration will tend toward 0 g (see the yellow line); the duration of that condition will depend on the height of freefall. Even though weightlessness during an ordinary fall is not as significant as that during a freefall, the vector sum of acceleration will still be substantially less than 1 g (while it is generally greater than 1 g under normal conditions)
2. aftermath: Generally speaking, the human body, after falling and making impact, can not rise immediately; rather it remains in a motionless position for a short period (or longer as a possible sign of unconsciousness)
3. impact: between the "start of the fall" and the "aftermath" phases, the human body will impact the ground or other objects; the acceleration curve shows this as a large shock.
4. comparing before and after: after a fall, the individual’s body will be in a different orientation than before, so the static acceleration in three axes will be different from the initial status before the fall. Suppose that the fall detector is belt-wired on the individual’s body, to provide the entire history of acceleration, including the initial status. We can read the acceleration data in all three axes and compare those sampling data with the initial status. In figure, it is evident that the body fell on its side, since the static acceleration has changed from –1 g on the Y axis to +1 g on the Z-axis. So the fourth basis for determining a fall is if the difference between sampling data and initial status exceeds a certain threshold, for example, 0.7 g.

Fall detection algorithm

These are the basic concept I will use to detect falls. So my algorithm will

• make a moving average of the accelerations of the last few seconds to determine the initial condition
• if the acceleration norm drops below a certain threshold, than expect an impact
• if the acceleration norm gets higher than a given threshold within 0.5 seconds, than an impact has been detected
• wait 5 seconds
• monitor activity for 5 seconds. if no activity is detected and the position is different from initial status, then a fall has occurred

Here is the code for the fall detection algorithm

int AceFallDetector::addSample(float ax, float ay, float az, float gx, float gy, float gz)
{
  int currMillis = millis();
  int delta = currMillis - prevMillis;
  prevMillis = currMillis;


  // store and update data about initial position
  SENSOR_DATA* sd = &values[valueIdx];
  sd->ax = ax;
  sd->ay = ay;
  sd->az = az;
  sd->gx = gx / FALLDET_DEG_SCALE;
  sd->gy = gy / FALLDET_DEG_SCALE;
  sd->gz = gz / FALLDET_DEG_SCALE;
  valueIdx = (valueIdx + 1) % FALLDET_MONITOR_NUM_SAMPLES;


  valuesCntr ++;
  if (valuesCntr >= FALLDET_CALC_NUM_SAMPLES)
  {
    if (status == FALLDET_NORMAL)
    {
      // make an average to level out peaks
      computeAverage(values, valueIdx, FALLDET_MONITOR_NUM_SAMPLES, valuesCntr, &avg);
      Serial.print("Updating average... "); Serial.print(avg.ax); Serial.print(", "); Serial.print(avg.ay); Serial.print(", "); Serial.print(avg.az); Serial.println();
    }
    
    valuesCntr = 0;
  }


  // compute the norm and make an average of the last five values to level out peaks
  float aNorm = sqrt((ax*ax) + (ay*ay) + (az*az));
  avgNorm = averageNorm(aNorm);


  if (currMillis - startMillis < 1000)
  {
    // do nothing for the first second
    return false;
  }
  
  if ((status == FALLDET_NORMAL) || (status == FALLDET_WAIT_IMPACT))
  {
    // detect free fall
    if (avgNorm < FALLDET_FREEFALL_THRESHOLD)
    {
      // free fall detected: wait for impact
      Serial.print("Free fall detected "); Serial.print(avgNorm); Serial.print(" / "); Serial.print(FALLDET_FREEFALL_THRESHOLD); Serial.println();
      
      status = FALLDET_WAIT_IMPACT; 
      impactMillis = currMillis; 
    }


    // detect impact
    if (avgNorm > FALLDET_IMPACT_THRESHOLD)
    {
      // impact detected
      Serial.print("Impact detected "); Serial.print(avgNorm); Serial.print(" / "); Serial.print(FALLDET_IMPACT_THRESHOLD); Serial.println();


      status = FALLDET_WAIT_REST;  
      impactMillis = currMillis; 
    }


    if (status == FALLDET_WAIT_IMPACT)
    {
      // timeout to wait for impact after free fall. If timeout expires, return to normal condition
      if (currMillis - impactMillis > FALLDET_IMPACT_DELAY_MS)
      {
        status = FALLDET_NORMAL;
        valuesCntr = 0;
      }
    }
  }
  else if (status == FALLDET_WAIT_REST)
  {
    // wait 5 seconds to reach a "stable" position
    delta = currMillis - impactMillis;
    if (delta > FALLDET_REST_DELAY_MS)
    {
      // start collecting post-fall data
      Serial.println("Collecting post data..."); 
      status = FALLDET_COLLECT;
      postValuesCntr = 0;
    }
  }
  else if (status == FALLDET_COLLECT)
  {
    // collect post-impact data
    SENSOR_DATA* psd = &postValues[postValuesCntr];
    psd->ax = sd->ax;
    psd->ay = sd->ay;
    psd->az = sd->az;
    psd->gx = sd->gx;
    psd->gy = sd->gy;
    psd->gz = sd->gz;
    
    postValuesCntr ++;
    if (postValuesCntr >= FALLDET_POST_NUM_SAMPLES)
    {
      status = FALLDET_CHECK;
    }
  }
  else if (status == FALLDET_CHECK)
  {
    // check post-impact data
    SENSOR_DATA avgPostNorm;
    computeAverage(postValues, 0, FALLDET_POST_NUM_SAMPLES, FALLDET_POST_NUM_SAMPLES, &avgPostNorm);


    Serial.print("Checking position "); Serial.print(fabs(avgPostNorm.ax)); Serial.print("\t"); Serial.print(fabs(avgPostNorm.ay)); Serial.print("\t"); Serial.print(fabs(avgPostNorm.az)); Serial.println();
    Serial.print("Previous position "); Serial.print(fabs(avg.ax)); Serial.print("\t"); Serial.print(fabs(avg.ay)); Serial.print("\t"); Serial.print(fabs(avg.az)); Serial.println();
    Serial.print("Deltas            "); Serial.print(fabs(avgPostNorm.ax-avg.ax)); Serial.print("\t"); Serial.print(fabs(avgPostNorm.ay-avg.ay)); Serial.print("\t"); Serial.print(fabs(avgPostNorm.az-avg.az)); Serial.println();


    if ( (fabs(avgPostNorm.ax-avg.ax) > FALLDET_DIFF_TRESHOLD) || (fabs(avgPostNorm.ay-avg.ay) > FALLDET_DIFF_TRESHOLD) || (fabs(avgPostNorm.az-avg.az) > FALLDET_DIFF_TRESHOLD) )
    {
      // fall detected
      Serial.println("FALL DETECTED!!!");
      
      status = FALLDET_FALL_DETECTED;
      return true;
    }
    else
    {
      // this is not a fall, return to normal condition
      status = FALLDET_NORMAL;
      startMillis = currMillis;
      valuesCntr = 0;
    }
  }


  return false;
}