Introduction
I've been dying to do something with this toy for ages, as it's been lying in a box, gathering dust, for a rather long time now.
This toy was originally purchased as a funny/silly (you pick) Halloween toy, which played a somewhat annoying tune and the turntable turned to mimic a disc scratch when you pressed a button... here's what I mean:
This months acoustics challenge was just the opportunity to bring a new lease of life to this skeleton disc-jockey. It also allowed me to learn and experiment with Fast Fourier Transforms, which has been on my todo list for some time now.
And here is the makeover:
Let me now explain how this was made.
I'll start with the hardware and then move onto the software.
Building Boogie Bones - The Hardware
Let me start with a high level tear down of the original boogie bones.
| {gallery:autoplay=false} Original Boogie Bones Tear Down |
|---|
|
|
|
|
|
|
|
|
Basically, it uses a transistor circuit to drive a DC motor bidirectionally through some unknown IC (hidden from view) with possibly an audio codec to play music through the 8 ohm 0.5 watt speaker. The circuit is powered by 3 x AA batteries or circa. 4.5V. And that is it.
So, not much to it and actually there was not much I wanted to reuse other than the dc motor, which turned the turntable.
What I decided to do was to change this toy into a visual light show, which responded to audio. The turntable movement would also be included based on specified audio spectrum conditions.
| {gallery:autoplay=false} Hardware Modules |
|---|
|
|
|
|
|
|
|
|
The parts/modules I used were as follows:
- 3 x NeoPixels (used Adafruit's NeoPixel Mini Button PCB's)
- 1 x microphone with amp (used the Adafruit Electret Microphone breakout which includes the Maxim MAX4466 amplifier and manual adjustable gain)
- 1 x microcontroller (used the Arduino MKR 1000)
- 1 x motor driver (used the Arduino MKR Motor Carrier Shield, which includes a TI DRV8871 motor driver for DC motor control)
The Arduino MKR Motor Carrier Shield was selected purely for convenience (as I had one) and it allowed me to insert the Arduino MKR into the shield. This shield also provided a nice battery connector with an on-off switch and had screw terminals for the motor connection. This shield also provided some smarts such as battery voltage monitoring as this shield included another SoC, i.e. an ATSAM11D, which controlled some of the motor and servo IO's.
However, I discovered a potential downside to using the Motor Carrier Shield. The DC motor voltage is not a regulated voltage but is connected to the raw battery voltage.
The battery I was using was one that came with the Arduino Engineering kit with is 11.1V. As the motor would only be triggered for a few milliseconds, I left it as is. This actually created an interesting side effect, namely, the motor whining noise. This noise, actually proved beneficial as it created a very nice audio effect when the turntable turned, which mimicked a disc scratching noise.
We then just use 2 GPIO's. We use a digital pin as an output for the NeoPixels and an analog input pin for the audio signal.
And that is all we need from a hardware perspective. Let's now look at the software, which is where all the clever stuff happens.
Bashing code for Boogie Bones - The Software
This project all hinges off software that interprets the spectrum analysis of a microphone audio signal.
As mentioned in the hardware section, I'm using an Adafruit breakout board, which includes a 20-20KHz electret microphone and a Maxim MAX4466 op-amp for audio amplification to capture an audio signal. According to the product description, this breakout is best used for projects such as voice changers, audio recording/sampling, and audio-reactive projects that use FFT. The manual gain control allows amplification adjustment from 25x up to 125x.
I decided to test how this amplitude adjustment works. Here is the result using the handy Arduino IDE's Serial Plotter to plot out the analogRead() input value from the microphone. I used a “white noise” YouTube video at a set volume to generate my noise signal and then manually adjusted the gain, back and forth. You can clearly see the difference amplitude gain makes:
The next thing I needed was to find and test one of the commonly available Fast Fourier Transform (fft) libraries. This will be used to tell me what audio frequencies are most prevalent in the audio signal for any given sample.
So, you may be asking how does FFT work. I'm not claiming this to be a definitive description, but all I know is that a Fast Fourier Transform is basically a clever algorithm (there are multiple options around) that can quickly compute a Fourier Transform, which is a special mathematical function that decomposes a time series waveform into discrete frequencies bands that make up that waveform. These frequencies are made up of real and imaginary values that form a complex number as the result. In other words, the absolute, or real, value of the Fourier transform represents the frequency value present in the original function and its complex argument, or imaginary value, represents the phase offset of the basic sinusoidal in that frequency. If you want to know more, there is a fair bit of info online, such as this web page, or alternatively speak to an Electronics Engineer who was able to grasp what Digital Signal Processing is all about.
For this project I settled on the arduinoFFT library as it just seemed easier to grasp and it also happened to be the one mentioned in that web page I linked above.
However, for some reason, most of the examples provided with this ArduinoFFT library use a simulated input, which was not very beneficial for my purposes. Some of the examples also make reference to cycles/second, which made no sense to me until I watched this YouTube video. It is well worth watching to see how a Fourier Transform works visually. The ArduinoFFT library example that used an actual analog input value was FFT_03.ino, which is what I decided to work with.
There are two key parameters used in this example, namely “samples” and “samplingFrequency”. The comments in the code are fairly self explanatory.
const uint16_t samples = 64; //This value MUST ALWAYS be a power of 2 const double samplingFrequency = 100; //Hz, must be less than 10000 due to ADC
As can be seen later in the example, the sampling frequency is used to determine the sampling period:
sampling_period_us = round(1000000*(1.0/samplingFrequency));
The number of samples then defines the number of discrete segments or bin values within the measured frequency range. So using the samplingFrequency of 100 and we divide this by our 64 bins or samples to then get 1.5625Hz as our bin size.
To perform a Fourier Transform we must first populate the input array vReal[samples] by reading our defined audio analog input pin at the specified sampling period.
/*SAMPLING*/
microseconds = micros();
for(int i=0; i<samples; i++)
{
vReal[i] = analogRead(CHANNEL);
vImag[i] = 0;
while(micros() - microseconds < sampling_period_us){
//empty loop
}
microseconds += sampling_period_us;
}
We can now apply our Fourier Transform on this array, using these functions:
FFT.Windowing(vReal, samples, FFT_WIN_TYP_HAMMING, FFT_FORWARD); /* Weigh data */ FFT.Compute(vReal, vImag, samples, FFT_FORWARD); /* Compute FFT */ FFT.ComplexToMagnitude(vReal, vImag, samples); /* Compute magnitudes */ double x = FFT.MajorPeak(vReal, samples, samplingFrequency); /* returns frequency that is the most dominant. */
You may have noticed the use of some defined parameters in these functions, such as FFT_WIN_TYP_HAMMING and FFT_FORWARD. So what do these mean?
To learn more I started by consulting the library readme. Here it tells that FFT_WIN_TYP_HAMMING is an option that performs windowing function on the values array. The possible windowing options are the following:
* FFT_WIN_TYP_RECTANGLE
* FFT_WIN_TYP_HAMMING
* FFT_WIN_TYP_HANN
* FFT_WIN_TYP_TRIANGLE
* FFT_WIN_TYP_NUTTALL
* FFT_WIN_TYP_BLACKMAN
* FFT_WIN_TYP_BLACKMAN_NUTTALL
* FFT_WIN_TYP_BLACKMAN_HARRIS
* FFT_WIN_TYP_FLT_TOP
* FFT_WIN_TYP_WELCH
If we look at the library code for the windowing function we see that these type definitions represent a weighting factor which is applied to the sample values:
for (uint16_t i = 0; i < (this->_samples >> 1); i++) {
double indexMinusOne = double(i);
double ratio = (indexMinusOne / samplesMinusOne);
double weighingFactor = 1.0;
// Compute and record weighting factor
switch (windowType) {
case FFT_WIN_TYP_RECTANGLE: // rectangle (box car)
weighingFactor = 1.0;
break;
case FFT_WIN_TYP_HAMMING: // hamming
weighingFactor = 0.54 - (0.46 * cos(twoPi * ratio));
break;
case FFT_WIN_TYP_HANN: // hann
weighingFactor = 0.54 * (1.0 - cos(twoPi * ratio));
break;
case FFT_WIN_TYP_TRIANGLE: // triangle (Bartlett)
weighingFactor = 1.0 - ((2.0 * abs(indexMinusOne - (samplesMinusOne / 2.0))) / samplesMinusOne);
break;
case FFT_WIN_TYP_NUTTALL: // nuttall
weighingFactor = 0.355768 - (0.487396 * (cos(twoPi * ratio))) + (0.144232 * (cos(fourPi * ratio))) - (0.012604 * (cos(sixPi * ratio)));
break;
case FFT_WIN_TYP_BLACKMAN: // blackman
weighingFactor = 0.42323 - (0.49755 * (cos(twoPi * ratio))) + (0.07922 * (cos(fourPi * ratio)));
break;
case FFT_WIN_TYP_BLACKMAN_NUTTALL: // blackman nuttall
weighingFactor = 0.3635819 - (0.4891775 * (cos(twoPi * ratio))) + (0.1365995 * (cos(fourPi * ratio))) - (0.0106411 * (cos(sixPi * ratio)));
break;
case FFT_WIN_TYP_BLACKMAN_HARRIS: // blackman harris
weighingFactor = 0.35875 - (0.48829 * (cos(twoPi * ratio))) + (0.14128 * (cos(fourPi * ratio))) - (0.01168 * (cos(sixPi * ratio)));
break;
case FFT_WIN_TYP_FLT_TOP: // flat top
weighingFactor = 0.2810639 - (0.5208972 * cos(twoPi * ratio)) + (0.1980399 * cos(fourPi * ratio));
break;
case FFT_WIN_TYP_WELCH: // welch
weighingFactor = 1.0 - sq((indexMinusOne - samplesMinusOne / 2.0) / (samplesMinusOne / 2.0));
break;
}
if (dir == FFT_FORWARD) {
this->_vReal[i] *= weighingFactor;
this->_vReal[this->_samples - (i + 1)] *= weighingFactor;
}
else {
this->_vReal[i] /= weighingFactor;
this->_vReal[this->_samples - (i + 1)] /= weighingFactor;
}
}
As to which one is best for which application. Not sure, to be honest. In my case this was not critical for the application. To learn more, I simply read the Wikipedia page on the Windowing Function. You will see in the first chart below how this impacts the raw values when we chart out the weighted values.
So, let's run this example using the same white noise as our audio source and see what happens. First we get our raw data, which is converted into “weighted data”:
The FFT library then converts this into Real and Imaginary data:
And finally it works out the prominent frequencies with the highest prominent frequency being 28.38Hz.
As mentioned earlier, sampling at 100Hz is not very informative for dissecting audio signals. Let's now increase our data sampling frequency to 9600 Hz. This is what we get - i.e. the shape of the raw and computed values are pretty similar as before but we get a much larger frequency spread and a much larger a bin size:
| {gallery:autoplay=false} 9600 Hz Sample Frequency |
|---|
|
|
|
|
|
|
If we now wanted to get a much finer frequency band assessment all we have to do is increase the number of samples taken. However, this is not a free lunch and comes at a cost. So, for example, if we used 1024 samples it would now take over a minute just to compute these results versus if we use 64 samples we are talking less than 100 milliseconds to compute. That's quite some difference. For my purposes, 64 or even 32 samples will work just fine.
But there was one thing about the frequency magnitude charts that had me puzzled. Why was the biggest frequency always the first bin (0Hz). Well, after a bit of head scratching and searching, I spotted that in the Adafruit Zero FFT Arduino library mic_test.ino example, the code included the following before it ran the Fourier Transfer function:
//remove DC offset and gain up to 16 bits avg = avg/DATA_SIZE; for(int i=0; i<DATA_SIZE; i++) data[i] = (data[i] - avg) * 64;
That line of code essentially rebased the data by removing the offset so that the average would be zero. The code also included a software based amplification where the values were multiplied by 64 (something I did not need). So, if I applied the rebasing method we solve the problem. The first bin (0Hz) is no longer always the major frequency.
| {gallery:autoplay=false} White Noise Analysis with offset removed |
|---|
|
|
|
|
|
|
Anyway, that's the testing out the way using white noise but the data shown so far was only for a single sample. I needed a continuous sampling method and I also needed to figure how to link the continuous audio spectrum analysis with the NeoPixel LED's.
So, I started by removing all the code from the fft03.ino example that was not necessary and then formatted the output so that it becomes meaningful for spectrum analysis, using the Arduino Serial Plotter. The Serial Plotter is a great tool as it provides a live visual indication of what is happening with the frequency values over time.
/*
This example is nased pm tje original example fft03.ino which is part of
the Arduino FFT libray that will compute FFT for a signal sampled through the ADC.
Libary Copyright (C) 2018 Enrique Condés and Ragnar Ranøyen Homb
This example written by BigG (Gerrikoio) (C) February 2020
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include "arduinoFFT.h"
arduinoFFT FFT = arduinoFFT(); /* Create FFT object */
/*
These values can be changed in order to evaluate the functions
*/
#define CHANNEL A2
const uint16_t samples = 64; //This value MUST ALWAYS be a power of 2
const double samplingFrequency = 9600; //Hz, must be less than 10000 due to ADC
unsigned int sampling_period_us;
unsigned long microseconds;
/*
These are the input and output vectors
Input vectors receive computed results from FFT
*/
double vReal[samples];
double vImag[samples];
double vAve = 0;
void setup()
{
while (!Serial) {;;}
Serial.begin(115200);
delay(10);
// We'll scrap the "ready" to make it serial port friendly
//Serial.println("Ready");
// Get the analog pin "warmed up" to get rid of initial spurious readings
for (uint8_t i = 0; i < 20; i++) {
analogRead(CHANNEL);
delay(1);
}
sampling_period_us = round(1000000*(1.0/samplingFrequency));
}
void loop()
{
/*SAMPLING*/
microseconds = micros();
for(int i=0; i<samples; i++)
{
vReal[i] = analogRead(CHANNEL);
vAve += vReal[i];
vImag[i] = 0;
while(micros() - microseconds < sampling_period_us){
//empty loop
}
microseconds += sampling_period_us;
}
// We now rebase our raw values
vAve /= samples;
for(int i=0; i<samples; i++) vReal[i] -= vAve;
/* we're only interested in the frequency values and want to repeat this frequenty */
/* we'll the rest as is */
FFT.Windowing(vReal, samples, FFT_WIN_TYP_HAMMING, FFT_FORWARD); /* Weight the data */
FFT.Compute(vReal, vImag, samples, FFT_FORWARD); /* Compute FFT */
FFT.ComplexToMagnitude(vReal, vImag, samples); /* Compute magnitudes */
Serial.println("Computed magnitudes:");
PrintVector(vReal, (samples >> 1));
delay(50); /* Repeat after short delay */
}
void PrintVector(double *vData, uint16_t bufferSize)
{
for (uint16_t i = 0; i < bufferSize; i++)
{
/* We'll skip the abscissa value printout as this does not change for time series analysis */
/* we'll print out the sample values as comma delimited */
Serial.print(vData[i], 2);
if (i < (bufferSize-1)) Serial.print(",");
}
// Now print a CRLF
Serial.println();
}
As seen in the video, the output is rather messy and is providing way more data than is necessary to visualise audio through the NeoPixels. By reducing the sample count we start to get more helpful data. However, as the following graphics show, reducing the sample count too much has its limitations too.
The frequency range for each bin is too wide to properly capture the unique elements that make up a music audio, namely bass, mids and treble. Using this online reference as a guide we see that the heavy bass is typically defined as frequencies up to 350Hz. It then defines “low mids”, which are frequencies between 350Hz and 2kHz, as being the fundamental frequencies of the most instruments. Then is splits the “high mids” into two categories, with 2kHz to 4kHz being important to make vocals and instruments to be more noticeable and then 4kHz and above offers clarity as we move into the range for treble.
This made it tricky to capture the full frequency range, as the ADC conversion speed has limitations which will then clip our max frequencies measured. Also, if our bands are too broad, as shown in above graphic, we will fail to pick out the fundamental music frequencies within the low mid range.
As such, for my project, I settled on using a sample size of 32 (giving me 16 bins) and then extracted out 4 frequencies to use for my lights. I then used three of these to define my RGB and then the most prominent one to define brightness. This is illustrated as follows:
Now that I had my frequencies bands sorted I needed to work out how to translate these bands into something meaningful for lighting effect. It's time to take a look at a NeoPixel library.
In this case, I chose to use the Adafruit NeoPixel library as I knew it worked well with the NeoPixels I was using. This library provides a number of examples and one of those, RGBWstrandtest.ino, seemed very suitable to what I needed. Within this example was a function called "colorWipe" that was near perfect to deliver the effect I needed.
// Fill strip pixels one after another with a color. Strip is NOT cleared
// first; anything there will be covered pixel by pixel. Pass in color
// (as a single 'packed' 32-bit value, which you can get by calling
// strip.Color(red, green, blue) as shown in the loop() function above),
// and a delay time (in milliseconds) between pixels.
void colorWipe(uint32_t color, int wait) {
for(int i=0; i<strip.numPixels(); i++) { // For each pixel in strip...
strip.setPixelColor(i, color); // Set pixel's color (in RAM)
strip.show(); // Update strip to match
delay(wait); // Pause for a moment
}
}
The NeoPixels side proved painless. To link frequency to RGB values I simply clipped the selected frequency values to a maximum value of 255 using the Arduino constrain function.
It was now a case of patching it all together with the MKRMotorCarrier library to deliver the turntable movement. This too proved painless as all I did was extract out code from the Motor_Test.ino example.
And here is the final code I came up with for this project:
#include <arduinoFFT.h>
#include <Adafruit_NeoPixel.h>
#include <MKRMotorCarrier.h>
// Remove as a comment for debug
// #define DEBUG 1 #define DEBUG
// Pinout Defs
#define CHANNEL A2
#define LED_PIN 0
#define INTERRUPT_PIN 6
// How many NeoPixels are attached to the Arduino?
#define LED_COUNT 3
const uint16_t SAMPLES = 32; //This value MUST ALWAYS be a power of 2
const double SAMPLEFREQ = 9600; //Hz, must be less than 10000 due to ADC
const double NOISETHRESHOLD = 3.875;
//Variable to change the motor speed and direction
const uint16_t M_CHECK = 5000;
const uint8_t M_DUTY = 45;
const uint8_t M_TIMEON = 50;
const uint8_t M_TIMEOFF = 250;
//Variable to store the battery voltage
static int batteryVoltage;
unsigned int sampling_period_us = round(1000000*(1.0/SAMPLEFREQ));
unsigned long microseconds = 0;
unsigned long t_motor = 0;
unsigned long t_check = 0;
/*
These are the input and output vectors
Input vectors receive computed results from FFT
*/
double vReal[SAMPLES];
double vImag[SAMPLES];
double vAve = 0;
uint16_t sCntr = 0;
uint8_t MotorState = 0;
boolean MotorAvailable = false;
arduinoFFT FFT = arduinoFFT(); /* Create FFT object */
// Declare our NeoPixel strip object:
Adafruit_NeoPixel strip(LED_COUNT, LED_PIN, NEO_GRBW + NEO_KHZ800);
void setup() {
// put your setup code here, to run once:
#ifdef DEBUG
while (!Serial) {;;}
Serial.begin(115200);
#endif
delay(10);
strip.begin(); // INITIALIZE NeoPixel strip object (REQUIRED)
strip.show(); // Turn OFF all pixels ASAP
strip.setBrightness(0); // Set BRIGHTNESS to about 1/5 (max = 255)
//Establishing the communication with the motor shield
if (controller.begin()) {
controller.reboot();
M1.setDuty(0);
M2.setDuty(0);
M3.setDuty(0);
M4.setDuty(0);
MotorAvailable = true;
t_check = millis();
}
microseconds = micros();
}
void loop() {
// put your main code here, to run repeatedly:
if ((micros() - microseconds) > sampling_period_us) {
vReal[sCntr] = analogRead(CHANNEL);
vAve += vReal[sCntr];
vImag[sCntr] = 0;
sCntr++;
microseconds = micros();
}
if (MotorAvailable) {
if (t_motor) {
if ((MotorState == 1) && (millis() - t_motor)> M_TIMEON) {
t_motor = millis();
MotorState = 2;
M1.setDuty(0);
}
else if ((MotorState == 2) && (millis() - t_motor)> M_TIMEOFF) {
if (battery.getConverted() >= 9) {
t_motor= millis();
MotorState = 3;
M1.setDuty(-M_DUTY);
}
else {
t_motor = 0;
MotorState = 0;
M1.setDuty(0);
}
}
if ((MotorState == 3) && (millis() - t_motor)> M_TIMEON) {
t_motor = 0;
MotorState = 0;
M1.setDuty(0);
}
}
}
if (sCntr >= SAMPLES) {
vAve /= SAMPLES;
for (uint16_t i = 0; i < SAMPLES; i++) vReal[i] = (vReal[i] - vAve);
/* Print the frequency results of the sampling according to time */
FFT.Windowing(vReal, SAMPLES, FFT_WIN_TYP_BLACKMAN_HARRIS, FFT_FORWARD); /* Weigh data */
FFT.Compute(vReal, vImag, SAMPLES, FFT_FORWARD); /* Compute FFT */
FFT.ComplexToMagnitude(vReal, vImag, SAMPLES); /* Compute magnitudes */
sCntr = 0;
vAve = 0;
PrintVector(vReal, (SAMPLES >> 1));
if (MotorAvailable && (millis()- t_check) > M_CHECK) {
controller.ping();
t_check = millis();
}
}
}
void PrintVector(double *vData, uint16_t bufferSize)
{
uint8_t NewSpecData[4] = {'\0'};
double vDat = 0.0;
uint8_t x = 0;
// We ignore the first two low frequencies as picks up too much noise etc.
//#ifdef DEBUG
//Serial.println(vData[0])
//#endif
for (uint16_t i = 0; i < bufferSize; i+=4) {
if (!i) {
vData[i+2] -= NOISETHRESHOLD;
vData[i+3] -= NOISETHRESHOLD;
vDat = (vData[i+2]+vData[i+3])/2.0;
}
else {
vData[i] -= NOISETHRESHOLD;
vData[i+1] -= NOISETHRESHOLD;
vDat = (vData[i]+vData[i+1])/2.0;
}
NewSpecData[x] = constrain((vDat+0.5), 0, 255);
x++;
}
// Now display NeoPixels in a unique colour
colorWipe(strip.Color(NewSpecData[0], NewSpecData[2], NewSpecData[3]), NewSpecData[1]);
#ifdef DEBUG
Serial.println(NewSpecData[1]);
#endif
if (MotorAvailable) {
if (NewSpecData[1] > 180) {
#ifdef DEBUG
//Serial.println(MotorState);
#endif
if (!MotorState && !t_motor) {
if (battery.getConverted() >= 9) {
MotorState = 1;
t_motor= millis();
M1.setDuty(M_DUTY);
}
}
}
}
}
// Fill strip pixels one after another with a color. Strip is NOT cleared
// first; anything there will be covered pixel by pixel. Pass in color
// (as a single 'packed' 32-bit value, which you can get by calling
// strip.Color(red, green, blue) as shown in the loop() function above),
// and a delay time (in milliseconds) between pixels.
void colorWipe(uint32_t color, uint8_t bright) {
strip.setBrightness(bright); // Set BRIGHTNESS to about 1/5 (max = 255)
for(int i=0; i<strip.numPixels(); i++) { // For each pixel in strip...
strip.setPixelColor(i, color); // Set pixel's color (in RAM)
strip.show(); // Update strip to match
}
}
Top Comments