Capturing the audio data from the microphone
The first thing that I need to verify is that I can successfully capture audio data from the microphone to use for keyword spotting using the impulse that was generated previously.
The Wio Terminal uses an analog microphone and the Nano 33 BLE Sense uses a digital PDM microphone. So, I'll need to digitize the analog signal with an ADC and be able to sample it at the required frequency (Edge Impulse suggests 16KHz).
The first step is to verify basic functionality. This is straightforward as you can use an analogRead(WIO_MIC) to get an ADC conversion of the microphone output. Seeed provides sample code to continuously read and plot the microphone signal on the LCD display.
Here's the code:
Wio_Terminal_Microphone_Chart.ino
#include"seeed_line_chart.h" //include the library
#include <math.h>
TFT_eSPI tft;
#define max_size 50 //maximum size of data
doubles data; //Initilising a doubles type to store data
TFT_eSprite spr = TFT_eSprite(&tft); // Sprite
void setup() {
pinMode(WIO_MIC, INPUT);
tft.begin();
tft.setRotation(3);
spr.createSprite(TFT_HEIGHT,TFT_WIDTH);
}
void loop() {
spr.fillSprite(TFT_DARKGREY);
int val = analogRead(WIO_MIC);
if (data.size() == max_size) {
data.pop();//this is used to remove the first read variable
}
data.push(val); //read variables and store in data
//Settings for the line graph title
auto header = text(0, 0)
.value("Microphone Reading")
.align(center)
.color(TFT_WHITE)
.valign(vcenter)
.width(tft.width())
.thickness(2);
header.height(header.font_height() * 2);
header.draw(); //Header height is the twice the height of the font
//Settings for the line graph
auto content = line_chart(20, header.height()); //(x,y) where the line graph begins
content
.height(tft.height() - header.height() * 1.5) //actual height of the line chart
.width(tft.width() - content.x() * 2) //actual width of the line chart
.based_on(0.0) //Starting point of y-axis, must be a float
.show_circle(true) //drawing a cirle at each point, default is on.
.y_role_color(TFT_WHITE)
.x_role_color(TFT_WHITE)
.value(data) //passing through the data to line graph
.color(TFT_RED) //Setting the color for the line
.draw();
spr.pushSprite(0, 0);
delay(50);
}
And a quick video showing the microphone digitized output plot on the LCD:
Configuring DMA on the ADC
The problem with sampling analog signals with an ADC is being able capture the data at a sufficiently high rate. The ADCs on the SAMD parts can be configured to sample at the required rate but the issue is being able to efficiently move that data to memory where it can be used by the impulse for inferencing. I got some feedback on the Edge Impulse forum from Jan Jongboom and Shawn Hymel that while interrupts using an ISR might work that the preferred method is to use double buffered DMA. I had hoped I could use the Adafruit_ZeroDMA library that was designed initially for the M0 SAMD21 but the ADC section hasn't been fully ported to the M4 SAMD51 that I'm using. Also, Adafruit uses a lower pin count SAMD51 so I may have had to modify the library anyway. So, I needed to figure out how to configure DMA on the SAMD51. Luckily, I came across a thread on the Arduino forum about "Setting up ISR for ADC on Zero" https://forum.arduino.cc/index.php?topic=685347.0 that helped me get started.
Figuring out the correct ADC input to use
The SAMD51 is is a highly configurable part which makes it somewhat difficult to set up if you don't have a library hiding all the gory details from you. The part has 2 ADCs that are shared across the analog signals and each ADC has up to 23 inputs (16 external channels) and some channels can be differential and some inputs have dedicated functions. So, how do you figure out what ADC channel the microphone is on? I looked in variant.h but that only gave me the equivalent pin number to use for the Arduino IDE. Turns out it was easy to find because Seeed did a great job of labeling the part body on the schematic.
If you look at this excerpt from the schematic you can determine that the MIC_OUTPUT is on Pad PC30 which connects to ADC1 channel 12 (AIN1.12).
To verify that I checked the SAM_D5xE5x_Family_Data_Sheet and it correlates.
Armed with that information I modified the code I found on the Arduino forum and created a program to test using DMA to read ADC1 into two data buffers. Each buffer holds 256 values and DMA alternates filling each buffer.
Here is the code:
Wio_Terminal_Microphone_DMA_Test.ino
// Use SAMD51's DMAC to read ADC1 on the Microphone input and alternately store results in two memory arrays
// Use DMAC channel0
#define NO_RESULTS 256
#define LED_PIN D0 // for debug
volatile boolean results0Ready = false;
volatile boolean results1Ready = false;
uint16_t adcResults0[NO_RESULTS]; // ADC results array 0
uint16_t adcResults1[NO_RESULTS]; // ADC results array 1
typedef struct // DMAC descriptor structure
{
uint16_t btctrl;
uint16_t btcnt;
uint32_t srcaddr;
uint32_t dstaddr;
uint32_t descaddr;
} dmacdescriptor ;
volatile dmacdescriptor wrb[DMAC_CH_NUM] __attribute__ ((aligned (16))); // Write-back DMAC descriptors
dmacdescriptor descriptor_section[DMAC_CH_NUM] __attribute__ ((aligned (16))); // DMAC channel descriptors
dmacdescriptor descriptor __attribute__ ((aligned (16))); // Place holder descriptor
void setup() {
Serial.begin(115200); // Start the native USB port
while(!Serial); // Wait for the console to open
// initialize digital pin LED_BUILTIN as an output.
pinMode(LED_PIN, OUTPUT);
DMAC->BASEADDR.reg = (uint32_t)descriptor_section; // Specify the location of the descriptors
DMAC->WRBADDR.reg = (uint32_t)wrb; // Specify the location of the write back descriptors
DMAC->CTRL.reg = DMAC_CTRL_DMAENABLE | DMAC_CTRL_LVLEN(0xf); // Enable the DMAC peripheral
DMAC->Channel[0].CHCTRLA.reg = DMAC_CHCTRLA_TRIGSRC(ADC1_DMAC_ID_RESRDY) | // Set DMAC to trigger when ADC1 result is ready
DMAC_CHCTRLA_TRIGACT_BURST; // DMAC burst transfer
descriptor.descaddr = (uint32_t)&descriptor_section[1]; // Set up a circular descriptor
descriptor.srcaddr = (uint32_t)&ADC1->RESULT.reg; // Take the result from the ADC1 RESULT register
descriptor.dstaddr = (uint32_t)adcResults0 + sizeof(uint16_t) * NO_RESULTS; // Place it in the adcResults0 array
descriptor.btcnt = NO_RESULTS; // Beat count
descriptor.btctrl = DMAC_BTCTRL_BEATSIZE_HWORD | // Beat size is HWORD (16-bits)
DMAC_BTCTRL_DSTINC | // Increment the destination address
DMAC_BTCTRL_VALID | // Descriptor is valid
DMAC_BTCTRL_BLOCKACT_SUSPEND; // Suspend DMAC channel 0 after block transfer
memcpy(&descriptor_section[0], &descriptor, sizeof(descriptor)); // Copy the descriptor to the descriptor section
descriptor.descaddr = (uint32_t)&descriptor_section[0]; // Set up a circular descriptor
descriptor.srcaddr = (uint32_t)&ADC1->RESULT.reg; // Take the result from the ADC1 RESULT register
descriptor.dstaddr = (uint32_t)adcResults1 + sizeof(uint16_t) * NO_RESULTS; // Place it in the adcResults1 array
descriptor.btcnt = NO_RESULTS; // Beat count
descriptor.btctrl = DMAC_BTCTRL_BEATSIZE_HWORD | // Beat size is HWORD (16-bits)
DMAC_BTCTRL_DSTINC | // Increment the destination address
DMAC_BTCTRL_VALID | // Descriptor is valid
DMAC_BTCTRL_BLOCKACT_SUSPEND; // Suspend DMAC channel 0 after block transfer
memcpy(&descriptor_section[1], &descriptor, sizeof(descriptor)); // Copy the descriptor to the descriptor section
NVIC_SetPriority(DMAC_0_IRQn, 0); // Set the Nested Vector Interrupt Controller (NVIC) priority for TCC1 OVF to 0 (highest)
NVIC_EnableIRQ(DMAC_0_IRQn); // Connect TCC1 to Nested Vector Interrupt Controller (NVIC)
DMAC->Channel[0].CHINTENSET.reg = DMAC_CHINTENSET_SUSP; // Activate the suspend (SUSP) interrupt on DMAC channel 0
ADC1->INPUTCTRL.bit.MUXPOS = ADC_INPUTCTRL_MUXPOS_AIN12_Val; // Set the analog input to AIN12
while(ADC1->SYNCBUSY.bit.INPUTCTRL); // Wait for synchronization
ADC1->SAMPCTRL.bit.SAMPLEN = 0x0a; // Set max Sampling Time Length to half divided ADC clock pulse (2.66us) if set to 0x0
while(ADC1->SYNCBUSY.bit.SAMPCTRL); // Wait for synchronization
ADC1->CTRLA.reg = ADC_CTRLA_PRESCALER_DIV256; // Divide Clock ADC GCLK by 256 (48MHz/256 = 187.5kHz)
ADC1->CTRLB.reg = ADC_CTRLB_RESSEL_16BIT | // Set ADC resolution to 12 bits
ADC_CTRLB_FREERUN; // Set ADC to free run mode
while(ADC1->SYNCBUSY.bit.CTRLB); // Wait for synchronization
ADC1->CTRLA.bit.ENABLE = 1; // Enable the ADC
while(ADC1->SYNCBUSY.bit.ENABLE); // Wait for synchronization
ADC1->SWTRIG.bit.START = 1; // Initiate a software trigger to start an ADC conversion
while(ADC1->SYNCBUSY.bit.SWTRIG); // Wait for synchronization
DMAC->Channel[0].CHCTRLA.bit.ENABLE = 1; // Enable DMAC ADC on channel 1
}
void loop()
{
if (results0Ready) // Display the results in results0 array
{
Serial.println(F("Results0"));
for (uint32_t i = 0; i < NO_RESULTS; i++)
{
Serial.print(i);
Serial.print(F(": "));
Serial.println(adcResults0[i]);
}
Serial.println();
results0Ready = false; // Clear the results0 ready flag
digitalWrite(LED_PIN, HIGH); // turn the LED on
}
if (results1Ready) // Display the results in results1 array
{
Serial.println(F("Results1"));
for (uint32_t i = 0; i < NO_RESULTS; i++)
{
Serial.print(i);
Serial.print(F(": "));
Serial.println(adcResults0[i]);
}
Serial.println();
results1Ready = false; // Clear the results1 ready flag
digitalWrite(LED_PIN, LOW); // turn the LED off
}
}
void DMAC_0_Handler() // Interrupt handler for DMAC channel 0
{
static uint8_t count = 0; // Initialise the count
if (DMAC->Channel[0].CHINTFLAG.bit.SUSP) // Check if DMAC channel 0 has been suspended (SUSP)
{
DMAC->Channel[0].CHCTRLB.reg = DMAC_CHCTRLB_CMD_RESUME; // Restart the DMAC on channel 0
DMAC->Channel[0].CHINTFLAG.bit.SUSP = 1; // Clear the suspend (SUSP)interrupt flag
if (count) // Test if the count is 1
{
results1Ready = true; // Set the results 1 ready flag
}
else
{
results0Ready = true; // Set the results 0 ready flag
}
count = (count + 1) % 2; // Toggle the count between 0 and 1
}
}
Here is a short video of the test:
I am writing the buffer data to the Serial Plotter tool of the Arduino IDE. Because it is autoscaling the noise looks larger in the absence of a valid input. It does seem quite noisy though.
Looks like I have a place to start. Next I'll try to incorporate this into the program that is using the impulse library for inferencing. It probably won't work initially. I'm probably going to have to play with the configuration parameters on the ADC to get the sampling rate correct and decide what resolution to use (8, 10, 12, 16 bit). I'm currently using 16 bit but may need to change that if I can't get a fast enough sampling rate. I'm also concerned about whether it is okay to use DMAC channel 0. If I get the impulse working I still have one more task which is to add the impulse into my multi-sensor application program. I'm concerned at that point that I could run into DMAC conflicts. Might be safer to select a higher DMAC channel?
Well, I need to take a break for now. Looking less likely that I'll have this working by Monday's deadline 
.
Links to related posts
Wio Terminal Sensor Fusion - Introduction
Wio Terminal Sensor Fusion - Sensor Integration
Wio Terminal Sensor Fusion - Remote Data Display and Control using Blynk
Wio Terminal Sensor Fusion - Remote Data Display and Control using Blynk continued
Top Comments