Post
Ask Question

Detect Noise Level Audio Decibels In MXChip Azure IoT DevKit

Introduction

 
Playing with Azure IoT DevKit MXChip is always fun. The device has many capabilities. For the past few days, I have been examining some of the capabilities of this device, like Atmospheric pressure, Temperature, Humidity through its sensors. In this article, I will show you how you can calculate the noise level using the microphone of the AZ3166 IoT Device. Now, let’s start implementing the same. I hope you will like it. You can always read this article on my blog here.
 
Background
 
In our last article, we have already seen how to read the temperature, humidity, atmospheric pressure from the MXChip AZ3166 sensors and send those to our Azure IoT Hub. Here in this article, let’s complete the following tasks.
  1. Find the Noise level using the AudioClassV2 class
  2. Send the Values to our IoT Hub
Source Code
 
Please feel free to play around with this repository.
 
Using the Code
 
Once you have your own work space, we can open the solution in VSCode and start coding.
 
main.ino
 
This is our solution's starting point. Every project must have its own sketch file. Usually, this file contains at least the functions loop() and setup().
 
Before we get started, let’s include the header files we are going to use.
  1. //Header files - Start//  
  2. #include "AZ3166WiFi.h"  
  3. #include "AzureIotHub.h"  
  4. #include "DevKitMQTTClient.h"  
  5. #include "config.h"  
  6. #include "utility.h"  
  7. #include "SystemTickCounter.h"  
  8. #include "RingBuffer.h"  
  9. #include "parson.h"  
  10. #include "EEPROMInterface.h"  
  11. #include "http_client.h"  
  12. #include <Arduino.h>  
  13. #include <stdio.h>  
  14. #include <math.h>  
  15. #include "OledDisplay.h"  
  16. #include "AudioClassV2.h"  
  17. #include "stm32412g_discovery_audio.h"  
  18. #include "RGB_LED.h"  
  19. #include <stdint.h>  
  20.   
  21. #define MFCC_WRAPPER_DEFINED  
  22. #include "featurizer.h"  
  23. //Header files - End//  
  24. //**********************//  
Now, we can declare our constants and variables.
  1. //Constants and variables- Start//  
  2. enum AppState  
  3. {  
  4.    APPSTATE_Init,  
  5.    APPSTATE_Error,  
  6.    APPSTATE_Recording  
  7. };  
  8.    
  9. static AppState appstate;  
  10. // These numbers need to match the compiled ELL models.  
  11. const int SAMPLE_RATE = 16000;  
  12. const int SAMPLE_BIT_DEPTH = 16;  
  13. const int FEATURIZER_INPUT_SIZE = 512;  
  14. const int FRAME_RATE = 33; // assumes a "shift" of 512 and 512/16000 = 0.032ms per frame.  
  15. const int FEATURIZER_OUTPUT_SIZE = 80;  
  16. const int CLASSIFIER_OUTPUT_SIZE = 31;  
  17. const float THRESHOLD = 0.9;  
  18. static int scaled_input_buffer_pos = 0;  
  19. static float scaled_input_buffer[FEATURIZER_INPUT_SIZE]; // raw audio converted to float  
  20. const int MAX_FEATURE_BUFFERS = 10; // set to buffer up to 1 second of audio in circular buffer  
  21. static float featurizer_input_buffers[MAX_FEATURE_BUFFERS][FEATURIZER_INPUT_SIZE]; // input to featurizer  
  22. static int featurizer_input_buffer_read = -1; // next read pos  
  23. static int featurizer_input_buffer_write = 0; // next write pos  
  24. static int dropped_frames = 0;  
  25. static float featurizer_output_buffer[FEATURIZER_OUTPUT_SIZE]; // 40 channels  
  26. static float classifier_output_buffer[CLASSIFIER_OUTPUT_SIZE]; // 31 classes  
  27. static int raw_audio_count = 0;  
  28. static char raw_audio_buffer[AUDIO_CHUNK_SIZE];  
  29. static int prediction_count = 0;  
  30. static int last_prediction = 0;  
  31. static int last_confidence = 0; // as a percentage between 0 and 100.  
  32. static uint8_t maxGain = 0;  
  33. static uint8_t minGain = 0;  
  34. int decibels = 0;  
  35. float min_level = 100;  
  36. float max_level = 0;  
  37. RGB_LED rgbLed;  
  38. static bool hasWifi = false;  
  39. static bool messageSending = true;  
  40. static uint64_t send_interval_ms;  
  41. int messageCount = 0; // holds ID  
  42. bool messageReceived = false;  
  43. char device_id[6];  
  44. AudioClass &Audio = AudioClass::getInstance();  
  45. //Constants and variables- End//  
  46. //************************************//  
Now, let us can add the codes for the configuration. Usually, you wouldn’t have to edit any codes in this section.
  1. void setup() {  
  2.     Screen.init();  
  3.     Screen.print(0, "IoT Device Demo");  
  4.     Screen.print(2, "Initializing...");  
  5.     Screen.print(3, " > Serial");  
  6.     Serial.begin(115200);  
  7.     // Initialize the WiFi module  
  8.     Screen.print(3, " > WiFi");  
  9.     hasWifi = false;  
  10.     initWifi();  
  11.     if (!hasWifi) {  
  12.         return;  
  13.     }  
  14.     Screen.print(3, " > Sensors");  
  15.     sensorInit();  
  16.     Screen.print(3, " > IoT Hub");  
  17.     DevKitMQTTClient_Init(true);  
  18.     DevKitMQTTClient_SetOption(OPTION_MINI_SOLUTION_NAME, "mlIoTPlatformDevice");  
  19.     DevKitMQTTClient_SetSendConfirmationCallback(sendConfirmationCallback);  
  20.     DevKitMQTTClient_SetMessageCallback(messageCallback);  
  21.     DevKitMQTTClient_SetDeviceTwinCallback(deviceTwinCallback);  
  22.     DevKitMQTTClient_SetDeviceMethodCallback(deviceMethodCallback);  
  23.     appstate = APPSTATE_Init;  
  24.     Serial.begin(115200);  
  25.     int filter_size = mfcc_GetInputSize(0);  
  26.     if (filter_size != FEATURIZER_INPUT_SIZE) {  
  27.         Serial.printf("Featurizer input size %d is not equal to %d\n", filter_size, FEATURIZER_INPUT_SIZE);  
  28.         show_error("Featurizer Error");  
  29.     }  
  30.     if (appstate != APPSTATE_Error) {  
  31.         ::memset(featurizer_input_buffers[0], 0, FEATURIZER_INPUT_SIZE);  
  32.         // give it a whirl !!  
  33.         mfcc_Filter(nullptr, featurizer_input_buffers[0], featurizer_output_buffer);  
  34.         // check audio gain and print the result.  
  35.         uint32_t id = Audio.readRegister(nau88c10_CHIPID_ADDR);  
  36.         if (id == NAU88C10_ID) {  
  37.             Serial.printf("Found audio device: NAU88C10\r\n");  
  38.         } else {  
  39.             Serial.printf("Found audio device: 0x%x\r\n", id);  
  40.         }  
  41.         // a default gain level of 4 seems to work pretty well.  
  42.         start_recording();  
  43.         Screen.clean();  
  44.         Screen.print(0, "Listening...");  
  45.         Screen.print(1, "A = min gain");  
  46.         Screen.print(2, "B = max gain");  
  47.         minGain = 0;  
  48.         maxGain = 7;  
  49.         set_gain();  
  50.         display_gain();  
  51.     }  
  52.     send_interval_ms = SystemTickCounterRead();  
  53. }  
As I mentioned earlier, every INO file will have its own setup() and loop() function. We can modify our setup() function as below.
  1. void setup() {  
  2.     Screen.init();  
  3.     Screen.print(0, "IoT Device Demo");  
  4.     Screen.print(2, "Initializing...");  
  5.     Screen.print(3, " > Serial");  
  6.     Serial.begin(115200);  
  7.     // Initialize the WiFi module  
  8.     Screen.print(3, " > WiFi");  
  9.     hasWifi = false;  
  10.     initWifi();  
  11.     if (!hasWifi) {  
  12.         return;  
  13.     }  
  14.     Screen.print(3, " > Sensors");  
  15.     sensorInit();  
  16.     Screen.print(3, " > IoT Hub");  
  17.     DevKitMQTTClient_Init(true);  
  18.     DevKitMQTTClient_SetOption(OPTION_MINI_SOLUTION_NAME, "mlIoTPlatformDevice");  
  19.     DevKitMQTTClient_SetSendConfirmationCallback(sendConfirmationCallback);  
  20.     DevKitMQTTClient_SetMessageCallback(messageCallback);  
  21.     DevKitMQTTClient_SetDeviceTwinCallback(deviceTwinCallback);  
  22.     DevKitMQTTClient_SetDeviceMethodCallback(deviceMethodCallback);  
  23.     appstate = APPSTATE_Init;  
  24.     Serial.begin(115200);  
  25.     int filter_size = mfcc_GetInputSize(0);  
  26.     if (filter_size != FEATURIZER_INPUT_SIZE) {  
  27.         Serial.printf("Featurizer input size %d is not equal to %d\n", filter_size, FEATURIZER_INPUT_SIZE);  
  28.         show_error("Featurizer Error");  
  29.     }  
  30.     if (appstate != APPSTATE_Error) {  
  31.         ::memset(featurizer_input_buffers[0], 0, FEATURIZER_INPUT_SIZE);  
  32.         // give it a whirl !!  
  33.         mfcc_Filter(nullptr, featurizer_input_buffers[0], featurizer_output_buffer);  
  34.         // check audio gain and print the result.  
  35.         uint32_t id = Audio.readRegister(nau88c10_CHIPID_ADDR);  
  36.         if (id == NAU88C10_ID) {  
  37.             Serial.printf("Found audio device: NAU88C10\r\n");  
  38.         } else {  
  39.             Serial.printf("Found audio device: 0x%x\r\n", id);  
  40.         }  
  41.         // a default gain level of 4 seems to work pretty well.  
  42.         start_recording();  
  43.         Screen.clean();  
  44.         Screen.print(0, "Listening...");  
  45.         Screen.print(1, "A = min gain");  
  46.         Screen.print(2, "B = max gain");  
  47.         minGain = 0;  
  48.         maxGain = 7;  
  49.         set_gain();  
  50.         display_gain();  
  51.     }  
  52.     send_interval_ms = SystemTickCounterRead();  
  53. }  
The function loop() will be called each 5 seconds, as I had set the INTERVAL as 5000 milliseconds.
  1. // Copyright (c) Microsoft. All rights reserved.  
  2. // Licensed under the MIT license.  
  3. // Interval time(ms) for sending message to IoT Hub  
  4. #define INTERVAL 5000  
  5. #define MESSAGE_MAX_LEN 256   
  6. #define TEMPERATURE_ALERT 30   
  7. #define DIRECT_METHOD_NAME "message"   
  8. // How many messages get sent to the hub before user has to press A to continue  
  9. #define MESSAGE_SEND_COUNT_LIMIT 350  
Now, we can edit our code of loop() function as below.
  1. void loop() {  
  2.     if (hasWifi) {  
  3.         if (messageSending && (int)(SystemTickCounterRead() - send_interval_ms) >= getInterval()) {  
  4.             if (appstate != APPSTATE_Error) {  
  5.                 if (dropped_frames > 0) {  
  6.                     Serial.printf("%d dropped frames\n", dropped_frames);  
  7.                     dropped_frames = 0;  
  8.                 }  
  9.                 // process all the buffered input frames  
  10.                 featurizer_input_buffer_read = next(featurizer_input_buffer_read);  
  11.                 decibels = get_prediction(featurizer_input_buffers[featurizer_input_buffer_read]);  
  12.             }  
  13.             // Send data  
  14.             char messagePayload[MESSAGE_MAX_LEN];  
  15.             float * newValues;  
  16.             newValues = setMessage(messageCount++, messagePayload, decibels);  
  17.             if (!messageReceived) {  
  18.                 // Update display  
  19.                 char buff[128];  
  20.                 sprintf(buff, "ID: %s \r\n Temp:%s°C \r\n Humidity:%s%% \r\n Pres:%smb \r\n", device_id, f2s( * (newValues), 1), f2s( * (newValues + 1), 1), f2s( * (newValues + 2), 1));  
  21.                 Screen.print(buff);  
  22.             }  
  23.             EVENT_INSTANCE * message = DevKitMQTTClient_Event_Generate(messagePayload, MESSAGE);  
  24.             DevKitMQTTClient_SendEventInstance(message);  
  25.             send_interval_ms = SystemTickCounterRead();  
  26.         } else {  
  27.             DevKitMQTTClient_Check();  
  28.         }  
  29.     }  
  30.     delay(10);  
  31. }  
utility.cpp
 
As you can see, once we get the values from the get_prediction () function, we are passing the decibel value to our setMessage() function, which we have defined in the utility.cpp file. Inside the setMessage () function, we will add the decibel value to JSON object using the function json_object_set_number ().
  1. // Copyright (c) Microsoft. All rights reserved.  
  2. // Licensed under the MIT license.  
  3. #include "HTS221Sensor.h"  
  4. #include "AzureIotHub.h"  
  5. #include "Arduino.h"  
  6. #include "parson.h"  
  7. #include <assert.h>  
  8. #include "config.h"  
  9. #include "RGB_LED.h"  
  10. #include "Sensor.h"  
  11. #include "LIS2MDLSensor.h"  
  12. #define RGB_LED_BRIGHTNESS 32  
  13. DevI2C * i2c;  
  14. HTS221Sensor * ht_sensor;  
  15. LPS22HBSensor * pressureSensor;  
  16. LSM6DSLSensor * acc_gyro;  
  17. LIS2MDLSensor * lis2mdl;  
  18. int gAxes[3];  
  19. int mAxes[3];  
  20. static RGB_LED rgbLed;  
  21. static int interval = INTERVAL;  
  22. int getInterval() {  
  23.     return interval;  
  24. }  
  25. void blinkLED() {  
  26.     rgbLed.turnOff();  
  27.     rgbLed.setColor(RGB_LED_BRIGHTNESS, 0, 0);  
  28.     delay(500);  
  29.     rgbLed.turnOff();  
  30. }  
  31. void blinkSendConfirmation() {  
  32.     rgbLed.turnOff();  
  33.     rgbLed.setColor(0, 0, RGB_LED_BRIGHTNESS);  
  34.     delay(500);  
  35.     rgbLed.turnOff();  
  36. }  
  37. void parseTwinMessage(DEVICE_TWIN_UPDATE_STATE updateState,  
  38.     const char * message) {  
  39.     JSON_Value * root_value;  
  40.     root_value = json_parse_string(message);  
  41.     if (json_value_get_type(root_value) != JSONObject) {  
  42.         if (root_value != NULL) {  
  43.             json_value_free(root_value);  
  44.         }  
  45.         LogError("parse %s failed", message);  
  46.         return;  
  47.     }  
  48.     JSON_Object * root_object = json_value_get_object(root_value);  
  49.     double val = 0;  
  50.     if (updateState == DEVICE_TWIN_UPDATE_COMPLETE) {  
  51.         JSON_Object * desired_object = json_object_get_object(root_object, "desired");  
  52.         if (desired_object != NULL) {  
  53.             val = json_object_get_number(desired_object, "interval");  
  54.         }  
  55.     } else {  
  56.         val = json_object_get_number(root_object, "interval");  
  57.     }  
  58.     if (val > 500) {  
  59.         interval = (int) val;  
  60.         LogInfo(">>> Device twin updated: set interval to %d", interval);  
  61.     }  
  62.     json_value_free(root_value);  
  63. }  
  64. void sensorInit() {  
  65.     i2c = new DevI2C(D14, D15);  
  66.     ht_sensor = new HTS221Sensor( * i2c);  
  67.     ht_sensor - > init(NULL);  
  68.     pressureSensor = new LPS22HBSensor( * i2c);  
  69.     pressureSensor - > init(NULL);  
  70.     acc_gyro = new LSM6DSLSensor( * i2c, D4, D5);  
  71.     acc_gyro - > init(NULL);  
  72.     acc_gyro - > enableAccelerator();  
  73.     lis2mdl = new LIS2MDLSensor( * i2c);  
  74.     lis2mdl - > init(NULL);  
  75. }  
  76. float readTemperature() {  
  77.     ht_sensor - > reset();  
  78.     float temperature = 0;  
  79.     ht_sensor - > getTemperature( & temperature);  
  80.     return temperature;  
  81. }  
  82. float readHumidity() {  
  83.     ht_sensor - > reset();  
  84.     float humidity = 0;  
  85.     ht_sensor - > getHumidity( & humidity);  
  86.     return humidity;  
  87. }  
  88. float readPressure() {  
  89.     float pressure = 0;  
  90.     pressureSensor - > getPressure( & pressure);  
  91.     return pressure;  
  92. }  
  93. void setAccelAxes() {  
  94.     acc_gyro - > getXAxes(gAxes);  
  95. }  
  96. void setMagAxes() {  
  97.     lis2mdl - > getMAxes(mAxes);  
  98. }  
  99. float * setMessage(int messageId, char * payload, int decibels) {  
  100.     static float newValues[3];  
  101.     JSON_Value * root_value = json_value_init_object();  
  102.     JSON_Object * root_object = json_value_get_object(root_value);  
  103.     char * serialized_string = NULL;  
  104.     json_object_set_number(root_object, "messageId", messageId);  
  105.     json_object_set_number(root_object, "decibels", decibels);  
  106.     // Obtain values  
  107.     float temperature = readTemperature();  
  108.     float humidity = readHumidity();  
  109.     float pressure = readPressure();  
  110.     setAccelAxes();  
  111.     setMagAxes();  
  112.     // Set new values  
  113.     newValues[0] = temperature;  
  114.     newValues[1] = humidity;  
  115.     newValues[2] = pressure;  
  116.     bool temperatureAlert = false;  
  117.     // Set temp json  
  118.     json_object_set_number(root_object, "temperature", temperature);  
  119.     // Set humidity json  
  120.     json_object_set_number(root_object, "humidity", humidity);  
  121.     // Set pressure json  
  122.     json_object_set_number(root_object, "pressure", pressure);  
  123.     // Set gyro axes  
  124.     json_object_set_number(root_object, "accelX", gAxes[0]);  
  125.     json_object_set_number(root_object, "accelY", gAxes[1]);  
  126.     json_object_set_number(root_object, "accelZ", gAxes[2]);  
  127.     // Set mag axes  
  128.     json_object_set_number(root_object, "magX", mAxes[0]);  
  129.     json_object_set_number(root_object, "magY", mAxes[1]);  
  130.     json_object_set_number(root_object, "magZ", mAxes[2]);  
  131.     serialized_string = json_serialize_to_string_pretty(root_value);  
  132.     snprintf(payload, MESSAGE_MAX_LEN, "%s", serialized_string);  
  133.     json_free_serialized_string(serialized_string);  
  134.     json_value_free(root_value);  
  135.     return newValues;  
  136. } 
You should also add the files featurizer.h and featurizer.s to get it working. You can get these files from the source code repository mentioned above.
 
Compile and Upload to the Device
 
As we have already made the needed changes, it is time to compile the device solution and upload the same to our device. Press F1 and select Azure IoT Device Workbench: Compile Device Code. If you ever get an error as “error: utility.h: No such file or directory”, please compile the device code again. If you are facing any unexpected errors, please delete the “.build” folder and compile it again.
 
Once you get a message as ” [Done] Finished verify sketch – Main.ino ” in your output window, you can upload the solution to your device. To do so, press F1 again, and select ” Azure IoT Device Workbench: Upload Device Code”. Just make sure that the device is connected to your machine. If everything goes well, you will be getting a message as ” [Done] Uploaded the sketch: Main.ino”.
 
Don't forget to see the GitHub repository for the full code.
 
Device to Cloud Messages
 
Now, your device will be sending the Decibels data to Azure IoT Hub. Let’s see that in the D2C Monitoring window.
 
Send MXChip Data to Cloud 
 
Send MXChip Data to Cloud
  1. {  
  2.     "messageId": 119,  
  3.     "decibels": 94,  
  4.     "temperature": 26.4,  
  5.     "humidity": 34.400002,  
  6.     "pressure": 994.585693,  
  7.     "accelX": -9,  
  8.     "accelY": -12,  
  9.     "accelZ": 993,  
  10.     "magX": -156,  
  11.     "magY": -5,  
  12.     "magZ": -253  
  13. } 

Conclusion

 
Wow!.So, we have learned,
  • How to detect the noise level in MXChip
  • How to use AudioClassV2
  • How to send device data to the Azure IoT Hub
Please consider reading my IoT articles here for more.
 
Your turn. What do you think?
 
Thanks a lot for reading. Did I miss anything that you think may be needed in this article? Did you find this post useful? Do share your feedback.


Similar Articles