Part 2.2: What does Minions have to do with MQTT, Processing and Chuck ?

I have been a fan of the audio programming language Chuck ever since i came across it few months ago. Its used by the Laptop Orchestras from the Princeton University and Stanford. For tinkerers like us its a wonderful tool to mix the hardware hacking with computer synthesized sound. I can’t play any instruments. But i can code. I understand math. So this tool frees me to apply my programming skills to create new sounds.

I intended to use Chuck in the Holiday Lights Road Test to add music and sound effects. With all the references to Minions through out element14 community, today it struck me that this is a great opportunity to give voice to them.

Installing and using chuck is as simple as using Arduino. So go ahead. Download. Install. Check out the tutorial. Play around. Integrate with your awesome projects.

Now here is how my part of it works.

For now i am creating the MQTT message through mosquitto_pub which i described in an earlier post. Once i get my shipment of components, my wireless controllers will generate the message. Actually i am going to add capacitive touch to the minion christmas ornaments  and when any one touches the ornaments a MQTT message will be sent out resulting in Chuck generating the sound. (Update Dec 20.. Built it.. See here.. ).. Oops..I forgot.. Robert Peter Oakes got those minions. Now i have to go shopping for some minions.. Unless Santa doctorcdf has surprised us with some minion ornaments in our package.

Screencast showing the software pieces in action:

Chuck Program for Minions (Called a Shred) (Minion.ck)

Note: This is a very simple use of sound buffer to load minion wav files. There are more powerful tools in Chuck to create and experiment with new sounds right from scratch. One of my todo list of projects is to create a DIY Drum kit from everyday objects and use chuck to generate actual sound.

You can copy this code in to miniAudicle, save it to a folder and drop the sound files in to a wav folder.I have attached the minion wav files for any one to use.

// OSC receiver
OscIn oin;
// OSC message
OscMsg msg;

// this is listening on port 6449 for OSC messages
6449 => oin.port;

// My address is this and i am waiting for a value of type integer
oin.addAddress( "/WiFiTree/Minion/Sound, i" );

//define sound buffer and chuck it to dac
SndBuf buf => dac;

// load the file paths into an array
[
me.dir() + "/wav/Aww.wav",
me.dir() + "/wav/bottom.wav",
me.dir() + "/wav/conv.wav",
me.dir() + "/wav/fight.wav",
me.dir() + "/wav/laugh.wav",
me.dir() + "/wav/ok.wav",
me.dir() + "/wav/what.wav",
me.dir() + "/wav/yay.wav",
me.dir() + "/wav/yell.wav"
] @=> string filePaths[];



while ( true )
{
    // waiting for OSC event
    oin => now;

    // when the message arrived
    while ( oin.recv(msg) != 0 )
    {
        // getInt fetches the expected int
        msg.getInt(0) => int soundIndex;
          <<< "got (via OSC):",soundIndex >>>;
  
          if(soundIndex >= 0 && soundIndex <9)
          {
          filePaths[soundIndex] => buf.read;
          }
        // print
        // set play pointer to beginning
        0 => buf.pos;
    }
}

MQTT Subscriber and OSC Message Sender

In Part 2.1 i described how i set up MQTT client libraries in Processing. Here, i created the following MQTT Subscriber for listening for those messages. Once the messages are received, I translate them to generate Open Sound Control (OSC) messages to pass them to Chuck.

OscMQTTSubscriber.java:

import org.eclipse.paho.client.mqttv3.*;
import oscP5.*;
import netP5.*;


public class OscMqttSubscriber implements MqttCallback {


OscP5 oscClient;
NetAddress chuckAddress;


public OscMqttSubscriber(OscP5 oscClient,NetAddress chuckAddress) {


    this.oscClient=oscClient;
    this.chuckAddress=chuckAddress;
}


@Override
public void messageArrived(String topic, MqttMessage message)
        throws Exception {
    println(message.toString());
    this.SendOscMessage(message.toString());
}


private void SendOscMessage(String message)
{
    //Create Osc message for address
    OscMessage minionSound = new OscMessage("/WiFiTree/Minion/Sound");

    minionSound.add(Integer.parseInt(message));


    oscClient.send(minionSound, chuckAddress);

    println("messsage sent");
}


@Override
public void connectionLost(Throwable cause) {
    //TODO: handle connection loss scenario
}




@Override
public void deliveryComplete(IMqttDeliveryToken token) {
}
}

minonsmic.pde

import oscP5.*;
import netP5.*;


OscP5 oscClient;
NetAddress chuckAddress;
MqttClient mqttClient;


void setup()
{
  setupOSCClient(); //set OSC Client first
  setupMQTTClient();
}


void setupOSCClient()
{
  //This sketch is listening for OSC packets as well at port 12000
  //Not necessary in this example but the library doesn't allow just a client creating
   oscClient=new OscP5(this,12000);
   chuckAddress = new NetAddress("127.0.0.1",6449);
}


void setupMQTTClient()
{
  try{
     mqttClient=new MqttClient("tcp://localhost:1883", "ProcessingTestClient");
     mqttClient.connect();

     OscMqttSubscriber oscMqttSub=new OscMqttSubscriber(oscClient,chuckAddress);
     mqttClient.setCallback(oscMqttSub);

     mqttClient.subscribe("WiFiTree/FromController/MinionSound");
  }catch(Exception ex)
  {
    println(ex.getMessage());
  }
}


void draw()
{
}

Hope this post helps some one to add a new tool to their kit.