[Upcycle it Design Challenge] Embedded Web SDR client on Analog Radio Receiver #10: Making the Web SDR Client work!

Decoding the WEB SDR transmission. Method & Software that works!

When I wrote the previous post I was really stuck on how to implement my idea. That is why I broadcasted that help was needed. Not only to the Element14 Community that is following the Upcycle it project but to other Ham Communities as well.

It was Andy that sent the first good response.

Andy Clark (Workshopshed)  May 23, 2017 9:34 PM

Perhaps if you can get access to the stream rather than the web page you could do something like this example.

 

http://www.codegist.net/snippet/javascript/headless-audio-streamjs_tam_javascript

That encouraged me and then Andy sent the right link.

Andy Clark (Workshopshed)  May 24, 2017 6:11 AM (in response to Konstantinos Konstas)

Yes, they want you to "look" at the page to see the adverts.

 

Here's some more headless browsers

 

https://github.com/dhamaniasad/HeadlessBrowsers/blob/master/README.md

A whole list of Headless Browsers. That must be the name of the game, I thought. I went through the list. At first I saw CEF, Chromium Embedded Framework, and then going down the list with the  many unknown buzzwords, I was caught by the name Headles Chromium. It explained: Chromium feature activated with the --headlesss flag, currently availible in the nightly build of Chromium, not yet released.

I hit the link https://chromium.googlesource.com/chromium/src/+/lkgr/headless/README.md and Bingo. What I read sounded very promising to me. Before going to writing any code I could easily try from bash on my Debian laptop. Great happiness for a 66 year old “terminal grown” command line lover.

$ chrome --headless --remote-debugging-port=9222 https://chromium.org

was the example that I transformed to :

$ chrome --headless -–disable-gpu --remote-debugging-port=9222 http://websdr.ewi.utwente.nl:8901/?tune=198am &

Live and let die I said to myself as I pressed the enter key after serious consideration. Bingo again, it worked and obviously “no ads”!

I heard the station on my headphones. Well I had something, as below there was sample js code!

Now I need to try it on Edison, but how? I started revising my development plan.

    Edison needs Chromium and some tools.  

    Write some code to test the task, but include the Client commands for my OLED and     joystick UI

    Make the audio from USB dongle play permanently stable. Priority One

All the above working out of the box last week! Great expectations. OK let us start.

Chromium on Yocto. I read a few things, I saw the recipes and started to wonder how to implement it. I also wanted to use latest Chromium version that could play on Edison not my Ubuntu 16.04 where both Chromium and Chrome amd64 version releases 59 (nightly)  installed with “sudo apt-get install xxx” quite smoothly from here:  https://www.google.com/chrome/browser/beta.html

And so I did. But I tried to find a 32bit version 59, but no chance.

I found Chromium version 57 but not in a format that could be installed with Yocto recipes.

I took the big decision to try Ubilinux and take it from there after. Another great task.

Nodejs, NPM, GYP, conversion of Edison OLED libraries to JS under a new environment, then install ALSA  not included in the main package, make Audio from USB play, install MRAA/UPM and play and find the right version of Ubilinux as Emutex has stopped the support.

Googling a lot I ended up to try flash the Jubilinux v0.1.1 which is the continuation of Ubilinux

according to what I read.

I got it from http://www.jubilinux.org/dist/jubilinux-v0.1.1.zip and decided to try flashing it to my so thought damaged Edison core that I put back to Intel’s breadboard.

I deployed it with dfu tool and strangly enough dfu recognized the “dead” board and started flashing. After ten minutes I had another working Edison, codename “jubilinux”. Why not? It saved my broken board, so it deserves it. More on Jubilinux from   http://www.jubilinux.org

I then set up Wifi , tested SSH and SCP and I was in bussines. Ubuntu SSH and SCP bash commands worked smoothly and soon I had my Debian Edison ready for me.

Sparkfun  has a nice tutorial on installing Ubilinux https://learn.sparkfun.com/tutorials/loading-debian-ubilinux-on-the-edison which I used for the Jubilinux as it is considered to be the successor of Ubilinux.

Then came MRRA. Again installation guide from here. https://learn.sparkfun.com/tutorials/installing-libmraa-on-ubilinux-for-edison and then some useful utils like htop and less from the debian repos. Simply do apt-get install.

I chose  not to install sudo yet to speed things up, but changed the default passwords for root and edison users with passwd command.

Time to make apt-get update and upgrade to make sure that everything goes well.

All fine. Next step ALSA Audio. $ apt-get install alsa-utils will do. I also scp a test wav file from my laptop for testing. If simply trying aplay xxxxxx.wav works, you are in business.

If it does not, you need to tweak the system.

First I looked here: https://raspberrypi.stackexchange.com/questions/19705/usb-card-as-my-default-audio-device  and then

root@jubilinux:~# aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: Loopback [Loopback], device 0: Loopback PCM [Loopback PCM]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7
card 0: Loopback [Loopback], device 1: Loopback PCM [Loopback PCM]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
aplay -l  Subdevice #7: subdevice #7
card 1: dummyaudio [dummy-audio], device 0: 14 []
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 2: Device [USB Audio Device], device 0: USB Audio [USB Audio]
  Subdevices: 0/1
  Subdevice #0: subdevice #0

I followed the instructions of the site.

I edit /etc/asound.conf to look like this to set the default:

$ nano /etc/asound.conf


pcm.!default {
type hw
card 2
device 0
}

It worked for me and aplay  xxxxxx.wav produced sound. No reboot needed, although I made one, just in case.

Next step. Installing nodejs. I initially installed from apt-get install nodejs old versions of node and npm that did not cope with the test js code that I had prepared meanwhile.

On my laptop $ node -v  produced version 7.10.0 while on jubilinux version 4.2.6 was installed.

I then removed and going through this repo https://tecadmin.net/install-latest-nodejs-npm-on-ubuntu/ and curl -sL https://deb.nodesource.com/setup_6.x |  bash -

Unfortunately version 7 that you can get from the same repo is 64 bit and does not work on Edison. So I used the previous version and crossed my fingers.

Then apt-get install nodejs and eventually installed smoothly together with npm, gyp and the rest utilities. It was time for the big test that I could not run on the Laptop, Compile the OLED driver.

Instructions from here: https://www.npmjs.com/package/edison-oled

$ npm install edison-oled

You may need to use the --unsafe-perm option if you get the warning "cannot run in wd".

$ npm install edison-oled --unsafe-perm

I started with the first but produced an error in a specific module.js in two instances. To shorten things, when I ran the second option, everything went well and edison-oled was installed into node_modules, as it had done on the Yocto machine.

Die hard test.

root@jubilinux:~# cd node_modules/edison-oled/tests
root@jubilinux:~/node_modules/edison-oled/tests# node mytest1.js
--setup screen
---width: 64
---height: 48
--show start
--buttons
press Button A:
pressed A: 0
Button A pressed.
press Button B:
Button B pressed.
press Select:
Button Select pressed.
move joystick up:
joystick pressed up.
move joystick down:
joystick pressed down.
move joystick left:
joystick pressed left.
move joystick right:
joystick pressed right.
--show main
All done.
root@jubilinux:~/node_modules/edison-oled/tests#

OK so far success. Test program too silly to publish. Wait for the real one. But by now Jubilinux module had a working OLED screen and Audio, while Node.js was working fine and I considerd it fully operational for development.

I thought of trying XDK, so I loaded it but no success with Jubilinux. Running out of time to solve out the issue. I can make my development from my plain editor. No big issue.

Next step install more tools that I use in my code. Namely

# npm install chrome-remote-interface

and

# npm install keypress

Both are needed from my code to run. I had to make the Terminal test WEB SDR Client really fast. Together with a bash script.

Then last but not the least,

# apt-get install chromium

I managed to get the following:

root@jubilinux:~/node_modules/edison-oled/tests# chromium –version
Chromium 57.0.2987.98 Built on 8.7, running on Debian 8.8

So it is working, while on my laptop I had:

kostas@kostas-HP-ProBook-450-G2:~$ chromium-browser --version
Chromium 58.0.3029.110 Built on Ubuntu , running on Ubuntu 16.04

Ok. Not bad again. Irish luck so far.

Next step the code

So next step is to go for the real code that I had meanwhile prepared.

I decided to run the same test script again as I had done from my laptop, now that I had Chromium installed on Jubilinux Edison.

root@jubilinux:~# chrome --headless –disable-gpu --remote-debugging-port=9222 http://websdr.ewi.utwente.nl:8901/?tune=198am &

Yes it did work and the voice of BBC4 came on my headphones. What can I say. I was really happy.

Now it was time for the final test with my code that I had compiled on my Laptop and it did work there. But Jubilinux Edison was the final frontier! But some more comments before that.

During the previous days I had issues with the SSH which at times seemed to slow down and even get stubbornly stuck. I had no clue about it, fearing the service of the provider, but on Sunday I presented my project to some fellow Hams. One of them commented that Edison module without a Wifi antenna could have issues.

Indeed my laptop and Jubilinux Edison were quite far from the router. Luckily I went to my lab and found a scrapped Wifi antenna system from an old laptop that had the same antenna connector with Edison, so I had it connected and the problem was gone.

The test WEB SDR Client  takes uses chrome-remote-interface and keypress, both installed with NPM. Initial parameters are passed from a bash script which includes the WEB SDR url and the tuning word as explained on my previous posts.  Initial Frequency and Mode are parsed along with Web browser name, the headless command and the call to the main js program websdr_controller.js

The bash script websdr.sh is given executable rights with chmod 755.

The code is here:

#!/bin/bash

RUNTIME=websdr_controller_edison.js
BROWSER=chromium
START_FREQ="${2:-198}"
START_MODE="${3:-am}"
URL=${1}/?tune=${START_FREQ}${START_MODE}

# Setup broweser in headless mode
$BROWSER --headless --disable-gpu --remote-debugging-port=9222 ${URL}&
PID=$!
# Start WebSDR controller

node ${RUNTIME} ${START_FREQ} ${START_MODE}

kill $PID

The program websdr_controller.js has a primitive keyboard interface that emulates the follow commands:

FREQUENCY

F+ (up)               <u>

F- (down)           <d>

FREQUENCY STEP SIZE

1 (0.1kHz)          <q>

2 (1kHz)            <w>

3 (9kHz)            <e>

MODE (DEMODULATION)

CW                    <z>

LSB                   <x>

USB                   <c>

AM                     <v>

FM                     <b>

AMSync             <n>

 

EXIT                   <ctrl + c>

Messages appear with the console.log directive.

Very primitive CP/M or DOS style, but serving well for testing and debugging purposes and shows what the final program will do on the OLED.

The code is here:

const CDP = require('chrome-remote-interface');
var keypress = require('keypress');
var verbose = true;
    
CDP((client) => {
    
    function killClient() {
        client.close();
    }
    
    var args = process.argv;
    if(args.length != 4) {
        killClient();
    } else {
        var step = 1;
        var currentMode = args[3];
        var currentFrequency = parseFloat(args[2]);
        
        function intro() {
            console.log("Step up ['u'] / down ['d'].\nStep size: 1 ['q'], 2 ['w'], 3 ['e'].\nMode: CW ['z'], LSB ['x'], USB ['c'], AM ['v'], FM ['b'], AMSync ['n'].\nCtrl^C to exit.")
            getFrequency();
            getMode();
        }
        
        function addKeyPressListener() {
            // make `process.stdin` begin emitting "keypress" events 
            keypress(process.stdin);
            // listen for the "keypress" event 
            process.stdin.on('keypress', keyPressEvent);
            process.stdin.setRawMode(true);
            process.stdin.resume();
        }


        function changeStep(stepIn) {
            step = stepIn;
            if(verbose)
                console.log('step = ' + step);
        }

        function changeFreq(up, step) {
            var str = up ? '+' : '-';
            client.send('Runtime.evaluate', {'expression': 'freqstep(' + str + step + '); nominalfreq();'}, (error, response) => {
                if(error) {
                    console.log(error);
                    return;
                }
                currentFrequency = response.result.value;
                getFrequency();
            });
            //if(verbose)
            //    console.log('stepped ' + (up ? 'up' : 'down'));
        }

        function setMode(mode) {
                client.send('Runtime.evaluate', {'expression': 'set_mode(\'' + mode + '\'); nominalfreq();'}, (error, response) => {
                if(error) {
                    console.log(error);
                    return;
                }
                currentFrequency = response.result.value;
                getFrequency();
            });
            currentMode = mode;
            getMode();
        }

        function queryFrequency() {
            client.send('Runtime.evaluate', {'expression': 'nominalfreq()'}, (error, response) => {
                if(error) {
                    console.log(error);
                    return;
                }
                currentFrequency = response.result.value;
            });
        }

        function getFrequency() {
            console.log('Frequency = ' + parseFloat(currentFrequency).toFixed(1) + 'kHz');
        }

        function getMode() {
            console.log('Mode = ' + currentMode);
        }

        function keyPressEvent(ch, key) {
            //console.log('got "keypress"', key);
            if (key && key.name == 'u') {
                changeFreq(true, step);
            }
            else if (key && key.name == 'd') {
                changeFreq(false, step);
            }
            else if(key && key.name == 'q') {
                changeStep(1);
            }
            else if(key && key.name == 'w') {
                changeStep(2);
            }
            else if(key && key.name == 'e') {
                changeStep(3);
            }
            else if(key && key.name == 'z') {
                setMode('cw');
                //queryFrequency();
            }
            else if(key && key.name == 'x') {
                setMode('lsb');
            }
            else if (key && key.ctrl && key.name == 'c') {
                process.stdin.pause();
                killClient();
            }
            else if(key && key.name == 'c') {
                setMode('usb');
            }
            else if(key && key.name == 'v') {
                setMode('am');
            }
            else if(key && key.name == 'b') {
                setMode('fm');
            }
            else if(key && key.name == 'n') {
                setMode('amsync');
            }
            //        getFrequency();
        }
        addKeyPressListener();
        intro();
    }
});

So ready for the last test. Let’s fire it up.

root@jubilinux:~# ls
01.wav    node          start.sh    websdr.sh
mraa    node_modules  websdr    websdr_controller.js

root@jubilinux:~# ./websdr.sh
[0529/094035.206268:WARNING:audio_manager.cc(321)] Multiple instances of AudioManager detected
[0529/094035.206863:WARNING:audio_manager.cc(278)] Multiple instances of AudioManager detected
Step up ['u'] / down ['d'].
Step size: 1 ['q'], 2 ['w'], 3 ['e'].
Mode: CW ['z'], LSB ['x'], USB ['c'], AM ['v'], FM ['b'], AMSync ['n'].
Ctrl^C to exit.
Frequency = 198.0kHz
Mode = am
step = 2 aka 1 kHz
Frequency = 199.0kHz
Frequency = 200.0kHz
Frequency = 201.0kHz
Frequency = 202.0kHz
Frequency = 203.0kHz
Frequency = 204.0kHz
Frequency = 205.0kHz
Frequency = 204.0kHz
Frequency = 203.0kHz
Frequency = 202.0kHz
Frequency = 201.0kHz
Frequency = 200.0kHz
Frequency = 199.0kHz
Frequency = 198.0kHz
Frequency = 197.0kHz
Frequency = 196.0kHz
Frequency = 195.0kHz
Frequency = 194.0kHz
Frequency = 193.0kHz
Frequency = 192.0kHz
Frequency = 191.0kHz
Frequency = 190.0kHz
Frequency = 191.0kHz
Frequency = 192.0kHz
Frequency = 193.0kHz
Frequency = 194.0kHz
Frequency = 195.0kHz
Frequency = 196.0kHz
Frequency = 197.0kHz
Frequency = 198.0kHz
Frequency = 199.0kHz
Frequency = 200.0kHz
Frequency = 201.0kHz
Frequency = 200.0kHz
Frequency = 199.0kHz
Frequency = 198.0kHz
step = 1 aka 0.1 kHz
Frequency = 198.1kHz
Frequency = 198.2kHz
Frequency = 198.3kHz
Frequency = 198.2kHz
Frequency = 198.1kHz
Frequency = 198.0kHz
step = 3 aka 9 kHz
Frequency = 207.0kHz
Frequency = 198.0kHz
Frequency = 189.0kHz
Frequency = 183.0kHz
Frequency = 171.0kHz
Frequency = 162.0kHz
Frequency = 153.0kHz
Frequency = 144.0kHz
Frequency = 153.0kHz
Frequency = 162.0kHz
Frequency = 171.0kHz
Frequency = 183.0kHz
Frequency = 189.0kHz
Frequency = 198.0kHz
Frequency = 207.0kHz
Frequency = 216.0kHz
Frequency = 225.0kHz
Frequency = 234.0kHz
Frequency = 243.0kHz
Frequency = 252.0kHz
Frequency = 261.0kHz
Frequency = 270.0kHz
Frequency = 279.0kHz
Frequency = 288.0kHz
Frequency = 297.0kHz
Frequency = 306.0kHz
Frequency = 315.0kHz
Frequency = 324.0kHz
Frequency = 333.0kHz
Frequency = 342.0kHz
Frequency = 351.0kHz
Frequency = 360.0kHz
Frequency = 369.0kHz
Frequency = 378.0kHz
Frequency = 387.0kHz
Frequency = 396.0kHz
Frequency = 405.0kHz
Frequency = 414.0kHz
Frequency = 423.0kHz
Frequency = 432.0kHz
Frequency = 441.0kHz
Frequency = 450.0kHz
Frequency = 459.0kHz
Frequency = 468.0kHz
Frequency = 477.0kHz
Frequency = 486.0kHz
Frequency = 495.0kHz
Frequency = 504.0kHz
Frequency = 513.0kHz
Frequency = 522.0kHz
Frequency = 531.0kHz
Frequency = 540.0kHz
Frequency = 549.0kHz
Frequency = 558.0kHz
Frequency = 567.0kHz
Frequency = 576.0kHz
Frequency = 585.0kHz
Frequency = 594.0kHz
Frequency = 603.0kHz
Frequency = 612.0kHz
Frequency = 621.0kHz
Frequency = 630.0kHz
Frequency = 639.0kHz
Frequency = 648.0kHz
Frequency = 657.0kHz
Frequency = 666.0kHz
Frequency = 675.0kHz
Frequency = 684.0kHz
Frequency = 693.0kHz
Frequency = 702.0kHz
Frequency = 711.0kHz
Frequency = 702.0kHz
Frequency = 693.0kHz
Frequency = 702.0kHz
Frequency = 711.0kHz
Frequency = 720.0kHz
Frequency = 729.0kHz
Frequency = 738.0kHz
Frequency = 747.0kHz
Frequency = 756.0kHz
Frequency = 765.0kHz
Frequency = 774.0kHz
Frequency = 783.0kHz
Frequency = 792.0kHz
Frequency = 801.0kHz
Frequency = 810.0kHz
Frequency = 819.0kHz
Frequency = 828.0kHz
Frequency = 837.0kHz
Frequency = 828.0kHz
step = 2 aka 1 kHz
Frequency = 827.0kHz
Frequency = 826.0kHz
Frequency = 825.0kHz
Frequency = 824.0kHz
Frequency = 823.0kHz
Frequency = 822.0kHz
Frequency = 821.0kHz
Frequency = 820.0kHz
Frequency = 821.0kHz
Frequency = 822.0kHz
Frequency = 823.0kHz
Frequency = 824.0kHz
Frequency = 825.0kHz
Frequency = 826.0kHz
Frequency = 827.0kHz
Frequency = 828.0kHz
Frequency = 829.0kHz
Frequency = 830.0kHz
Frequency = 831.0kHz
Frequency = 832.0kHz
Frequency = 833.0kHz
Frequency = 832.0kHz
Frequency = 831.0kHz
Frequency = 830.0kHz
Frequency = 829.0kHz
Frequency = 830.0kHz
Frequency = 831.0kHz
Frequency = 832.0kHz
Frequency = 833.0kHz
Frequency = 834.0kHz
Frequency = 833.0kHz
Frequency = 832.0kHz
Frequency = 831.0kHz
Frequency = 830.0kHz
Frequency = 829.0kHz
Frequency = 828.0kHz
step = 3 aka 9 kHz
Frequency = 837.0kHz
Frequency = 846.0kHz
Frequency = 855.0kHz
Frequency = 864.0kHz
Frequency = 873.0kHz
Frequency = 882.0kHz
Frequency = 891.0kHz
Frequency = 900.0kHz
Frequency = 909.0kHz
Frequency = 918.0kHz
Frequency = 927.0kHz
Frequency = 936.0kHz
Frequency = 945.0kHz
Frequency = 954.0kHz
Frequency = 963.0kHz
Frequency = 972.0kHz
Frequency = 981.0kHz
Frequency = 990.0kHz
Frequency = 999.0kHz
Frequency = 1008.0kHz
Frequency = 1017.0kHz
Frequency = 1008.0kHz
root@jubilinux:~#

So, it does work. I have to fix the warning issue, but otherwise I listen with it from now on. Even on its present state with Console control!

Now I must write the final code and put everything together till Saturday. I think it is highly feasible.

I only have to go down town to a robotics specialty shop to get a joystick button that I do not have that retails for 1.5 Euro. I think it is worth the cost!

I will put another post up with the pics and the final code and a small video. Last task. Stay tuned!