element14 Community
element14 Community
    Register Log In
  • Site
  • Search
  • Log In Register
  • Community Hub
    Community Hub
    • What's New on element14
    • Feedback and Support
    • Benefits of Membership
    • Personal Blogs
    • Members Area
    • Achievement Levels
  • Learn
    Learn
    • Ask an Expert
    • eBooks
    • element14 presents
    • Learning Center
    • Tech Spotlight
    • STEM Academy
    • Webinars, Training and Events
    • Learning Groups
  • Technologies
    Technologies
    • 3D Printing
    • FPGA
    • Industrial Automation
    • Internet of Things
    • Power & Energy
    • Sensors
    • Technology Groups
  • Challenges & Projects
    Challenges & Projects
    • Design Challenges
    • element14 presents Projects
    • Project14
    • Arduino Projects
    • Raspberry Pi Projects
    • Project Groups
  • Products
    Products
    • Arduino
    • Avnet & Tria Boards Community
    • Dev Tools
    • Manufacturers
    • Multicomp Pro
    • Product Groups
    • Raspberry Pi
    • RoadTests & Reviews
  • About Us
    About the element14 Community
  • Store
    Store
    • Visit Your Store
    • Choose another store...
      • Europe
      •  Austria (German)
      •  Belgium (Dutch, French)
      •  Bulgaria (Bulgarian)
      •  Czech Republic (Czech)
      •  Denmark (Danish)
      •  Estonia (Estonian)
      •  Finland (Finnish)
      •  France (French)
      •  Germany (German)
      •  Hungary (Hungarian)
      •  Ireland
      •  Israel
      •  Italy (Italian)
      •  Latvia (Latvian)
      •  
      •  Lithuania (Lithuanian)
      •  Netherlands (Dutch)
      •  Norway (Norwegian)
      •  Poland (Polish)
      •  Portugal (Portuguese)
      •  Romania (Romanian)
      •  Russia (Russian)
      •  Slovakia (Slovak)
      •  Slovenia (Slovenian)
      •  Spain (Spanish)
      •  Sweden (Swedish)
      •  Switzerland(German, French)
      •  Turkey (Turkish)
      •  United Kingdom
      • Asia Pacific
      •  Australia
      •  China
      •  Hong Kong
      •  India
      •  Japan
      •  Korea (Korean)
      •  Malaysia
      •  New Zealand
      •  Philippines
      •  Singapore
      •  Taiwan
      •  Thailand (Thai)
      •  Vietnam
      • Americas
      •  Brazil (Portuguese)
      •  Canada
      •  Mexico (Spanish)
      •  United States
      Can't find the country/region you're looking for? Visit our export site or find a local distributor.
  • Translate
  • Profile
  • Settings
EZ-EV Challenge
  • Challenges & Projects
  • Design Challenges
  • EZ-EV Challenge
  • More
  • Cancel
EZ-EV Challenge
Forum SmartAssist EV - Mobile Robot Control and Live Video Streaming - Part 7
  • News
  • Projects
  • Forum
  • DC
  • Leaderboard
  • Files
  • Members
  • More
  • Cancel
  • New
Join EZ-EV Challenge to participate - click to join for free!
Actions
  • Share
  • More
  • Cancel
Forum Thread Details
  • Replies 0 replies
  • Subscribers 60 subscribers
  • Views 31 views
  • Users 0 members are here
Related

SmartAssist EV - Mobile Robot Control and Live Video Streaming - Part 7

jelektro
jelektro 5 hours ago

As part of this project, we have developed a fully wireless control system for a wheeled robot integrated with a real-time live video stream. The heart of the entire setup is the Arduino Uno Q board.

Architecture and Technologies Used

The project leverages the unique, dual-processor architecture of the Arduino Uno Q and the Arduino App Lab ecosystem, which allows splitting system operations into two separate layers:

1. Application Layer (MPU Processor running Linux/Zephyr):

  • Responsible for directly handling the USB camera plugged into the board's port.
  • Runs a native server hosting the Web User Interface (WebUI).
  • Streams live video directly to any web browser in real-time, completely bypassing the need for heavy external libraries like Flask or OpenCV.

2. Hardware Layer (MCU STM32 Microcontroller):

  • Handles real-time, low-level execution of the digital H-bridge pins (four input pins controlling motor direction).
  • Generates diagnostic signals for the onboard peripherals of the Arduino Uno Q.

The engineered system seamlessly combines mechanical movement with advanced visual telemetry broadcasted directly on the robot itself:

1. Virtual Joystick (Local WebUI): The web interface is completely free of external network dependencies (no CDNs, no Cloudflare). The entire touch-control script is embedded directly into the index.html file within the assets folder. Touch drags on a smartphone screen or mouse movements on a PC are instantly calculated into directional vectors (F - forward, B - backward, L - left, R - right, S - stop).

2. Inter-Process Communication (RPC Bridge): To transmit data from the web interface (JavaScript/Python code) to the motors (C++ code), we used the official, built-in Arduino_RouterBridge communication bridge. It handles asynchronous requests, guaranteeing minimal latency and instantaneous drive response.

3. Color Status Signaling (RGB LED): The system integrates the native mapping of the onboard LED4, which operates in an inverted logic state (Active LOW). When moving forward, the LED lights up green; when reversing, it turns red; during turns, it lights up blue; and it automatically turns off when the robot is idle.

4. 8×13 Directional LED Matrix: The large LED dot matrix embedded into the Arduino Uno Q is used to visualize the robot's heading. Utilizing the low-level matrix.draw() function and raw pixel bitmaps (104-byte arrays defining pixel brightness on a 0–7 scale), the matrix renders crisp, bright arrows (▲, ▼, <, >) in real-time corresponding to the active drive direction.

By strictly utilizing the native libraries of the Arduino App Lab ecosystem (arduino.app_bricks.web_ui and Arduino_LED_Matrix), this solution boasts exceptional stability:It completely eliminates compilation errors related to memory pointer casting during array parsing.It ensures total offline independence – the robot operates entirely within a local Wi-Fi network.The codebase serves as a production-ready template for Smart Assist EV applications and autonomous inspection platforms.

To run the entire system, you need an external USB hub with a USB-C plug that connects to the Arduino Uno Q, featuring USB-C Power Delivery ports and USB 2.0 or USB 3.0 ports to connect a USB camera.

I tested several cameras. Unfortunately, some of them do not work properly with the Arduino Q. Despite my efforts, I could not get the rather old Megapixel USB2 Webcam Live WB-5400 - 15007 (which runs on USB 2.0) to work. The project uses a 3-in-1 Waterproof USB Endoscope Inspection Camera. The integrated ultra-bright LEDs and 1200P HD camera make it able to present images with a color effect while retaining high resolution and quality even in dark places. In addition, the lens is equipped with automatic exposure and blue light technology, which can effectively filter stray light, restore real scenes, and make images clearer. 

A web page containing a camera feed along with a joystick can be displayed on a computer or a mobile device. This offers tremendous possibilities for the user, allowing them to control the vehicle not only locally from an apartment but also remotely. The view of the page is shown in the image. After refreshing the page in your phone/computer browser: Below the video window, you will see a dark gray circle with a dashed light blue border. When you click or tap inside this circle, a light blue, movable joystick will immediately appear there, allowing you to control the vehicle.

For diagnostic purposes, the built-in LEDs on the Arduino Uno board were used to indicate the vehicle's status: a lit blue LED indicates turning left or right, green indicates moving forward, and red indicates moving backward. Additionally, the matrix display shows arrows indicating the direction in which the vehicle is moving.

WebApp


Here is the complete project, fully adapted to the native environment of Arduino App Lab, configured for the Elecrow car kit.

1. Microcontroller Code (sketch.ino)
Upload this code to the MCU section. It handles the H-bridge motors, the LED4 RGB status light, and renders the arrows using the native matrix.draw() function when an RPC command arrives from the webpage.
Below is the basic code that allows you to control the vehicle through a web application.
#include <Arduino_RouterBridge.h> // Official RPC header for Uno Q
#include <Arduino_LED_Matrix.h>   // Library for the 8x13 LED matrix

Arduino_LED_Matrix matrix; // Initialize the LED matrix

// Pin definitions for the L9110 motor drivers from the Elecrow kit
const int MOTOR_L_IA = 4; // Left motor - Input A (IA1)
const int MOTOR_L_IB = 5; // Left motor - Input B (IB1)
const int MOTOR_R_IA = 6; // Right motor - Input A (IA2)
const int MOTOR_R_IB = 7; // Right motor - Input B (IB2)

// Motor speed setting (0 - stopped, 255 - maximum speed)
// The L9110 bridge responds perfectly to PWM values between 180 and 220
const int MOTOR_SPEED = 200; 

const uint8_t FRAME_SIZE = 8 * 13; // 104 pixels

// Arrow definitions for the matrix (brightness values 0-7)
uint8_t arrow_up[FRAME_SIZE] = {
    0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 7, 7, 7, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 7, 7, 7, 7, 7, 0, 0, 0, 0,
    0, 0, 0, 7, 7, 0, 7, 0, 7, 7, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0
};

uint8_t arrow_down[FRAME_SIZE] = {
    0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 7, 7, 0, 7, 0, 7, 7, 0, 0, 0,
    0, 0, 0, 0, 7, 7, 7, 7, 7, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 7, 7, 7, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0
};

uint8_t arrow_left[FRAME_SIZE] = {
    0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 7, 7, 0, 0, 0, 0, 0, 0, 0, 0,
    0, 0, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7,
    0, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7,
    0, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7,
    0, 0, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7,
    0, 0, 0, 7, 7, 0, 0, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0, 0, 0
};

uint8_t arrow_right[FRAME_SIZE] = {
    0, 0, 0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 0, 0, 7, 7, 0, 0, 0,
    7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 0, 0,
    7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 0,
    7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 0,
    7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 0, 0,
    0, 0, 0, 0, 0, 0, 0, 0, 7, 7, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0
};

uint8_t stop_icon[FRAME_SIZE] = { 0 }; // All pixels off

// Control function for onboard LED4 (Active LOW: HIGH = OFF, LOW = ON)
void set_led4_color(bool r, bool g, bool b) {
  digitalWrite(LED4_R, r ? LOW : HIGH);
  digitalWrite(LED4_G, g ? LOW : HIGH);
  digitalWrite(LED4_B, b ? LOW : HIGH);
}

// Drive logic for the L9110 bridge using analogWrite (PWM) signals
void move_robot_mcu(String direction) {
  if (direction == "F") { // FORWARD (IA = PWM, IB = 0)
    analogWrite(MOTOR_L_IA, MOTOR_SPEED); analogWrite(MOTOR_L_IB, 0);
    analogWrite(MOTOR_R_IA, MOTOR_SPEED); analogWrite(MOTOR_R_IB, 0);
    
    set_led4_color(false, true, false); // Green
    matrix.draw(arrow_up);
  } 
  else if (direction == "B") { // BACKWARD (IA = 0, IB = PWM)
    analogWrite(MOTOR_L_IA, 0); analogWrite(MOTOR_L_IB, MOTOR_SPEED);
    analogWrite(MOTOR_R_IA, 0); analogWrite(MOTOR_R_IB, MOTOR_SPEED);
    
    set_led4_color(true, false, false); // Red
    matrix.draw(arrow_down);
  } 
  else if (direction == "L") { // LEFT (Pivot turn)
    analogWrite(MOTOR_L_IA, 0); analogWrite(MOTOR_L_IB, MOTOR_SPEED);
    analogWrite(MOTOR_R_IA, MOTOR_SPEED); analogWrite(MOTOR_R_IB, 0);
    
    set_led4_color(false, false, true); // Blue
    matrix.draw(arrow_left);
  } 
  else if (direction == "R") { // RIGHT (Pivot turn)
    analogWrite(MOTOR_L_IA, MOTOR_SPEED); analogWrite(MOTOR_L_IB, 0);
    analogWrite(MOTOR_R_IA, 0); analogWrite(MOTOR_R_IB, MOTOR_SPEED);
    
    set_led4_color(false, false, true); // Blue
    matrix.draw(arrow_right);
  } 
  else { // STOP ("S") (IA = 0, IB = 0)
    analogWrite(MOTOR_L_IA, 0); analogWrite(MOTOR_L_IB, 0);
    analogWrite(MOTOR_R_IA, 0); analogWrite(MOTOR_R_IB, 0);
    
    set_led4_color(false, false, false); // Off
    matrix.draw(stop_icon);
  }
}

void setup() {
  // Initialize onboard LED4 pins
  pinMode(LED4_R, OUTPUT); pinMode(LED4_G, OUTPUT); pinMode(LED4_B, OUTPUT);
  set_led4_color(false, false, false);

  // Initialize LED matrix
  matrix.begin();
  matrix.setGrayscaleBits(3); 
  matrix.clear();

  // Initialize official RPC communication bridge with Linux (main.py)
  Bridge.begin();
  Bridge.provide("drive", move_robot_mcu); 
  
  // Configuring motor driver pins as outputs is not working correctly – this is a known issue in the underlying Zephyr core
  // pinMode(MOTOR_L_IA, OUTPUT); pinMode(MOTOR_L_IB, OUTPUT);
  // pinMode(MOTOR_R_IA, OUTPUT); pinMode(MOTOR_R_IB, OUTPUT);
  
  move_robot_mcu("S"); // Load initial state (STOP)
}

void loop() {
  delay(10); // RPC runs on background threads automatically
}

After adding sensors, the code that additionally implements obstacle detection is as follows:

#include <Arduino.h>
#include <IRremote.hpp> 
#include <Arduino_RouterBridge.h> // Official RPC header for Uno Q
#include <Arduino_LED_Matrix.h>   // Library for the 8x13 LED matrix

Arduino_LED_Matrix matrix; // Initialize the large LED matrix

// --- HARDWARE PIN CONFIGURATION ---
const int motorPinA_1A = 5; 
const int motorPinA_1B = 6; 
const int motorPinB_1A = 9; 
const int motorPinB_1B = 10;

const int IR_RECEIVE_PIN = 2; 

const int trigPin = 4;      
const int echoPin = 3;      

// Physical Tactile Bumpers
const int CollisionPin_L = 18; // Pin A4
const int CollisionPin_R = 19; // Pin A5

// Line Tracking Photo-interrupters
const int TrackingPin_L = A2;
const int TrackingPin_R = A3;

// --- ROBOT PARAMETERS AND MODES ---
enum RobotMode { MODE_MANUAL, MODE_AUTONOMOUS };
RobotMode currentMode = MODE_MANUAL; 

const int motorSpeed = 200; 
int spd = 195;              // Specific heavy escape/evasive speed vector
int trackSpeed = 200;       

unsigned long lastCommandTime = 0;
const unsigned long commandTimeout = 250;

const uint8_t FRAME_SIZE = 8 * 13; // 104 pixels for the LED matrix

// --- LED MATRIX ARROW ARRAYS (Brightness levels 0-7) ---
uint8_t arrow_up[FRAME_SIZE] = {
    0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 7, 7, 7, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 7, 7, 7, 7, 7, 0, 0, 0, 0,
    0, 0, 0, 7, 7, 0, 7, 0, 7, 7, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0
};

uint8_t arrow_down[FRAME_SIZE] = {
    0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 7, 7, 0, 7, 0, 7, 7, 0, 0, 0,
    0, 0, 0, 0, 7, 7, 7, 7, 7, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 7, 7, 7, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0
};

uint8_t arrow_left[FRAME_SIZE] = {
    0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 7, 7, 0, 0, 0, 0, 0, 0, 0, 0,
    0, 0, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7,
    0, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7,
    0, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7,
    0, 0, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7,
    0, 0, 0, 7, 7, 0, 0, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0, 0, 0
};

uint8_t arrow_right[FRAME_SIZE] = {
    0, 0, 0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 0, 0, 7, 7, 0, 0, 0,
    7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 0, 0,
    7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 0,
    7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 0,
    7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 0, 0,
    0, 0, 0, 0, 0, 0, 0, 0, 7, 7, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0
};

uint8_t stop_icon[FRAME_SIZE] = { 0 }; 

// --- ONBOARD RGB LED CONTROL FUNCTIONS ---
void set_led3_color(int r, int g, int b) {
  analogWrite(LED3_R, r);
  analogWrite(LED3_G, g);
  analogWrite(LED3_B, b);
}

void set_led4_color(bool r, bool g, bool b) {
  digitalWrite(LED4_R, r ? LOW : HIGH);
  digitalWrite(LED4_G, g ? LOW : HIGH);
  digitalWrite(LED4_B, b ? LOW : HIGH);
}

// --- MOTOR DRIVE DRIVER ---
void motor(int A1, int A2, int B1, int B2) {
  analogWrite(motorPinA_1A, A1);
  analogWrite(motorPinA_1B, A2);
  analogWrite(motorPinB_1A, B1);
  analogWrite(motorPinB_1B, B2);
}

// --- WEB INTERFACE INTERCEPTOR (Fixed String parsing bug to const char*) ---
void move_robot_mcu(const char* direction_ptr) {
  if (currentMode != MODE_MANUAL) return; 

  String direction = String(direction_ptr);

  if (direction == "F") {      
    motor(0, motorSpeed, 0, motorSpeed);
    set_led4_color(false, true, false);  
    set_led3_color(0, 200, 0);           
    matrix.draw(arrow_up);
  } 
  else if (direction == "B") { 
    motor(motorSpeed, 0, motorSpeed, 0);
    set_led4_color(true, false, false);  
    set_led3_color(200, 0, 0);           
    matrix.draw(arrow_down);
  } 
  else if (direction == "L") { 
    motor(motorSpeed, 0, 0, motorSpeed);
    set_led4_color(false, false, true);  
    set_led3_color(0, 0, 200);           
    matrix.draw(arrow_left);
  } 
  else if (direction == "R") { 
    motor(0, motorSpeed, motorSpeed, 0);
    set_led4_color(false, false, true);  
    set_led3_color(0, 0, 200);           
    matrix.draw(arrow_right);
  } 
  else { 
    motor(0, 0, 0, 0);
    set_led4_color(false, false, false); 
    set_led3_color(0, 0, 0);
    matrix.draw(stop_icon);
  }
}

float getDistance() {
  digitalWrite(trigPin, LOW);
  delayMicroseconds(2);
  digitalWrite(trigPin, HIGH);
  delayMicroseconds(10);
  digitalWrite(trigPin, LOW);
  
  long duration = pulseIn(echoPin, HIGH, 30000); 
  float d = duration * 0.0343 / 2;
  if (d == 0) return 999.0; 
  return d;
}

void toggle_mode_from_web(const char* cmd_ptr) {
  if (currentMode == MODE_MANUAL) {
    currentMode = MODE_AUTONOMOUS;
    set_led3_color(200, 200, 0); 
    Serial.println("System Notification: AUTONOMY MODE ENGAGED VIA WEB.");
  } else {
    currentMode = MODE_MANUAL;
    motor(0, 0, 0, 0); 
    set_led3_color(0, 0, 0);
    matrix.draw(stop_icon);
    Serial.println("System Notification: MANUAL OVERRIDE ENGAGED VIA WEB.");
  }
}

void exec_cmd(byte key_val) {
  switch (key_val) {
    case 0x1C: 
      if (currentMode == MODE_MANUAL) {
        currentMode = MODE_AUTONOMOUS;
        set_led3_color(200, 200, 0);
        Serial.println("System Notification: AUTONOMY MODE ENGAGED.");
      } else {
        currentMode = MODE_MANUAL;
        motor(0, 0, 0, 0); 
        set_led3_color(0, 0, 0);
        matrix.draw(stop_icon);
        Serial.println("System Notification: MANUAL OVERRIDE ENGAGED.");
      }
      break;

    case 0x18: if (currentMode == MODE_MANUAL) { motor(0, motorSpeed, 0, motorSpeed); matrix.draw(arrow_up); } break;
    case 0x08: if (currentMode == MANUAL_MODE_COMPAT) { motor(motorSpeed, 0, 0, motorSpeed); matrix.draw(arrow_left); } // Fixed fallback if needed
               if (currentMode == MODE_MANUAL) { motor(motorSpeed, 0, 0, motorSpeed); matrix.draw(arrow_left); } break;
    case 0x5A: if (currentMode == MODE_MANUAL) { motor(0, motorSpeed, motorSpeed, 0); matrix.draw(arrow_right); } break;
    case 0x52: if (currentMode == MODE_MANUAL) { motor(motorSpeed, 0, motorSpeed, 0); matrix.draw(arrow_down); } break;
    default:   if (currentMode == MODE_MANUAL) { motor(0, 0, 0, 0); matrix.draw(stop_icon); } break;
  }
}

void setup() {
  Serial.begin(9600);
  

// Configuring motor driver pins as outputs is not working correctly – this is a known issue in the underlying Zephyr core // pinMode(motorPinA_1A, OUTPUT); // pinMode(motorPinA_1B, OUTPUT); // pinMode(motorPinB_1A, OUTPUT); // pinMode(motorPinB_1B, OUTPUT); IrReceiver.begin(IR_RECEIVE_PIN, ENABLE_LED_FEEDBACK);
  pinMode(trigPin, OUTPUT);
  pinMode(echoPin, INPUT);

  pinMode(CollisionPin_L, INPUT_PULLUP);
  pinMode(CollisionPin_R, INPUT_PULLUP);
  pinMode(TrackingPin_L, INPUT_PULLUP);
  pinMode(TrackingPin_R, INPUT_PULLUP);
  
  pinMode(LED4_R, OUTPUT); pinMode(LED4_G, OUTPUT); pinMode(LED4_B, OUTPUT);
  set_led3_color(0, 0, 0);
  set_led4_color(false, false, false);

  matrix.begin();
  matrix.setGrayscaleBits(3);
  matrix.clear();

  // --- RPC BRIDGE REGISTRATION ---
  Bridge.begin();
  Bridge.provide("drive", move_robot_mcu); 
  Bridge.provide("toggle_mode", toggle_mode_from_web);
  
  Serial.println("System Core Ready.");
}

void loop() {
  if (IrReceiver.decode()) {
    byte command = IrReceiver.decodedIRData.command;
    exec_cmd(command);
    if (currentMode == MODE_MANUAL) lastCommandTime = millis();
    IrReceiver.resume(); 
  }

  if (currentMode == MODE_AUTONOMOUS) {
    float distance = getDistance();
    int sumVal = (digitalRead(CollisionPin_L) == LOW ? 1 : 0) * 2 + (digitalRead(CollisionPin_R) == LOW ? 1 : 0);
    
    if (sumVal > 0) {
      Serial.print("TACTILE COLLISION TRIGGERED. Code: ");
      Serial.println(sumVal);
      matrix.draw(stop_icon);
      
      if (sumVal == 1) {      
        motor(spd, 0, spd, 0); delay(2000);
        motor(spd, 0, 0, spd); delay(2000);
      }
      else if (sumVal == 2) { 
        motor(spd, 0, spd, 0); delay(2000);
        motor(0, spd, spd, 0); delay(2000);
      }
      else if (sumVal == 3) { 
        motor(spd, 0, spd, 0); delay(2000);
        motor(spd, 0, 0, spd); delay(2000);
      }
    }
    else if (distance < 30) {
      matrix.draw(arrow_left);
      motor(motorSpeed, 0, 0, motorSpeed); 
      delay(300);
    }
    else {
      int trackL = digitalRead(TrackingPin_L);
      int trackR = digitalRead(TrackingPin_R);
      int Track = trackL * 2 + trackR;
      
      if (Track == 0)      { motor(0, 0, 0, 0); matrix.draw(stop_icon); }
      else if (Track == 1) { motor(0, trackSpeed, trackSpeed, 0); matrix.draw(arrow_right); }
      else if (Track == 2) { motor(trackSpeed, 0, 0, trackSpeed); matrix.draw(arrow_left); }
      else if (Track == 3) { motor(0, trackSpeed, 0, trackSpeed); matrix.draw(arrow_up); }
      delay(10);
    }
  }
  Bridge.update(); 
}

2. Python Web Server (main.py)

Save this to the MPU/Python editor. It initializes your USB camera, acts as the API backend for your local interface touch events, and redirects commands to the MCU code.
# SPDX-FileCopyrightText: Copyright (C) Arduino s.r.l. and/or its affiliated companies
# SPDX-License-Identifier: MPL-2.0

from arduino.app_utils import *
from arduino.app_bricks.web_ui import WebUI
from arduino.app_peripherals.camera import Camera

# Initialize the onboard USB camera module
camera = Camera(resolution=(640, 480), fps=15)  
ui = WebUI()
camera.start()

# API endpoint executed when a drag action occurs on the webpage
def handle_joystick(data: dict):
    char_cmd = data.get('cmd', 'S')
    # Forward command directly to the registered C++ function
    Bridge.call("drive", char_cmd)
    return {"status": "ok"}

# Expose API and camera routes to App Lab web interface
ui.expose_api("POST", "/api/move", handle_joystick)
ui.expose_camera("/camera", camera)

App.run()

3. Local Dashboard UI (assets/index.html)

Save this code to your index.html inside your project's assets directory. It renders a clean grid, displays your camera stream, and tracks touch drags or mouse movements inside a virtual joystick container without needing internet-based libraries.
<!doctype html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Uno Q Robot Dashboard</title>
    <style>
      body { font-family: Arial, sans-serif; text-align: center; background: #111; color: #fff; margin: 0; padding: 10px; }
      #control-pad { width: 220px; height: 220px; background: #222; border-radius: 50%; margin: 25px auto; position: relative; border: 3px dashed #00ffcc; touch-action: none; }
      img { border: 4px solid #00ffcc; border-radius: 12px; max-width: 100%; height: auto; display: block; margin: 0 auto; }
      .status { font-weight: bold; color: #00ffcc; font-size: 1.2em; }
    </style>
  </head>
  <body>
    <h1>Uno Q Vehicle Control System</h1>
    <div>
      <img src="/camera" />
    </div>
    
    <div id="control-pad"></div>
    <p>Drag your finger or mouse inside the circle to move the robot</p>
    <p>Status: <span id="status-val" class="status">STOP</span></p>

    <script>
      const pad = document.getElementById('control-pad');
      const statusText = document.getElementById('status-val');
      let isMoving = false;
      let lastDirection = "S";
      let lastSend = 0;

      // Create a visual handle inside the control pad area
      const knob = document.createElement('div');
      knob.style.width = '60px'; knob.style.height = '60px'; knob.style.background = '#00ffcc';
      knob.style.borderRadius = '50%'; knob.style.position = 'absolute';
      knob.style.left = '80px'; knob.style.top = '80px'; knob.style.pointerEvents = 'none';
      pad.appendChild(knob);

      function startAction() { isMoving = true; }

      function moveAction(e) {
        if (!isMoving) return;
        e.preventDefault();
        
        const rect = pad.getBoundingClientRect();
        const input = e.touches ? e.touches[0] : e;
        
        const centerX = rect.width / 2;
        const centerY = rect.height / 2;
        
        let x = input.clientX - rect.left - centerX;
        let y = input.clientY - rect.top - centerY;
        
        const distance = Math.sqrt(x*x + y*y);
        if (distance > 80) {
          x = (x / distance) * 80;
          y = (y / distance) * 80;
        }
        
        knob.style.transform = `translate(${x}px, ${y}px)`;

        if (Date.now() - lastSend < 70) return;
        
        let cmd = "S";
        if (Math.abs(x) > Math.abs(y)) {
          cmd = x > 25 ? "R" : (x < -25 ? "L" : "S");
        } else {
          cmd = y > 25 ? "B" : (y < -25 ? "F" : "S");
        }

        if (cmd !== lastDirection) {
          lastDirection = cmd;
          lastSend = Date.now();
          
          const names = {"F": "FORWARD ▲", "B": "BACKWARD ▼", "L": "LEFT Arrow backward", "R": "RIGHT Arrow forward", "S": "STOP"};
          statusText.innerText = names[cmd];
          
          sendCmd(cmd);
        }
      }

      function endAction() {
        isMoving = false;
        knob.style.transform = 'translate(0px, 0px)';
        if (lastDirection !== "S") {
          lastDirection = "S";
          statusText.innerText = "STOP";
          sendCmd("S");
        }
      }

      pad.addEventListener('mousedown', startAction);
      window.addEventListener('mousemove', moveAction);
      window.addEventListener('mouseup', endAction);

      pad.addEventListener('touchstart', startAction);
      window.addEventListener('touchmove', moveAction, { passive: false });
      window.addEventListener('touchend', endAction);

      function sendCmd(character) {
        fetch('/api/move', {
          method: 'POST',
          headers: { 'Content-Type': 'application/json' },
          body: JSON.stringify({ cmd: character })
        }).catch(err => console.log(err));
      }
    </script>
  </body>
</html>


  • Sign in to reply
  • Cancel
element14 Community

element14 is the first online community specifically for engineers. Connect with your peers and get expert answers to your questions.

  • Members
  • Learn
  • Technologies
  • Challenges & Projects
  • Products
  • Store
  • About Us
  • Feedback & Support
  • FAQs
  • Terms of Use
  • Privacy Policy
  • Legal and Copyright Notices
  • Sitemap
  • Cookies

An Avnet Company © 2026 Premier Farnell Limited. All Rights Reserved.

Premier Farnell Ltd, registered in England and Wales (no 00876412), registered office: Farnell House, Forge Lane, Leeds LS12 2NE.

Follow element14

  • X
  • Facebook
  • linkedin
  • YouTube