A 4DOF Keyestudio KS0198X robotic arm trained in simulation using reinforcement learning to reach a target, and transferred via serial to the physical arm. Task: reach — moving the tip of the arm (tool) towards a red dot generated within the arm's real workspace, using PPO (Proximal Policy Optimization, Stable-Baselines3) on a PyBullet model built from the real URDF/meshes of the mechanism.
Devices & Components
Arduino Uno Rev3
4DOF Robot Arm Kit
Mearm_real_env.py
Gymnasium environment for the MeArm/Keyestudio KS0198X robotic arm, simulated in PyBullet from the real URDF (URDF and mashes available on GitHub).
Neural network (view code on GitHub):
An input layer with 17 neurons (one for each observation value: 4 (positions) + 4 (velocities) + 3 (ee_pos) + 3 (target_pos) + 3 (delta_pos))
An output layer with 4 neurons (one for each motor).
Serial_bridge.py
Sim-to-real bridge: reads a trained policy and sends the corresponding servo angles to the physical Keyestudio arm via serial.
Train.py
PPO training (with linear reward) of the 'reach' task on the real MeArm model.Run on an M1 for 1M steps across 4 parallel environments (you can use even more steps if necessary for your specific case). PPO takes the data from all 4, averages it, and updates the weights much more stably.To monitor training progress with TensorBoard, open the terminal:tensorboard --logdir ./tb_logs/"
Evaluate.py
Loads the trained model and shows it in action within the PyBullet GUI.
CODE
Top Comments