RaspberryPi car with FPV camera control by a smartphone [Out Of the box Project] -

Uncategorized Maciej 23 March 2019

RaspberryPi car with FPV camera control by a smartphone [Out Of the box Project]

About tutorial

In this article I will demonstrate how to make a robot car controlled using web browser. It will also stream a video feed using a camera connected to the car. You don’t have to be an expert to make this robot. All the required code is already available on RemoteMe.

At the bottom of the post, the principle of the car’s operation is described in more detail.

The Tutorial

In a previous tutorial I showed how to control the position of the webcam and capture the image on the browser link and here control the camera. Now we will expand this project and build a car with 4-wheel drive (controlled like a tank – differential steering ). Check out the video tutorial for more details of all the steps.

Delays

local network 140ms, outside local network (with turn server) 170ms

Parts

Connections

Raspberry Pi controls the servos via the PWM module and the driver by setting the pins on the H-bridge and providing PWM signal from the same module that sends the signal to the servos. (Thanks to the module for generating the PWM signal, we do not have to generate these signals by Raspberry Pi itself – just by I2C we send the appropriate instructions to the servo driver (through the python library provided by adafruit), and this generates the appropriate PWM signal for the servo mechanisms and bridge H)

Schematic:

Via the I2C bus RPi controls the servo controller, the two last controller’s outputs are connected to the H-bridge, and the PWM signal is used to set the speed of the motors. The RPi-pins are also connected to the H-bridge. The states on the appropriate pins will determine the direction of rotation of the motors. – I refer to the TB6612FNG documentation for details. You might as well use the L298N bridge or similar.

I have designed a PCB, which you can use eagle, gerber etc. here pcb.zip. The circuit diagram coincides with the diagram above

PCB:

PWM input to drive motors should be connected to PWM module 15th and 16th pins

Power input for drive motors (check what maximum voltage your motors can work with)
Power for Raspberry Pi and PWM module – have to be exactly 5V
Drive motor outputs (there are two outputs because we will pair drive motors )
Servo Mechanisms power – most 9G servos need 6v to work properly though make sure you refer the datasheet of your servos
Jumper its present power from 5th input will also power motor drives, so no need to connect anything to input 2
Jumper if present Raspberry Pi will be power from 3rd input – before adding a jumper here check if connections are OK – You don’t want to burn your RPi
Solder a jumper here since we don’t need logic converters

Of course, you still need the right power supply, I connect the 6v to the drive and servo motors, and 5v for Raspberry Pi and controllers.

below are some illustrative photos of the whole car:

I used plastic spacers to connect the tiles to the car.

Programming

Before creating the project, enable the camera and I2C communications using the raspi-config described here

This project is one of the ready projects that you can easily implement with a few clicks:

Open project

Go to the “Build It” tab and follow the steps, at the end click on “Build Project” and you will receive:

After clicking “Get QR” you will see a QR code that you can scan with your smartphone, or simply open the page with the “Open” button in the browser on your computer.

I do not advise changing the position of the camera so long as you do not set the servo position – in some cases you can damage your servos

The car should drive like in the video above – it can happen that it turns in the wrong direction or the servos are badly calibrated, below in the discussion of the code I wrote how to fix it.

Discussing what happened

As you noticed, creating the project was highly automated. Therefore, I will now discuss what exactly happened and how to modify your project.

First of all, two variables were created (variables tab)

Variable cameraPos transmits camera positions and joystick positions. Both are the type “Small int x2” which means that the value of a variable is two integer numbers.

The website changes our variables, and the Python script records these changes and reacts accordingly (as we will open any variables, we will see that the device that listens for changes is the Python script). Let’s see what the Python code looks like (more about variables here)

Python

The Python script was automatically uploaded. Of course, we can preview and modify it (to create a new Python script eg for other projects see here). With the information you need now, the Python script is another device in remoteMe is managed by a script (run by ./runme.sh), we can send messages to this device, the device also reacts to changes in variables that it observes and can change these variables .

To open the python script, click python.py and select Edit:

It looks as follows. below I will discuss interesting fragments

import logging
import socket
import math
import struct
import sys
import os

os.chdir(sys.argv[1])
sys.path.append('../base')

import remoteme

import Adafruit_PCA9685
import time
import RPi.GPIO as GPIO

motorAIn1 = 25  # GPIO25
motorAIn2 = 8  # GPIO8

motorBIn1 = 24  # 24
motorBIn2 = 23  # 23

motors = [[motorAIn1, motorAIn2], [motorBIn1, motorBIn2]]

motorsPWM = [14, 15]
pwm = None;

def motorForward(motorId):
    GPIO.output(motors[motorId][0], GPIO.LOW)
    GPIO.output(motors[motorId][1], GPIO.HIGH)


def motorBackward(motorId):
    GPIO.output(motors[motorId][0], GPIO.HIGH)
    GPIO.output(motors[motorId][1], GPIO.LOW)


def motorSoftStop(motorId):
    GPIO.output(motors[motorId][0], GPIO.LOW)
    GPIO.output(motors[motorId][1], GPIO.LOW)
	
def setMotor(motorId, speed):
    if speed == 0:
        motorSoftStop(motorId)
    elif speed > 0:
        motorForward(motorId)
    elif speed < 0:
        motorBackward(motorId)
        speed=-speed

    logger.info("set speed {} for motor {} ".format(speed,motorId))
    pwm.set_pwm(motorsPWM[motorId], 0, int(speed))





def onCameraPosChange(i1, i2):
    global pwm
    logger.info("on camera change {} , {}".format(i1, i2))
    pwm.set_pwm(1, 0, i1)
    pwm.set_pwm(0, 0, i2)
    pass

def onDriveChange(x, y):
    logger.info("on drive change  x {} , y {}".format(x, y))
    global pwm
    
    left=y
    right=y
    
    left+=x
    right-=x
    
    delta=(left+right)/2
    
    
    left+=delta
    right+=delta
    
    # when your car doesnt drive as suppose try to swich right and left variable below
	# or remove add minuses next to 2
	# another way is to switch cables conencted to motors
    setMotor(0, 2*left)
    setMotor(1, 2*right)
    pass


def setupPWM():
    global pwm
    pwm = Adafruit_PCA9685.PCA9685()
    pwm.set_pwm_freq(80)

def setupPins():
    global GPIO
    GPIO.setmode(GPIO.BCM)  # Broadcom pin-numbering scheme

    for motor in motors:
        for pinId in motor:
            GPIO.setup(pinId, GPIO.OUT)
    
try:

    logging.basicConfig(level=logging.DEBUG,
    format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
    datefmt='%d.%m %H:%M',
    filename="logs.log")

    logger = logging.getLogger('application')
    setupPWM()
    setupPins()
    remoteMe = remoteme.RemoteMe()

    remoteMe.startRemoteMe(sys.argv)
    remoteMe.getVariables().observeSmallInteger2("cameraPos" ,onCameraPosChange);
    remoteMe.getVariables().observeSmallInteger2("drive" ,onDriveChange);
    remoteMe.wait()

finally:
    pass

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

import logging

import socket

import math

import struct

import sys

import os

os.chdir(sys.argv[1])

sys.path.append('../base')

import remoteme

import Adafruit_PCA9685

import time

import RPi.GPIO as GPIO

motorAIn1 = 25 # GPIO25

motorAIn2 = 8 # GPIO8

motorBIn1 = 24 # 24

motorBIn2 = 23 # 23

motors = [[motorAIn1, motorAIn2], [motorBIn1, motorBIn2]]

motorsPWM = [14, 15]

pwm = None;

def motorForward(motorId):

GPIO.output(motors[motorId][0], GPIO.LOW)

GPIO.output(motors[motorId][1], GPIO.HIGH)

def motorBackward(motorId):

GPIO.output(motors[motorId][0], GPIO.HIGH)

GPIO.output(motors[motorId][1], GPIO.LOW)

def motorSoftStop(motorId):

GPIO.output(motors[motorId][0], GPIO.LOW)

GPIO.output(motors[motorId][1], GPIO.LOW)

def setMotor(motorId, speed):

if speed == 0:

motorSoftStop(motorId)

elif speed > 0:

motorForward(motorId)

elif speed < 0:

motorBackward(motorId)

speed=-speed

logger.info("set speed {} for motor {} ".format(speed,motorId))

pwm.set_pwm(motorsPWM[motorId], 0, int(speed))

def onCameraPosChange(i1, i2):

global pwm

logger.info("on camera change {} , {}".format(i1, i2))

pwm.set_pwm(1, 0, i1)

pwm.set_pwm(0, 0, i2)

pass

def onDriveChange(x, y):

logger.info("on drive change x {} , y {}".format(x, y))

global pwm

left=y

right=y

left+=x

right-=x

delta=(left+right)/2

left+=delta

right+=delta

# when your car doesnt drive as suppose try to swich right and left variable below

# or remove add minuses next to 2

# another way is to switch cables conencted to motors

setMotor(0, 2*left)

setMotor(1, 2*right)

pass

def setupPWM():

global pwm

pwm = Adafruit_PCA9685.PCA9685()

pwm.set_pwm_freq(80)

def setupPins():

global GPIO

GPIO.setmode(GPIO.BCM) # Broadcom pin-numbering scheme

for motor in motors:

for pinId in motor:

GPIO.setup(pinId, GPIO.OUT)

try:

logging.basicConfig(level=logging.DEBUG,

format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',

datefmt='%d.%m %H:%M',

filename="logs.log")

logger = logging.getLogger('application')

setupPWM()

setupPins()

remoteMe = remoteme.RemoteMe()

remoteMe.startRemoteMe(sys.argv)

remoteMe.getVariables().observeSmallInteger2("cameraPos" ,onCameraPosChange);

remoteMe.getVariables().observeSmallInteger2("drive" ,onDriveChange);

remoteMe.wait()

finally:

pass

Motor control:

def motorForward(motorId):
    GPIO.output(motors[motorId][0], GPIO.LOW)
    GPIO.output(motors[motorId][1], GPIO.HIGH)


def motorBackward(motorId):
    GPIO.output(motors[motorId][0], GPIO.HIGH)
    GPIO.output(motors[motorId][1], GPIO.LOW)


def motorSoftStop(motorId):
    GPIO.output(motors[motorId][0], GPIO.LOW)
    GPIO.output(motors[motorId][1], GPIO.LOW)
	
def setMotor(motorId, speed):
    if speed == 0:
        motorSoftStop(motorId)
    elif speed > 0:
        motorForward(motorId)
    elif speed < 0:
        motorBackward(motorId)
        speed=-speed

    logger.info("set speed {} for motor {} ".format(speed,motorId))
    pwm.set_pwm(motorsPWM[motorId], 0, int(speed))

def motorForward(motorId):

GPIO.output(motors[motorId][0], GPIO.LOW)

GPIO.output(motors[motorId][1], GPIO.HIGH)

def motorBackward(motorId):

GPIO.output(motors[motorId][0], GPIO.HIGH)

GPIO.output(motors[motorId][1], GPIO.LOW)

def motorSoftStop(motorId):

GPIO.output(motors[motorId][0], GPIO.LOW)

GPIO.output(motors[motorId][1], GPIO.LOW)

def setMotor(motorId, speed):

if speed == 0:

motorSoftStop(motorId)

elif speed > 0:

motorForward(motorId)

elif speed < 0:

motorBackward(motorId)

speed=-speed

logger.info("set speed {} for motor {} ".format(speed,motorId))

pwm.set_pwm(motorsPWM[motorId], 0, int(speed))

the setMotor function for motorId 1 or 2 sets the speed (can be negative). Just on the appropriate pins we set the states accordingly (forward and backward movement), and then depending on the speed, set the PWM filling accordingly using the servo controller.

def onCameraPosChange(x, y):
    global pwm
    logger.info("on camera change {} , {}".format(x, y))
    pwm.set_pwm(1, 0, x)
    pwm.set_pwm(0, 0, y)
    pass

def onCameraPosChange(x, y):

global pwm

logger.info("on camera change {} , {}".format(x, y))

pwm.set_pwm(1, 0, x)

pwm.set_pwm(0, 0, y)

pass

so far, we will not discuss it, because earlier we have to change the range of servo traffic.

def onDriveChange(x, y):
    logger.info("on drive change  x {} , y {}".format(x, y))
    global pwm
    
    left=y
    right=y
    
    left+=x
    right-=x
    
    delta=(left+right)/2
    
    left+=delta
    right+=delta
    
    # when your car doesnt drive as suppose try to swich right and left variable below
	# or remove add minuses next to 2
	# another way is to switch cables conencted to motors
    setMotor(0, 2*left)
    setMotor(1, 2*right)
    pass

def onDriveChange(x, y):

logger.info("on drive change x {} , y {}".format(x, y))

global pwm

left=y

right=y

left+=x

right-=x

delta=(left+right)/2

left+=delta

right+=delta

# when your car doesnt drive as suppose try to swich right and left variable below

# or remove add minuses next to 2

# another way is to switch cables conencted to motors

setMotor(0, 2*left)

setMotor(1, 2*right)

pass

this function will be called as the variable drive changes – eg after changing the joystick on the page. x, y is just a joystick’s coordinates Based on these variables, we calculate the speed of the left and right side of the car.

If the car turns, instead of going forward, it goes back instead of going forward:

setMotor(0, Y2*left)

setMotor(1, X2*right)

give the cons in different combinations (in X and/or Y place ) until the car moves forward – when the joystick is tilted up.

Then, if the right and left sides are swapped, in the above function, swap left and right places.

def setupPWM():
    global pwm
    pwm = Adafruit_PCA9685.PCA9685()
    pwm.set_pwm_freq(80)

def setupPins():
    global GPIO
    GPIO.setmode(GPIO.BCM)  # Broadcom pin-numbering scheme

    for motor in motors:
        for pinId in motor:
            GPIO.setup(pinId, GPIO.OUT)

def setupPWM():

global pwm

pwm = Adafruit_PCA9685.PCA9685()

pwm.set_pwm_freq(80)

def setupPins():

global GPIO

GPIO.setmode(GPIO.BCM) # Broadcom pin-numbering scheme

for motor in motors:

for pinId in motor:

GPIO.setup(pinId, GPIO.OUT)

We simply set the appropriate pins (these to control the H bridge) on the outputs. And we create an object to control the servos control. (Note that the server driver should work properly, we must enable I2C communications using raspi-config more here)

remoteMe.startRemoteMe(sys.argv)
remoteMe.getVariables().observeSmallInteger2("cameraPos" ,onCameraPosChange);
remoteMe.getVariables().observeSmallInteger2("drive" ,onDriveChange);
remoteMe.wait()

remoteMe.startRemoteMe(sys.argv)

remoteMe.getVariables().observeSmallInteger2("cameraPos" ,onCameraPosChange);

remoteMe.getVariables().observeSmallInteger2("drive" ,onDriveChange);

remoteMe.wait()

RemoteMe configuration and setting which functions are to be called when the variables for camera and drive control are changed.

That is all in the Python script , as you can see, it is not too complicated, for more information go here

WebPage

In fact, two web pages have been created – one for calibration, the other – a page for controlling the car and displaying the image from the webcam. Open the calibration page:

A web page opens with two sliders. Set the top and bottom to the middle position – the camera will move – the upper slider should move the camera in the x-axis, the lower y-axis. If it is not the case, swap the servo signal wires.

Then, using the sliders, set the maximum deflection of the x-axis and the y-axis, in my case:

x: 298 – 830 and the central position. it’s important that the central position is exactly between the min and max values. At my case it is ((298 + 830) / 2 = 564)
y: 223 – 723 and similarly the central position of the camera in the y axis should fall in the middle the two values

Let’s write numbers somewhere in the notepad and open the page to control the car for editing;

<camera autoConnect="true" showInfo="true" class="cameraView"></camera>
<connectionstatus webSocket="true" directConnection="false" camera="true"></connectionstatus>
<variable component="cameraMouseTrack" type="SMALL_INTEGER_2" style="display:block" name="cameraPos" xMin="298"
          xMax="830" invertX="true" yMin="223" yMax="723" invertY="true" requiredMouseDown="true"
          reset="true" onlyDirect="true"></variable>

<div class="joystickButtons">
    <div class="buttons">
        <variable class="gyroscope" component="gyroscope" type="SMALL_INTEGER_2" name="cameraPos"
                  label="Gyroscope Camera" orientationSupport="true"
                  xMin="298" xMax="830" xRange="19" invertX="true" yMin="223" yMax="723" yRange="20"
                  invertY="false" onlyDirect="true"></variable>
        <variable class="gyroscope" component="gyroscope" type="SMALL_INTEGER_2" name="drive"
                  label="Gyroscope Drive"
                  xMin="-512" xMax="512" xRange="19" invertX="false" yMin="-512" yMax="512" yRange="20"
                  invertY="false" onlyDirect="true"></variable>
        <button class="mdl-button mdl-js-button mdl-button--raised mdl-js-ripple-effect gyroscope" onClick="toggleFullScreen()">fullscreen	</button>

    </div>
    <div class="joystickParent">
        <variable class="joystick" component="joystick_simple" type="SMALL_INTEGER_2" name="drive"
                  xRange="1024" yRange="1024" onlyDirect="true"></variable>
    </div>
    <div style="clear: both;"/>
</div>

<variable component="cameraMouseTrack" type="SMALL_INTEGER_2" style="display:block" name="cameraPos" xMin="298"

xMax="830" invertX="true" yMin="223" yMax="723" invertY="true" requiredMouseDown="true"

reset="true" onlyDirect="true"></variable>

<variable class="gyroscope" component="gyroscope" type="SMALL_INTEGER_2" name="cameraPos"

label="Gyroscope Camera" orientationSupport="true"

xMin="298" xMax="830" xRange="19" invertX="true" yMin="223" yMax="723" yRange="20"

invertY="false" onlyDirect="true"></variable>

<variable class="gyroscope" component="gyroscope" type="SMALL_INTEGER_2" name="drive"

label="Gyroscope Drive"

xMin="-512" xMax="512" xRange="19" invertX="false" yMin="-512" yMax="512" yRange="20"

invertY="false" onlyDirect="true"></variable>

<button class="mdl-button mdl-js-button mdl-button--raised mdl-js-ripple-effect gyroscope" onClick="toggleFullScreen()">fullscreen </button>

</div>

<variable class="joystick" component="joystick_simple" type="SMALL_INTEGER_2" name="drive"

xRange="1024" yRange="1024" onlyDirect="true"></variable>

</div>

</div>

These are automatic components for controlling the car and displaying video.

The page you have opened already contains the components you need to control your car. The only thing you need to do is change the range of motion for servo mechanisms so replace the values of xMin, xMax, yMin, yMax, values you received on the previous page with the sliders.

If you want to create your own website with your components, it is best to create it from the beginning shown here – it will allow you to add components in an easy wizard where you can set the necessary parameters. Or just edit the source of this page – it’s worth creating a copy before if something went wrong :).

After changing the value of x / yMin / Max, we can open our website eg on a smartphone, click on index.html, but this time we choose the options to get anonymous link Next, click the QR code icon, and the code that appears scan by a smartphone.

Of course, the control works also via the Internet – not only in the local network, but in some cases an additional configuration is needed more about it here

Some technical details (not mandatory)

RemoteMe serves a website to control your car (and additionally it logs into your account – hence the token in the link opened by the smartphone) and participates in negotiating the creation of a WebRTC connection. The webRTC connection is a direct connection (RaspberryPi – browser (in some cases, for example, a turn server is needed for NAT)). After creating the webRTC connection, the variable state is not sent to RemoteMe at all (because the field “onlyDirect” in the components is set to true)

The program on Raspberry Pi which you run with the ./runme.sh command creates a websocket connection with the RemoteMe platform and manages the Python script (sends messages to it from the script to the platform etc). The operation of the Python script is possible thanks to the additional RemoteMe libraries (located in the leafDevices/base/ directory).

The websocket website itself connects to the RemoteMe platform (this is allowed by the javascript scripts imported in the header of the index.html file in paths start with /libs/). They facilitate and establish communication with the RemoteMe platform. Same components inserted in the index.html of the type: <variable component="**" In the remoteMeComponents.js file functions are replaced with “standard” and “bootstrap” html components in addition to the components are pinned events responding to user actions and sending relevant messages to Python script. You can see how remoteMeComponents.js creates new components – it can be interesting when you have to create your own components that RemoteMe does not allow to add from the wizard. In most cases, the actions of your components will perform variable changes as

RemoteMe.getInstance().getVariables(). setSmallInteger2("cameraPos",123,456,true);

which will send information about the variable change to the Python script, we also control the engines by setting the appropriate variable.

Summary

By creating this project I wanted to show you how to easily control your car with FPV view. The project can be expanded, or changed eg when using a different H bridge than I. So I encourage you to experiment with the sources of the project :). When this is your first project, I encourage you to do a project with a flashing diode at the beginning and read the documentation at www.remoteme.org

Cheers,

Maciej

Articles