
Control Camera position with smartphone gyroscope
Beginning
In the previous tutorial (here) I showed how to display the image from the Raspberry Pi camera in the browser window. In this tutorial I will show you how to control the position of the camera. We will control it in two ways
- By moving the mouse / finger on the video component (window)
- Tilting the phone and using the built-in gyroscope
All this on the website opened in the browser (I used Chrome for both the desktop and smartphone, because it has the best support for the webRTC protocol).
Don’t forget to checkout the video tutorial :
How it’s going to work
Just as the camera image is sent via the webRTC protocol, we will use the same channel to send instructions to control the position of the camera. Messages from the browser will go to the Python script on Raspberry Pi, and this script will drive the servo driver accordingly. It sounds simple and so it will be – sending messages is completely on the RemoteMe side, in the Python script you only need to implement a function that is called after changing the value of the variable – more about variables here
What is needed:
- Camera preview
- Control servo
- 16-bit PWM Adafruit Channel controller I2C
- Two 9g servos
- Servo Pan-tilt Mechanism (Pan-tilt with servos included)
- 16-bit PWM Adafruit Channel controller I2C
- Power – it does not give specific applications – because they depend on the motors and on what kind of batteries we have at our disposal. I used LiPo 4 * 1.2V batteries with a step-down converter
Connecting servo mechanisms
To control the servo mechanisms, we will use a 12-channel LED driver 16-bit PWM I2C compatible with Adafruit. It communicates with Rpi via the I2C interface. Adafruit provides a python library that works out-of-the-box, If in the second step RemoteMe installation on RPi you have chosen the [Yes] option, you have this library already installed;).
Connections:
SDA, SCL, and ground pins connect to RPI as shown:
Rpi pins:
Mounting the webcam tilt handle – it should not be difficult, the only thing you need to pay attention to is the assembly of the holder, so that it looks straight ahead and what is most important – the servo mechanisms were in a central position.
In the pictures and the movie you can see the green PCB to which the RPI and the servo driver are mounted, the files you can download from here it was designed in such a way that it can be easily performed using the thermal transfer method. In the repository there are also gerber files for easy ordering, eg from allpcb.com where I ordered my PCB.
The plate will be more useful when we add engine control to our car in the next tutorial.
How its going to work:
- For camera control we will use a variable of the type SMALL_INTEGER_2, which sends two numerical values, one of them will control the axis x with the second axis y.
- The variable will be changed by phone or computer where the image will be displayed
- By moving the mouse / finger around the area where the video is displayed
- Using a gyroscope in the smartphone where the rotation from the starting position will control the camera axes
Let’s start:
We will start by creating the variables – tab variables
-> add
the appeared window we complete:
We create by clicking Submit
more about variables here
Then, connect our Raspberry Pi to RemoteMe.org, how to connect here. Next, on the bar of the connected raspberry, click the burger menu and “Add External script with wizard”
In the first step, select the variable cameraPos
Thanks to this, in the generated code we will have functions ready to be implemented to interact with our variable.
In the second step, we name our device “python” and any unused Id. In the last step, click “Create device”. More about python devices here.
In the open window, we will add a code to support our servo mechanisms:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
import logging import socket import struct import sys import os os.chdir(sys.argv[1]) sys.path.append('../base') import remoteme import Adafruit_PCA9685 # Added import time # Added import RPi.GPIO as GPIO # Added pwm = None; # Added def setCameraPos(i1, i2): remoteMe.getVariables().setSmallInteger2("cameraPos", i1, i2) def onCameraPosChange(i1, i2): global pwm # Added logger.info("on camera change {} , {}".format(i1, i2)) # Added pwm.set_pwm(1, 0, i1) # Added pwm.set_pwm(0, 0, i2) # Added pass # Added def setupPWM(): # Added global pwm # Added pwm = Adafruit_PCA9685.PCA9685() # Added pwm.set_pwm_freq(80) # Added try: logging.basicConfig(level=logging.DEBUG, format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s', datefmt='%d.%m %H:%M', filename="logs.log") logger = logging.getLogger('application') setupPWM() # Added remoteMe = remoteme.RemoteMe() remoteMe.startRemoteMe(sys.argv) remoteMe.getVariables().observeSmallInteger2("cameraPos" ,onCameraPosChange); remoteMe.wait() finally: pass |
The added lines are marked with the #Added comment
Code overview
1 2 3 4 5 |
import Adafruit_PCA9685 # Added import time # Added import RPi.GPIO as GPIO # Added pwm = None; # Added |
pwm
to control the PWM controller (Adafruit_PCA9685)
1 2 3 4 5 6 |
def onCameraPosChange(i1, i2): global pwm logger.info("on camera change {} , {}".format(i1, i2)) pwm.set_pwm(1, 0, i1) pwm.set_pwm(0, 0, i2) pass |
1 2 3 4 |
def setupPWM(): global pwm pwm = Adafruit_PCA9685.PCA9685() pwm.set_pwm_freq(80) |
After saving the code, the program on Raspberry Pi will be reset to load the new script.
Web Page
Now we will add a web page with two sliders to find the central position of the camera and maximum swing. In the ideal world, the numbers would be, for example, 0.100 for both values, i.e. sending a variable of value (50.50), we would set our camera in the central position, and the value (100,100) would direct it up and right. Unfortunately, these numbers will certainly be different and we have to find them. For this purpose, we will create a website with two sliders that will send variables and we will know what are the variables.
To add a website – choose “New Device” -> “Webpage” and complete it as follows:
After adding, expand the beam, click index.html and “edit with wizard”. Then “Insert component” and complete:
More about components here
The most important information
- we will control two sliders of the CameraPos variable
- the minimum and maximum number that we will send is 0 – 1000 for both axes
Click the insert
, close the wizard window, and open the page in the new tab by clicking index.html
and selecting Open in new tab
.
We will get our website. The first slider is responsible for the X-axis of the camera (horizontal movements – ie looking left right) and the second one for the Y axis (ie looking up, down). When the sliders control differently, we replace the servo plugs with mechanisms in places. When the servos do not respond, check to see if they have been plugged into the right place.
Moving the sliders set the camera in a central position for me it is x = 564 and y = 474.
Then we tilt the camera up to the minimum values for me then x = 298. y = 223.
Then, calculate the maximum deflection for the camera, the calculation is to make the central position in the middle of the interval. So simple math
- Xmax = Xcenter-Xmin + Xcenter = 564 – 298 + 564 = 830
- Ymax = Ycenter-Ymin + Ycenter = 474 – 223+ 474 = 725
That is, Xmin = 298, Xcenter = 564, Xmax = 830. has an axis looks like this:
The point is that the distances between the values were the same and additionally the Xcenter value indicated the central position of the camera in the horizontal plane.
Y-axe : Ymin=223, Ycenter=474, Ymax=725
On the sliders, check if the maximum values set the camera in the right position without any problem and the servos do not block – if they block, increase Xmin and repeat the calculation (or calculate Xmin based on Xmax).
We are going back to the index.html page and delete the sliders removing the line from the page’s sources:
1 |
<variable component="slider" type="SMALL_INTEGER_2" name="cameraPos" label="camera" min="0" max="1000" valuebox="true" ></variable> |
And we re-open the Components Wizard and insert the components
camera:
Status – in the upper right corner it will display the connection status:
Finger / mouse control by moving on the image from the camera:
Please note that I have supplemented the Xmin values with the calculated values.
and Gyroscope – so that we can control the camera position by tilt the smartphone :
I also put the calculated values here. More about components and configurations here
It’s also worth changing the size of the video in index.html sources instead of width
and height
, enter style = "width: calc (100vmin); height: calc (75vmin)"
– this will fill the camera image when the phone is in a vertical position
We open our website in Smartphone – the easiest way is by clicking on index.html
-> Get anonymous Link
-> click on the QR code icon and scan it with a smartphone.
We test by sliding the finger on the screen, when the camera instead of pointing up its moving down in the component <variable component = "cameraMouseTrack" index.html
change invertX = "false"
into invertX = "true"
If the same applies to the Y axis, we change the value of invertY
in the same component
We turn on the gyroscope by clicking the Gyroscope button – it remembers the position of the smartphone and changes the position of the camera – when titl the phone , when moving is opposite then should be we changed similar like in previous step invertX
invertY
but now in component <variable component = "gyroscope"
. If we want the gyroscope to react more gently, we increase the values of the xRange
and yRange
properties
That’s all for today in the video at the beginning of the course are all steps.
Greetings,
Maciej