I would like to present a simple project which involves Raspberry Pi with a camera. Sounds boring, right! But the camera can be tilted in two axis using two servos which are directly controlled via STM32 microcontroller which in turn communicates with Raspberry Pi. Furthermore, the RPi is hosting a web server with interface to control position and speed of the camera and of course the video is streamed so you can see what is going on i.e. in your room. If you are even a bit intrigued then keep reading.

The basic idea was presented at the beginning. The whole project is a nice example of a hybrid system presented in one of my recent articles (Design of embedded system). The code for this whole project is available at my github account. You can find the link to it at the bottom of this page. Before you begin you can have a look and check out how it works in below video:

 

Architecture

Below image shows all the subsystems in the project and how they are connected to each other:

There are three main components of the system:

• Raspberry Pi — SBC responsible for hosting a web page with video live stream to the Internet, capturing video from camera module and sending control commands to the STM32,
• STM32 — a Blue Pill board with a STM32 microcontroller which communicates with the Raspberry Pi through I2C bus and controls camera module view angles,
• Camera module — a Raspberry Pi Camera sensor mounted on a manipulator with 2 DoF which are controlled with RC servos.

Each component will be described in detail below starting with Raspberry Pi. All aspects of this subsystem will be brought up — setting a web server, capturing video from Raspberry Pi camera sensor and communicating with STM32 microcontroller. Then embedded software will be disused where I have highlighted the problematic I2C communication and a solution to this issue. Since Camera module is a Raspberry Pi camera sensor with two RC servos it will not be comprehensively discuses. However, if you are interested in basics of controlling RC servo you can go to this link and read more about it.

Raspberry Pi

Here I will describe the basic set up for the mini computer. There are three main elements of the system which have to be present to make it all work:

• camera,
• web server,
• communication.

Camera is the obvious part, a standard dedicated for RPi camera module was used. However, any other, like USB webcam, can be used but it requires slightly different configuration. If you would like to follow this path read Using webcam with Raspberry Pi instead to set up a webcam.

What is really cool about RPi is that almost all kinds of software work on it and among those is Apache. The Apache server can be used to host a webserver it will be used along with PHP to run some scripts on server side (Raspberry Pi).

The last part where RPi needs to be configured is communication. A communication interface is required to communicate with a microcontroller — for this purpose I2C was used.

Camera

There is a preliminary step which has to be done before working with the camera. The camera interface has to be turned on. Unless it is already turned on you have to use raspi-config command, like this:

sudo raspi-config

There you can navigate through the interface and set camera interface to Enabled. After that reboot RPi and you are good to go. I assume that the camera is already connected to the mini computer.

UV4L

Now, we need some video streaming server. UV4L will be a good choice. Here I have written a short tutorial how to set up UV4L for Raspberry Pi.

LAMP

Another part of this post is setting up a LAMP. LAMP stands for Linux Apache MySQL PHP. This means that we use a Linux operating system with three additional set of packages. There is well rooted debate about why not to use Apache and use webserver such as NGINX instead. The point of this tutorial is not to involve into this debate but present a working and well maintained solution. As a side note it is worthy of note to say that NGNIX is more efficient solution and is better suited for low performance hardware.

Let’s install or required packages:

sudo apt-get install apache2 php5 libapache2-mod-php5 mysql-server php5-mysql -y

Above will install all required packages. After installation make sure that Apache service is working. It can be done with restarting the service which will restart it or start it if it is not running.

sudo service apache2 restart

One more remark on Apache. You should remember to restart it every time you change its configuration.

Next step is to check if everything is working properly. Open your web browser and enter URL: localhost. It should display standard info page of Apache. You can access you RPi from locall network by entering URL: RPI_LOCAL_IP where RPI_LOCAL_IP is yours RPi’s local IP address. Now let’s check if PHP is working as it should, run following:

sudo rm /var/www/html/index.html

It will remove default html. If you prefer to leave it you should change its name with:

sudo mv /var/www/html/index.html /var/www/html/index.html.old

After that run:

sudo echo "<?php phpinfo(); ?>" > /var/www/html/index.php

Presented command will create a index.php file with a single PHP line of code which will display PHP configuration. Now, access or refresh the content of the web browser and you should see a PHP configuration. This is all for now. If you have any troubles you can follow this tutorial where Author is setting up LAMP for WordPress on Raspberry Pi. What is important is the path /var/www/html this is where you should keep web content (php files, images, etc.).

Finally, it is nice to keep things clean. Usually, the directory /var/www/html should belong to user www-data. This user already exists in your OS. As the name suggest it is related to www content. As pi user on RPi you should add this user to www-data group with this

sudo usermod -a -G www-data pi

You should relog yourself for this to take effect. Now, it is advised to change privileges of web related content:

sudo chown -R www-data:www-data /var/www/html

This is not mandatory but setting your environment properly is a good practise. However, above is not always true. You may have some reasons, and often justified, to do otherwise.

Web page

After setting up the LAMP we are now able to create a web page which will be the front end for our little surveillance webservice. This can be divided into two groups:

• the front end, which is the user is able to see,
• the back end this is what’ under the hood.

As for the front end I have created a responsive web site using Bootstrap. This is a framework for web development. Using it it is easy to create fully responsive websites with little effort. The code can be found at the end of this tutorial in Repositories section.

Login panel:

Sweep panel:

Position panel:

Communication

Since, the front end is a state-of-the-art approach the back end requires a bit more explanation. The front end can communicate with STM32 through the interface provided with RESTful API. Below you can see a snipped:

$(function () {
        $('#formSendPosition').bind('click', function (event) {

            event.preventDefault();// stops page refresh
            event.stopPropagation();// stops page refresh, some additional
            
            var dataString = $("#formHorizontal, #formVertical, #formHiddenPassword").serialize();
          $.ajax({
            type: 'POST',
            url: 'controller.php',
            data: dataString,
            success: function (data) {
                //alert(data);
            },
            error: function (data) {

            }
          });
          
          return false; // stops page refresh, just a precaution

        });
    
        $('#formSendSweep').bind('click', function (event) {

            event.preventDefault();// stops page refresh
            event.stopPropagation();// stops page refresh, some additional
            
            var dataString = $("#formMinMax, #formSpeed, #formAxis, #formHiddenPassword").serialize();
          $.ajax({
            type: 'POST',
            url: 'controller.php',
            data: dataString,
            success: function (data) {
                //alert(data);
            },
            error: function (data) {

            }
          });
          
          return false; // stops page refresh, just a precaution

        });
      });

Above are two binding functions to Send buttons available in web interface. It gathers all information from the forms like what speed or position should be send to the controller. The information is send through controller.php end-point and it looks like this:

<?php
include_once 'config.php';

if (isset($_POST['hiddenPassword'])) {
    if ($_POST['hiddenPassword'] === $apikey) {
        if (isset($_POST['action'])) {
            if ($_POST['action'] === 'position') {
                if (isset($_POST['hvalue'])) {
                    
                    $hvalue = $_POST['hvalue'];
                    
                    $value = 256 - $hvalue;
                    $command = "sudo i2cset -y 1 0x33 0x00 0 " . $value . " 0x00 0x00 0x00 0x00 0x00 i";
                    shell_exec($command);
                }
                if (isset($_POST['vvalue'])) {
                    
                    $vvalue = $_POST['vvalue'];
                    
                    $value = 256 - $vvalue;
                    $command = "sudo i2cset -y 1 0x33 0x00 1 " . $value . " 0x00 0x00 0x00 0x00 0x00 i";
                    shell_exec($command);
                }
            } else if ($_POST['action'] === 'sweep') {
                if (isset($_POST['sweepAxis'])) {
                    $axis = 0;
                    if ($_POST['sweepAxis'] === 'vertical')
                        $axis = 1;
                    
                    // minvalue=128&maxvalue=128&action=sweep&speedvalue=128&sweepAxis=vertical
                    
                    $speed = $_POST['speedvalue'];
                    $minvalue = 256 - $_POST['minvalue'];
                    $maxvalue = 256 - $_POST['maxvalue'];
                    
                    if ($minvalue > $maxvalue) {
                        $tmp = $minvalue;
                        $minvalue = $maxvalue;
                        $maxvalue = $tmp;
                    }
                    
                    $command = "sudo i2cset -y 1 0x33 1 " . $axis . " ";
                    $command .= intval($speed % 256) . " " . intval($speed / 256) . " ";
                    $command .= $maxvalue . " 0 " . $minvalue . " 0 i";
                    shell_exec($command);
                }
            } else {
                echo "Bad action!";
            }
        } else {
            echo "No action!";
        }
    } else {
        echo "Bad key!";
    }
} else {
    echo "Unauthorized access!";
}

?>

It simply runs i2cset command on Raspberry Pi. This is how the communication from the perspective of back end works. Also some precautions were taken into consideration like security. Since controller.php is openly available via Internet unauthorized person can send come command which will affect the work of STM32 microcontroller. However, if you look closely there is a line:

if ($_POST['hiddenPassword'] === $apikey) {

This allows to validate if the commands are coming from the web interface and not from direct calls. The API key is also embedded in the web interface during parsing the PHP file — panel.php, like this:

<div class="form-group">
<input class="form-control" type="hidden" name="hiddenPassword" value="<?php echo $apikey; ?>">
</div>

All settings, there are not many, can be configured via editing config.php file. A template was provided which looks like this:

<?php
    $apikey = "some api key like 0123456789";
    $loginPassword = "some password";
    $streamURL = "some URL";
    $streamWidth = 640;
    $streamHeight = 480;
?>

The code is fairly simple but secure. To quickly sum up with the list of the most important files and their purpose:

• assets/js/data.js — holds JavaScript code which allows to bind click action on a button to send data to the controller.php file,
• cam.php — main file, it includes config.php file, displays login form (form.php) if there was no authentication and if there was a prositive authentication then it displays panel.php (shows live feed from camera and delivers some controls),
• config_template.php — a template version of config.php, after setting it up it should be renamed to config.php,
• config.php — holds current configuration,
• controller.php — runs bash commands like i2cset to actually control servos via sending commands through I2C bus to STM32 microcontroller,
• form.php — show simple login form,
• index.php — loads cam.php and starts session
• panel.php — it is divided into two pars, at the top a live feed is visible and at the bottom there controls for two different settings (setting desired position of the servos or setting boundaries and speeds for sweep mode)

STM32

The project for STM32 microcontroller is quite simple. I am using so called blue pill which is a basic development board with STM32F103C8T6. The uC has 64 kB of Flash memory and 20 kB of RAM, and it can run blazing fast — 72MHz. Most importantly it has a Cortex-M3 core. Well, this is more than enough for this type of project. The project itself involves two things. The first is controlling the servo position with a PWM signal and the second one is I2C communication.

The device can control servos in on of two modes (but there is still some place for further development):

• setting desired position of a channel — servo,
• maintaining desired velocity on a channel — sweep mode.

Here is a configuration of the microcontroller from STMCubeMx software:

The pins TIM1_CHx where x is 1, 2, 3 or 4 are PWM output channels. However, only two (TIM1_CH1 and TIM1_CH2) are being used. The I2C_SDA and I2C_SCL are ports used during I2C communication.

Communication

For the communication between Raspberry Pi and the STM32 I am using I2C interface. This is quite reliable and easy to implement. However, in most cases you probably use I2C to read some data from sensors connected to the uC. In this case, it is the other way around. The uC is being used as a sensor. Having this in mind, the STM32 has to work in slave mode:

ret = HAL_I2C_Slave_Receive_IT(&hi2c1, buffer, 8);

In cube you can configure the slave address and this is pretty much all:

The one other thing which has to be taken into account is writing a callback function for receiving data. It looks like this:

void HAL_I2C_SlaveRxCpltCallback(I2C_HandleTypeDef *hi2c) {
    receiveCompleted = 1;
    dataReady = 1;
}

Protocol

For exchanging information between uC and RPi we need some kind of a protocol. I have proposed a very simple one. The length of the message is constant and is equal to 8. The first carries information about the mode. For now it can be either 0 or 1. 0 for setting position and 1 for sweep mode. Depending on each mode we are in the rest of the message is treated accordingly.

For mode 0 — position setting the frame is following:

byte 0 — mode,
byte 1 — channel (the servo),
byte 2-3 — angle, a new value for servo,
byte 4-7 — ignored.

For mode 1 — sweep frame interpretation:

byte 0 — mode,
byte 1 — channel (the servo),
byte 2-3 — speed for servo 1LSB = [0.1 impulses / sec],
byte 4-5 — max angle, the maximum limit for the servo position,
byte 6-7 — min angle, the minimum limit for the servo position.

Get rid of bad data

During testing I have noticed that sometimes there is a glitch on the line. It also happens when the device is already running and the RPi is just starting. There are two main reasons for this:

• interference,
• fixed length of the message.

The second one is actually related to the interference. For example, if I send some data via the I2C which is less then the requested length the I2C communication will be stuck! The solution is quite simple and very robust. If there is on going transmission, some data is already in the buffer, wait some time and reset the communication. The reset is only performed when the uC was stuck — it received less data then it was supposed to.

if (receiveCompleted == 1) {
    receiveCompleted = 0;
    ret = HAL_I2C_Slave_Receive_IT(&hi2c1, buffer, 8);
}

//check if there is ongoing OLD transfer
if (hi2c1.XferCount < 8) {
    if (get_time == 1) {
        last_time = HAL_GetTick();
        get_time = 0;
    } else {
        time = HAL_GetTick();
        if (time - last_time > 200) {
            //reset transmission
            HAL_I2C_DeInit(&hi2c1);
            HAL_Delay(20);
            MX_I2C1_Init();
            HAL_Delay(20);
            ret = HAL_I2C_Slave_Receive_IT(&hi2c1, buffer, 8);
            get_time = 1;
        }
    }
}

Mode control

In the example I provide, in the last section, I am using a single timer — TIM1 to generate a PWM signal. The prescaler was set to 719 and the period to 2000 (20 ms). To read more about RC servos you can refer to an article I have written a very long time ago — Servo. There is also another timer (TIM2) but it is used for sweep mode only.

if (dataReady == 1) {
    dataReady = 0;
    command = (uint8_t) buffer[0];
    switch (command) {  
        //set position
        case 0: {
            //0 -- command
            //1 -- channel
            //2-3 -- set value
            channel = buffer[1];
            value = *(uint16_t*) &buffer[2];

            __HAL_TIM_SET_COMPARE(&htim1, availableChannels[channel], value);

            mode = 0;
            break;
        }       
        //scan
        case 1: {
            //0 -- command
            //1 -- channel
            //2-3 -- set value (speed) [ 0.1 impulses / sec]
            //4-5 -- max value
            //6-7 -- min value

            channel = buffer[1];
            speed = *(uint16_t*) &buffer[2];
            value_max = *(uint16_t*) &buffer[4];
            value_min = *(uint16_t*) &buffer[6];

            //get current value to start from
            value = __HAL_TIM_GET_COMPARE(&htim1, availableChannels[channel]);

            if (value > value_max)
                value = value_max;
            if (value < value_min)
                value = value_min;

            value_float = value;
            sweep_direction = +1;

            mode = 1;
            break;
        }
    }
}

if (mode_flag == 1) {
    mode_flag = 0;
    if (mode == 1) {

        if (value >= value_max)
            sweep_direction = -1;
        if (value <= value_min)
            sweep_direction = +1;

        // direction * time base * speed * sweep coefficient
        value_float += (float) sweep_direction * 0.001f * (float) speed * 0.1f;
        value = value_float;

        __HAL_TIM_SET_COMPARE(&htim1, availableChannels[channel], value);
    }   
}

if (mode != 1)
    HAL_Delay(10);

There are three if statements. The first one checks if there was some new data and if there was it switches to a new mode and sets new control values for servos.

The second if is dedicated to mode=1 — sweep. Its main purpose is to check limit values. In other words, it prevents the servo to turn more then the limit values. The third if is for optimization. If we are in mode 0 then it will be execute and once every iteration of while loop we will wait for 10 ms. However, if we are in sweep mode we can not afford to wait that long because the new position calculation for high speeds can take less then these 10 ms and we won’t be able to achieve high speeds. It could be done differently but I had my reasons.

Conclusion

The purpose of this project was to mainly test the I2C communication between Raspberry Pi and STM32. As you could see it was a success. The other advantage of this project is remotely controlled web cam 🙂

Repositories

You can find full project sources at my github repository.

It was divided into two parts: web and stm32. In the first part you can find web server and Raspberry Pi related files and in stm32 directory you can find full project for Pan Tilt Camera in Atollic TrueSTUDIO for STM32. If you have any questions regarding this project feel free to leave a comment.