RGB Camera

The camera gives an image of what is in front of it. You need it to find and read wall tokens. The starter robot has one camera, named camera, of 64 × 40 pixels.

Reading pixels

An image is a grid of pixels. Each one has a red, green and blue value from 0 to 255, where (0, 0, 0) is black and (255, 255, 255) is white.

Python

from controller import Robot

robot = Robot()
timeStep = int(robot.getBasicTimeStep())

camera = robot.getDevice("camera")
camera.enable(timeStep)
width = camera.getWidth()
height = camera.getHeight()
print("The camera has", width, "x", height, "pixels")

while robot.step(timeStep) != -1:
    image = camera.getImage()
    # The colour of the pixel in the center of the image
    r = camera.imageGetRed(image, width, width // 2, height // 2)
    g = camera.imageGetGreen(image, width, width // 2, height // 2)
    b = camera.imageGetBlue(image, width, width // 2, height // 2)
    print("Center:", r, g, b)

C/C++

#include <iostream>
#include <webots/Robot.hpp>
#include <webots/Camera.hpp>

using namespace webots;
using namespace std;

int main() {
  Robot *robot = new Robot();
  int timeStep = (int)robot->getBasicTimeStep();

  Camera *camera = robot->getCamera("camera");
  camera->enable(timeStep);
  int width = camera->getWidth();
  int height = camera->getHeight();
  cout << "The camera has " << width << " x " << height << " pixels" << endl;

  while (robot->step(timeStep) != -1) {
    const unsigned char *image = camera->getImage();
    // The colour of the pixel in the center of the image
    int r = Camera::imageGetRed(image, width, width / 2, height / 2);
    int g = Camera::imageGetGreen(image, width, width / 2, height / 2);
    int b = Camera::imageGetBlue(image, width, width / 2, height / 2);
    cout << "Center: " << r << " " << g << " " << b << endl;
  }

  delete robot;
  return 0;
}

Using OpenCV

OpenCV is a library to process images: find shapes, compare colours and much more. Webots images have 4 values per pixel, in BGRA order, which OpenCV can use directly. This program shows the image, and its bright parts, in two windows:

Python

from controller import Robot
import cv2
import numpy as np

robot = Robot()
timeStep = int(robot.getBasicTimeStep())

camera = robot.getDevice("camera")
camera.enable(timeStep)

while robot.step(timeStep) != -1:
    # Webots images have 4 channels per pixel, in BGRA order
    image = np.frombuffer(camera.getImage(), np.uint8).reshape((camera.getHeight(), camera.getWidth(), 4))
    frame = cv2.cvtColor(image, cv2.COLOR_BGRA2BGR)
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    _, thresh = cv2.threshold(gray, 200, 255, cv2.THRESH_BINARY)  # Bright parts in white

    cv2.imshow("camera", cv2.resize(frame, None, fx=4, fy=4, interpolation=cv2.INTER_NEAREST))
    cv2.imshow("threshold", cv2.resize(thresh, None, fx=4, fy=4, interpolation=cv2.INTER_NEAREST))
    cv2.waitKey(1)  # Shows the windows

C/C++

#include <opencv2/opencv.hpp>
#include <webots/Robot.hpp>
#include <webots/Camera.hpp>

using namespace webots;

int main() {
  Robot *robot = new Robot();
  int timeStep = (int)robot->getBasicTimeStep();

  Camera *camera = robot->getCamera("camera");
  camera->enable(timeStep);

  while (robot->step(timeStep) != -1) {
    // Webots images have 4 channels per pixel, in BGRA order
    cv::Mat image(camera->getHeight(), camera->getWidth(), CV_8UC4, (void *)camera->getImage());
    cv::Mat frame, gray, thresh;
    cv::cvtColor(image, frame, cv::COLOR_BGRA2BGR);
    cv::cvtColor(frame, gray, cv::COLOR_BGR2GRAY);
    cv::threshold(gray, thresh, 200, 255, cv::THRESH_BINARY); // Bright parts in white

    cv::Mat big;
    cv::resize(frame, big, cv::Size(), 4, 4, cv::INTER_NEAREST);
    cv::imshow("camera", big);
    cv::resize(thresh, big, cv::Size(), 4, 4, cv::INTER_NEAREST);
    cv::imshow("threshold", big);
    cv::waitKey(1); // Shows the windows
  }

  delete robot;
  return 0;
}

On the left, an image of the starter robot’s camera in front of a letter victim. On the right, its bright parts, as this program finds them.

Note

For Python, OpenCV is installed with the other Python libraries. For C++, you need to install OpenCV and add it when compiling. The C++ example was tested with OpenCV 4.

Windows like these are for testing at home: turn them off for the competition.

Resolution

Higher resolutions see tokens from further away, but cost more and take longer to process. You choose the resolution of each camera in the Robot Customizer.