To improve video capture for object tracking analysis, the robot allows recording stable video for later analysis while also providing a platform for testing tracking algorithms using the attached PTZ (pan–tilt–zoom) camera.

The robot’s motors are controlled by a LEGO EV3 brick running Debian 8 (“jessie”) controlled via Ethernet over Bluetooth from a host computer.

The main calculations, including processing of video frames and motor commands are done by the host computer, with only motor and sensor commands being sent over the network based on the video input. The processing is done by OpenCV v3.0.0 in 64-bit Python. The commands are sent using the rpyc library using Bluetooth.

The robot is being used for a variety of tasks, including acquiring video for optic flow analysis, using the camera and attached motors to avoid edges based on various edge detection and tracking algorithms, and other interesting projects.

Samples

MOSSE tracking algorithm

Below is a video sample of the robot using the adaptive MOSSE tracking algorithm to center the camera on a light switch as it moves: