A Human-Computer Collaborative Workflow for the Acquisition
and Analysis of Terrestrial Insect Movement in Behavioral Field Studies

Acquiring insect behavioral data from video sequences is a common technique in behavioral ecology and entomology. Entomologists collect video recordings of insect collectives, track the movement of individual insects, and analyze their motion patterns in order to understand their behavior. To do this, researchers often rely on purpose-built video processing programs that can perform this task automatically. Yet, the vast majority of available tools require highly-controlled lighting conditions and can only work in the lab. There are virtually no tools available that can perform accurate tracking of insects in outdoor environments where the insects normally operate.

This web page provides two software tools to help fill this gap:

  1. A video processing program that allows entomologists to process field-recorded videos, and extract the motion of insects from noisy frames recorded in outdoor environments.
     
  2. A visualization program that can be used to explore a large collection of insect trajectories, and to look for common behavioral patterns.

These two programs can be integrated together into a workflow for the acquisition and analysis of insect behavior from field-recorded video sequences [1]. The tools were developed during an interdisciplinary research project on the navigational strategies of Kenyan seed harvester ants [3]. Because no other tool fit our requirements, we had to develop our own tools in order to process and analyze the data. You can download the source code of the programs below. Additionally, we are also releasing the video dataset we collected during our field trip in 2012 (approximately 450 video segments comprising behavioral experiments and trail observations), in the hope that it will be useful for other researchers.

Video processing

This tool provides human-guided, semi-automated video processing functionalities to track the motion of terrestrial insects (such as ants) in highly dynamic environments (outdoor natural environments, for instance). The programs takes video segments (in AVI format) and tracks the movement of insects, records their trajectories, and save the trajectories to TXT files (one file per insect). To reduce error and improve tracking accuracy, the program expects a human analyst to supervise the tracking process. The user can click on an insect to instruct the program to track it. Additionally, the user may correct errors in the trajectory by removing erroneous 'jumps'.

The tool is written in C++ using openFrameworks and the OpenCV image processing library. Compilation instructions are provided in a README file included with the package.

Source code: antVideoProcessor-src.zip (25 MB)
 
Binary (for MacOS X 10.8): antVideoProcessor-bin.zip (4.2 MB)

Visualization

This visualization displays insect trajectories in a small-multiples layout, allowing an analyst to look at a large set of trajectories simultaneously [1, 2]. This layout facilitates comparison between trajectories, which may reveal different aspects of insect behavior under different experimental conditions. The program also allows researchers to explore hypotheses pertaining to spatio-temporal regularities in the trajectory data, and see whether those hypotheses are supported by the data in a visual manner.

The program is implemented in OpenGL, making it scalable to Large, High-Resolution displays. To compile the program, you need SDL, GLEW, and the freetype2 libraries. The source code is written in C++ and should compile on Linux and MacOS X without modifications (and on Windows with very few modifications). A Makefile is provided to help compilation. Included with the source code is a sample dataset comprising approximately 450 Seed harvest ant trajectories (in TXT format).

Source code: ants3d.tar.gz (5.2 MB)

Dataset

Coming up soon!

References

  1. K. Reda, V. Mateevitsi, and C. Offord. "A Human-Computer Collaborative Workflow for the Acquisition and Analysis of Terrestrial Insect Movement in Behavioral Field Studies (under review)
     
  2. K. Reda, A. Johnson, V. Mateevitsi, C. Offord, and J. Leigh. "Scalable Visual Queries for Data Exploration on Large, High-Resolution 3D Displays". 7th Ultrascale Visualization Workshop. Poceedings of the 2012 SC Companion: High Performance Computing, Networking Storage and Analysis, pages 196-205, Salt Lake City, Utah, Nov, 2012, IEEE Computer Society. [PDF]
     
  3. C. Offord, K. Read, and V. Mateevitsi. "Context-dependent navigation in a collectively foraging species of ant, Messor cephalotes". Insectes Sociaux, Springer (in press) [LINK]
     

For inquiries, please contact Khairi Reda: mreda2 (at) uic (dot) edu