The software code, documentation and data contained in this repository corresponds to the animal behaviour research described in the following research paper.
A robust and flexible deep-learning workflow for animal tracking
Abhishek Dutta, Natalia Perez-Campanero, Graham K Taylor, Andrew Zisserman, Cait Newport
bioRxiv 2023.04.20.537633; doi: https://doi.org/10.1101/2023.04.20.537633
This open source code repository contains all the software code, scripts and data files (e.g. manual annotations and videos) that is required to analyse motion of a Picasso triggerfish navigating through a set of obstacles in a tank as shown in this video. We use computer vision based tools to determine fish body centroid, speed and motion direction in each video frame as well as the obstacles crossed by the fish during its journey towards the food target as shown in this output video.
We provide a sample video to demonstrate the automatic processing capabilities of this code repository. The results, for example see this video, obtained by running this demo are available in the data/tmp/F41_T22.mp4-EXAMPLE folder.
export ROOT=$HOME # files are stored in $ROOT/fish-tank-obstacles/ folder
export GPU_ID=0 # fish detector and tracker will use this GPU
cd $ROOT
git clone --recurse-submodules https://gitlab.com/vgg/fish-tank-obstacles.git
## Optional
## You may need to run these commands to install some tools (e.g. ffmpeg) used by processing scripts.
## $ sudo apt-get install python3-venv
## $ sudo apt-get install ffmpeg
cd fish-tank-obstacles/tools
./demo.sh
The process-video.sh script will first install all the required python dependencies and then extract frames from the video to detect and track fish. The results from this automatic analysis gets saved in data/tmp/F41_T22.mp4/ folder. As a reference, we have provided the results of processing this sample
video in the data/tmp/F38_T21.mp4-EXAMPLE folder.
Here are more detailed tutorials about training automatic detectors and automatic processing of videos.
- Motion analysis of Picasso triggerfish
- Training Fish and Obstacle Detectors
- Batch processing of multiple videos
- Frequently Asked Questions (FAQ)
- Project page: https://www.robots.ox.ac.uk/~vgg/research/fish/
- Preprint: PDF (10MB), bioRxiv
- Training Set (20 videos) : fish-tank-obstacles-train-20.zip (45Mb)
- Evaluation Set (15 videos) : fish-tank-obstacles-eval-15.zip (34Mb)
- Test Set (419 videos) : fish-tank-obstacles-test-419.zip (992Mb)
- Pre-trained models
All the assets (code, research paper, videos and documentation) described in this research project are licensed under the Creative Commons CC BY-NC 4.0 license.
This code repository is developed and maintained by Abhishek Dutta. For feedback and queries related to this project, contact Abhishek Dutta.