This repository is the official implementation of DeepWDT (Deep Waggle Dance Translator), the most efficient and easy-to-use deep learning framework to detect and decode waggle dances.

To-do

• Make a "quick start" google colab of the inference pipeline
• Add a lightweight version of the deep learning model for quicker inference

Quick Start

Google colab version coming soon.

Requirements

We recommend you to use Anaconda to create a conda environment:

conda create -n deepwdt python=3.10

Then, activate the environment:

conda activate deepwdt

Then, set up your working environment ot our repository:

cd path_to_DeepWDT

Then install requirements:

pip install -r requirements.txt 

Dataset preparation

Prepare our dataset

As an example, the Apis dorsata training dataset can be downloaded at link coming soon.

To prepare it, copy the 'videos' folder and 'annotations.csv' into 'data/bengaluru_01' and launch the following command line:

python src/dataset/unpack.py --dataset name_of_the_dataset

Feel free to augment the dataset with your own data to improve the model's generalizability.

Custom data collection advice

As an example, for Apis mellifera videos, we recommend to capture videos with the camera pointing directly and straight at the hive frame and with the hive frame being fully inside the video frame. An ibservation hive is the optimal setup.

For technical details, please refer to the inference part.

Training and Validation

Training

To retrain the network on the training dataset, you can use the following command line:

python main.py --mode train --config src/config/train.yaml

If using our training dataset, you can use the config file located at src/config/train.yaml in our repository.

We recommend to use MLflow to follow the training metrics. To do this, set "MLFLOW" to "true" in the training config file and launch the MLflow session with:

mlflow ui

Validation

To run the validation using specific weights, use the following command line:

python main.py --mode eval --configuration src/config/eval.yaml

Don't forget to check that the configuration matches the one used when training the weights and to add the path to your trained weights at eval:model:resume in the yaml configuration file.

Results

Our results on a dataset made of 253 videos of Apis dorsata and Apis mellifera:

clip orientation	clip size	detected runs (%)	angle rmse (rad)	duration rmse (frame)
backward	8	100	0.34	4.93
backward	16	100	0.39	5.22
centered	3	99.7	0.42	7.79
centered	7	100	0.47	3.81
centered	17	100	0.33	2.44

Inference

Infer on a new dataset

python main.py --mode infer --configuration src/config/infer.yaml

Our training dataset is made of videos taken at 30 fps, with a definition of 1920x1080 and downscaled by 0.5x, which helps to reduce the inference time.

Thus, if you use our weight, we suggest to use 30 fps 1920x1080 videos and a downscale_factor of 0.5.

for each video in the infered dataset.

Translation to geographic coordinates

In order to properly translate dances to geographic targets, videos should be named as the following template: video-name_x_y_yyyy_mm_dd_hh_mm_ss.

Results of the inference are saved to runs/infer as:

• an annotated video
• a dataframe of the detections for each frame
• a gpkg containing the colony and the translated target positions
• a html map to visualize the translated targets

Currently, the model used to translate durations to distances is from https://doi.org/10.1016/j.anbehav.2019.01.016.

Parameters are ...

License

Our software is released under the MIT license.

Contact

DeepWDT is developed and maintained by Sylvain Grison (sylvain.grison@fieldphenomics.com).