|
1 | 1 | # BuzzDetect
|
2 | 2 |
|
3 |
| -## Get started by accessing the terminal |
4 |
| -1. Log in |
| 3 | +## Getting set up on OSC |
| 4 | +1. Log in to OSC |
5 | 5 |
|
6 |
| - i. Visit www.osc.edu and click “login” in the top-right corner |
7 |
| - |
8 |
| - ii. Click “OSC OnDemand” |
| 6 | + i. Visit www.osc.edu, click the "Access OSC" dropdown in the top right, and click System Gateway |
| 7 | +<img src="/documentation/OSC_login.png" width="500" |
9 | 8 |
|
10 |
| - iii. Enter your username and password |
| 9 | + ii. Log in with your username and password |
11 | 10 |
|
12 | 11 | 2. Navigate to the project directory
|
13 | 12 |
|
14 |
| - i. Click files |
15 |
| - |
16 |
| - ii. Navigate to /fs/ess/PAS1063/buzzdetect |
17 |
| - |
18 |
| - iii. Within the buzzdetect folder, click “open in terminal” |
19 |
| - |
20 |
| -3. There is a conda environment that has all necessary modules installed and configurations made. Load the environment by typing `conda activate buzzdetect-py39` |
| 13 | + i. Click the "Files" dropdown in the top left |
21 | 14 |
|
22 |
| - - If you get the error `CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.`, run `conda init bash` and then rerun the activate command |
23 |
| - |
24 |
| -## To analyze audio in bulk (several audio files) |
25 |
| - |
26 |
| -1. Follow the instructions under “Get started by accessing the terminal” (step 3 is important) |
27 |
| -2. Create a folder containing the audio files you want. The folder should not contain any additional files. It’s recommended you put the folder inside of the loadingdock folder. |
28 |
| -3. Complete any preprocessing |
29 |
| - |
30 |
| - a. Files must be .wav files |
31 |
| - |
32 |
| - b. Use `python buzzdetect.py --action preprocess --preprocesspath [path to directory]` to ensure each .wav file is 16bit |
33 |
| - |
34 |
| - Note: preprocesspath demands a directory containing the audio files to preprocess. If you are looking to preprocess a single audio file, put it in an otherwise empty directory and use that directory’s path |
35 |
| -4. Run the buzzdetect.py script, including the following flags: |
| 15 | + ii. Click "/fs/ess/PAS1063" |
36 | 16 |
|
37 |
| - `--action analyze` |
| 17 | + You are now in the "project directory" for the Bee Lab. Files made here will, by default, be accessible to anyone else on this project. This differs from your home directory, which has a path of `/fs/users/PAS1063/[your username]`. Note: if you were first granted access to the supercomputer on another project, your home directory won't have `PAS1063`, but the ID of the first project you were granted access to. |
38 | 18 |
|
39 |
| - `--type bulk` |
| 19 | + ii. Navigate to /fs/ess/PAS1063/buzzdetect |
40 | 20 |
|
41 |
| - `--analyzesrc [path to directory]` |
| 21 | + iii. Within the buzzdetect folder, click “open in terminal” |
42 | 22 |
|
| 23 | +4. Activate the conda environment |
| 24 | + - The conda environment is a location where all packages for the buzzdetect tool are pre-configured |
| 25 | + - If there is a directory within buzzdetect/ called "environment", the conda environment has already been set up. Load the environment by typing `conda activate buzzdetect-py39` |
| 26 | + - If you get the error `CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.`, run `conda init bash` and then rerun the `activate` command |
| 27 | + - If there is no "environment" directory, see [the time first setup documentation](https://github.com/OSU-Bee-Lab/BuzzDetect/blob/main/documentation/documentation_firstTimeSetup.md), then return to this step. |
| 28 | + |
| 29 | + |
| 30 | +## Getting set up locally |
| 31 | +1. Ensure all your files are up-to-date with the GitHub repo |
| 32 | +2. Install python and conda on your device [need more detail!] |
| 33 | +3. Go through the [time first setup process](https://github.com/OSU-Bee-Lab/BuzzDetect/blob/main/documentation/documentation_firstTimeSetup.md) |
| 34 | +4. You're ready to go! Load or train a model and get detecting. |
| 35 | + |
| 36 | +## To analyze audio |
| 37 | +What follows is a basic workflow for the buzzdetect tool. If you want refined control over the analysis, see the [command line documentation](https://github.com/OSU-Bee-Lab/BuzzDetect/blob/main/documentation/documentation_CLI.md). |
| 38 | +1. Place the files you wish to analyze in ./buzzdetect/audio_in |
| 39 | + - At the time of writing, files need to be WAV files with 16 bit depth and cannot be hours long (exact filesize limit unknown) |
| 40 | + - At the time of writing, buzzdetect's preprocessing function is not functioning |
| 41 | +2. Have a trained model ready in the ./buzzdetect/models/ directory |
| 42 | + - See the section "To train a model" below |
| 43 | +4. Open a terminal in the project root directory (./buzzdetect/) and activate the conda environemnt with `conda activate ./environment` |
| 44 | +5. Analyze the audio files with the following command: `python buzzdetect.py analyze --modelname [your model's name here]` |
43 | 45 | - See the comprehensive command-line documentation [here](https://github.com/OSU-Bee-Lab/BuzzDetect/blob/main/documentation/documentation_CLI.md) for additional configuration options.
|
| 46 | +6. The analysis will be output as .txt files in the directory ./buzzdetect/output |
| 47 | + - Note: running the analysis multiple times will append (not overwrite) the new information to the old file; this may cause problems |
44 | 48 |
|
45 |
| - - The whole command will look something like this: `python buzzdetect.py --action analyze --type bulk --analyzesrc loadingdock/togo` |
46 |
| - |
47 |
| - - By default .txt files will be output into the bulkoutput folder. If the file already exists, it will be appended to (not overwritten), which may cause problems |
48 |
| - |
49 |
| - - Audio files will not be deleted. |
50 |
| - |
51 |
| -## To analyze a single audio file |
| 49 | +## To train a model |
| 50 | +New models can be trained with the command `python buzzdetect.py train --modelname [your model name here] --trainingset [your training set here]` |
| 51 | +"Training set" refers to a metadata CSV, stored in ./buzzdetect/training/ with the name of "metadata_[setname].csv". The trainingset option looks for a metadata CSV with a matching name, subsets the training data (in the ./buzzdetect/training/audio directory) for matching audio files, and trains a model on them. |
52 | 52 |
|
53 |
| -Follow the same instructions as above, but for step 3b, use --type bulk, and input the path to the file for --analyzesrc |
54 |
| - |
55 |
| -## Retraining the model |
| 53 | +The model is output as a folder to ./buzzdetect/models. The folder is then read by the analyze function of buzzdetect.py for analysis. |
56 | 54 |
|
57 | 55 | ## Uploading files with FileZilla
|
58 | 56 | Files can be uploaded with drag-and-drop, or with the “Upload” button, however, when you need to upload a lot of data (many files or few, large files), FileZilla makes things easier.
|
59 | 57 |
|
60 | 58 | 1. Download/install at https://filezilla-project.org/download.php
|
61 | 59 | 2. Click the “Site Manager” button in the top left:
|
62 | 60 |
|
63 |
| - <img src="/documentation/filezilla_sitemanager.png" width="400">= |
| 61 | + <img src="/documentation/filezilla_sitemanager.png" width="400"> |
64 | 62 | 3. The lefthand file selector is for files on your computer; the righthand file selector contains files on the server
|
65 |
| -4. Navigate to /users/PAS1063/jcunderwood/bee-audio on the righthand pane and drag the files you want to move from your computer |
66 |
| - |
67 |
| -## Complete buzzdetect.py command-line documentation |
68 |
| -Complete buzzdetect.py command-line documentation can be found [here](https://github.com/OSU-Bee-Lab/BuzzDetect/blob/main/documentation/documentation_CLI.md). (still a wip) |
69 |
| - |
70 |
| -## Misc. |
71 |
| -The legacy folder is a sort of graveyard of old scripts, as well as testing data, that are not used by BuzzDetect. If you’re trying to run some kind of analysis/testing/processing that is not supported off-the-shelf, you may find a script here that does what you want. |
72 |
| - |
73 |
| - |
74 |
| -— James Underwood / jcunderwood [at] protonmail [dot] com |
| 63 | +4. Navigate to /fs/ess/PAS1063/buzzdetect on the righthand pane and drag the files you want to move from your computer |
0 commit comments