Merge pull request #33 from Noble-Lab/varun

Pull request for some issue fixing (#19, #22, #23, #30, #31, #34, #25)

Author: melihyilmaz

README.md

@@ -5,14 +5,43 @@
 
 Data and pre-trained model weights are available [here](https://zenodo.org/record/5976003).
 
-## How to get started with Casanovo?
+A link to the preprint of the paper where we discuss our methods and tests can be found [here](https://www.biorxiv.org/content/10.1101/2022.02.07.479481v1).
 
-Install `casanovo` as a Python package from this repo (requires Python 3.7+, dependencies will be installed automatically as needed):
+# How to get started with Casanovo?
+## Our recommendation:
+
+Install **Anaconda**! It helps keep your environment for casanovo and its dependencies separate from your other Python environments. **This is especially helpful because casanovo works within a specific range of Python versions (3.7 > Python version > 3.10).**
+
+- Check out the [Windows](https://docs.anaconda.com/anaconda/install/windows/#), [MacOS](https://docs.anaconda.com/anaconda/install/mac-os/), and [Linux](https://docs.anaconda.com/anaconda/install/linux/) installation instructions.
+
+Once you have Anaconda installed, you can use this helpful [cheat sheet](https://docs.conda.io/projects/conda/en/4.6.0/_downloads/52a95608c49671267e40c689e0bc00ca/conda-cheatsheet.pdf) to see common commands and what they do.
+
+## Environment creation:
+
+Open up the anaconda prompt and run this command:
+```
+conda create --name casanovo_env python=3.7
+```
+This will create an anaconda environment called `casanovo_env` that has python 3.7 installed (You can check if it was created by typing `conda env list`). 
+
+You can activate this environment by typing:
+```
+conda activate casanovo_env
+```
+To the left of your anaconda prompt line it should now say **(casanovo_env)** instead of **(base)**. If this is the case, then you have set up anaconda and the environment properly.
+
+**Be sure to retype in the activation command into your terminal when you reopen anaconda and want to use casanovo.** The base environment most likely will not work.
+
+## Installation:
+
+Install `casanovo` as a Python package from this repo (requires 3.7 > [Python version] > 3.10 , dependencies will be installed automatically as needed):
 ```
 pip install git+https://github.com/Noble-Lab/casanovo.git#egg=casanovo
 ```
 
-Once installed, Casanovo can be used with a simple command line interface. Run `casanovo --help` for more details. All auxiliary data, model and training-related variables can be specified in a user created `.py` file, see `casanovo/config.py` for the default configuration that was used to obtain the reported results.
+Once installed, Casanovo can be used with a simple command line interface. **Run `casanovo --help` for more details.** All auxiliary data, model and training-related variables can be specified in a user created `.yaml` file, see `casanovo/config.yaml` for the default configuration that was used to obtain the reported results.
+
+# Example Commands:
 
 - To evaluate _de novo_ sequencing performance of a pre-trained model (peptide annotations are needed for spectra):
 ```
@@ -28,5 +57,43 @@ casanovo --mode=denovo --model_path='path/to/pretrained' --test_data_path='path/
 ```
 casanovo train --mode=train --model_path='path/to/pretrained' --train_data_path='path/to/train/mgf/files/dir'  --val_data_path='path/to/validation/mgf/files/dir' --config_path='path/to/config'
 ```
+# Example Job:
+## A small walkthrough on how to use casanovo with a very small spectra (~100) set
 
+### The spectra file (.mgf) that we will be running this job on can be seen in the sample_data folder.
+
+- Step 1: Install casanovo (see above for details)
+- Step 2: Download the casanovo_pretrained_model_weights.zip from [here](https://zenodo.org/record/5976003). Place these models in a location that you can easly access and know the path of.
+    - We will be using pretrained_excl_mouse.ckpt for this job.
+- Step 3: Ensure you are in the proper anaconda environment by typing ```conda activate casanovo_env```. (If you named it differently, type in that name instead)
+- Step 4: Run this command:
+```
+casanovo --mode=denovo --model_path='[PATH_TO]/pretrained_excl_mouse.ckpt' --test_data_path='sample_data' --preprocess_spec=False
+```
+Make sure you have the proper filepath to the pretrained_excl_mouse.ckpt file.
+ - Note: If you want to get the ouput csv in a place OTHER than where you ran this command, specify where you would like the output to be placed by specifying a directory in the --output_path CLI field
+    - It would look like ```--output_path='path/to/output/location'``` appended onto the end of the above command. Be sure to provide a directory, not a file!
+
+This job should take very little time to run (< 1 minute), and the result should be a file named ```casanovo_output.csv``` wherever you specified.
+
+If the first few lines look like:
+```
+spectrum_id,denovo_seq,peptide_score,aa_scores
+0,LAHYNKR,0.9912219984190804,"[1.0, 1.0, 1.0, 0.99948...
+```
+Congratulations! You got casanovo to work!
+
+# Common Troubleshooting/FAQ
+
+## Installed casanovo and it worked before, but I reopened Anaconda again and now it says casanovo is not installed
+Make sure you are in the `casanovo_env` environment. You can make sure you are in it by typing
+```
+conda activate casanovo_env
+```
+## What CLI Prompts can I use?
+Run the following command in your command prompt:
+```
+casanovo --help
+```
+It should give you a comprehensive list of all CLI options you can tag onto a casanovo job and how/why to use them.
 

casanovo/casanovo.py

@@ -1,45 +1,86 @@
 """The command line entry point for casanovo"""
+from email.policy import default
 import click, logging
-from casanovo.denovo import train, test_evaluate, test_denovo
-
+import yaml
+import os
+from casanovo.denovo import train, evaluate, denovo
 
+#Required options
 @click.command()
-@click.option("--mode", default='eval', help="Choose to train a model or test denovo predictions")
-@click.option("--model_path", help="Specify path to pre-trained model weights for testing or to continue to train")
-@click.option("--train_data_path", help="Specify path to mgf files to be used as training data")
-@click.option("--val_data_path", help="Specify path to mgf files to be used as validation data")
-@click.option("--test_data_path", help="Specify path to mgf files to be used as test data")
-@click.option("--config_path", help="Specify path to config file which includes data and model related options")
-@click.option("--output_path", help="Specify path to output de novo sequences")
+@click.option("--mode", required=True, default='eval', help="Choose on a high level what the program will do. \"train\" will train a model from scratch or continue training a pre-trained model. \"eval\" will evaluate de novo sequencing performance of a pre-trained model (peptide annotations are needed for spectra). \"denovo\" will run de novo sequencing without evaluation (specificy directory path for output csv file with de novo sequences).", type=click.Choice(['train', 'eval', 'denovo']))
+@click.option("--model_path", required=True, help="Specify path to pre-trained model weights (.ckpt file) for testing or to continue to train.", type=click.Path(exists=True, dir_okay=False, file_okay=True))
+#Base options
+@click.option("--train_data_path", help="Specify path to .mgf files to be used as training data", type=click.Path(exists=True, dir_okay=True, file_okay=False))
+@click.option("--val_data_path", help="Specify path to .mgf files to be used as validation data", type=click.Path(exists=True, dir_okay=True, file_okay=False))
+@click.option("--test_data_path", help="Specify path to .mgf files to be used as test data", type=click.Path(exists=True, dir_okay=True, file_okay=False))
+@click.option("--config_path", help="Specify path to custom config file which includes data and model related options. If not included, the default config.yaml will be used.", type=click.Path(exists=True, dir_okay=False, file_okay=True))
+@click.option("--output_path", help="Specify path to output de novo sequences. Output format is .csv", type=click.Path(exists=True, dir_okay=True, file_okay=False))
+#De Novo sequencing options
+@click.option("--preprocess_spec", default=None, help="True if spectra data should be preprocessed, False if using preprocessed data.", type=click.BOOL)
+@click.option("--num_workers", default=None, help="Number of workers to use for spectra reading.", type=click.INT)
+@click.option("--gpus", default=(), help="Specify gpus for usage. For multiple gpus, use format: --gpus=0 --gpus=1 --gpus=2... etc. etc.", type=click.INT, multiple=True)
 
 def main(
+    #Req + base vars
     mode,
     model_path,
     train_data_path,
     val_data_path,
     test_data_path,
     config_path,
-    output_path
+    output_path,
+    #De Novo vars
+    preprocess_spec,
+    num_workers,
+    gpus
 ):
-    """The command line function"""
+    """
+    The command line function for casanovo. De Novo Mass Spectrometry Peptide Sequencing with a Transformer Model.
+    
+    \b
+    Training option requirements:
+    mode, model_path, train_data_path, val_data_path, config_path
+    
+    \b
+    Evaluation option requirements:
+    mode, model_path, test_data_path, config_path
+    
+    \b
+    De Novo option requirements:
+    mode, model_path, test_data_path, config_path, output_path
+    """
     logging.basicConfig(
     level=logging.INFO, 
     format="%(levelname)s: %(message)s",
 )
+    if config_path == None:
+        abs_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'config.yaml')
+        with open(abs_path) as f:
+            config = yaml.safe_load(f)
+    else:
+        with open(config_path) as f:
+            config = yaml.safe_load(f)
+
+    if(preprocess_spec != None): 
+        config['preprocess_spec'] = preprocess_spec
+    if(num_workers != None): 
+            config['num_workers'] = num_workers
+    if(gpus != ()):
+            config['gpus'] = gpus
     if mode == 'train':
-        
+
         logging.info('Training Casanovo...')
-        train(train_data_path, val_data_path, model_path, config_path)
+        train(train_data_path, val_data_path, model_path, config)
 
     elif mode == 'eval':
 
         logging.info('Evaluating Casanovo...')
-        test_evaluate(test_data_path, model_path, config_path)
+        evaluate(test_data_path, model_path, config)
 
     elif mode == 'denovo':
 
         logging.info('De novo sequencing with Casanovo...')
-        test_denovo(test_data_path, model_path, config_path, output_path)        
+        denovo(test_data_path, model_path, config, output_path)  
 
     pass
 

casanovo/config.py

@@ -0,0 +1,90 @@
+###
+# Config file for casanovo. Edit this to change parameters for train, eval, and denovo.
+# Note that blank entries are interpreted as "None"
+###
+
+#Random seed to be used across the pipeline
+random_seed: 454
+
+#Train data options
+train_annot_spec_idx_path: casanovo_train.hdf5 #path to write the training data index file, starts at os.getcwd()
+train_spec_idx_overwrite: True
+
+#Validation data options
+val_annot_spec_idx_path: casanovo_val.hdf5 #path to write the validation data index file, starts at os.getcwd()
+val_spec_idx_overwrite: True
+
+#Test data options
+test_annot_spec_idx_path: casanovo_test.hdf5 #path to write the test data index file
+test_spec_idx_overwrite: True
+
+#Preprocessing parameters
+preprocess_spec: True #Should be True when using raw mass spec data. Use False when reproducing Casanovo results with the provided benchmark data set which is pre-processed
+n_peaks: 150
+min_mz: 50.52564895 #1.0005079 * 50.5 
+max_mz: 2500
+min_intensity: 0.01
+fragment_tol_mass: 2  # Da
+
+#Hardware options
+num_workers: 8
+gpus: [0] #None for CPU, int list to specify GPUs
+
+#Model options
+dim_model: 512
+n_head: 8
+dim_feedforward: 1024
+n_layers: 9
+dropout: 0
+dim_intensity: 
+custom_encoder: 
+max_length: 100
+residues:
+  G: 57.021463735
+  A: 71.037113805
+  S: 87.032028435
+  P: 97.052763875
+  V: 99.068413945
+  T: 101.047678505
+  C+57.021: 160.030644505 #103.009184505 + 57.02146
+  L: 113.084064015
+  I: 113.084064015
+  N: 114.042927470
+  D: 115.026943065
+  Q: 128.058577540
+  K: 128.094963050
+  E: 129.042593135
+  M: 131.040484645
+  H: 137.058911875
+  F: 147.068413945
+  R: 156.101111050
+  Y: 163.063328575
+  W: 186.079312980
+  # AA mods:
+  M+15.995: 147.035399645 #131.040484645 + 15.994915  # Met Oxidation
+  N+0.984: 115.02694347 #114.042927470 + 0.984016,  # Asn Deamidation
+  Q+0.984: 129.04259354 #128.058577540 + 0.984016,  # Gln Deamidation
+max_charge: 10
+n_log: 1
+tb_summarywriter: 
+warmup_iters: 100000
+max_iters: 600000
+learning_rate: 5e-4
+weight_decay: 1e-5
+
+#Training/inference options
+train_batch_size: 32
+val_batch_size: 1024
+test_batch_size: 1024
+
+accelerator: "ddp"
+logger: 
+max_epochs: 30
+num_sanity_val_steps: 0
+
+train_from_scratch: True
+
+save_model: False
+model_save_folder_path: ''
+save_weights_only: True
+every_n_epochs: 1

casanovo/config.yaml

@@ -1,5 +1,5 @@
 """The de novo sequencing model"""
 from .model import Spec2Pep
 from .dataloaders import DeNovoDataModule
-from .train_test import train, test_denovo, test_evaluate
+from .train_test import train, denovo, evaluate