Download weight file from URL

[Lilferrit]

In order to address #108, the --model parameter now supports URLs. If the --model parameter is a valid URL the Casanovo CLI will first look in the cache for the weight file (cache resolution is done via hashing the URL), and if the weight file isn't cached the CLI will attempt to download the weight file and cache it. The CLI will raise an error if the --model parameter is set but it's not a valid URL or file in the local filesystem.

However this implementation does have an issue where if the weight file at a URL is updated, but the CLI has previously downloaded weights from that URL, the new weights won't be downloaded because there will still be a cache hit. To fix this I could introduce a --force option that skips the cache and always attempts to download and cache weights given a URL.

[codecov]

Codecov Report

Attention: Patch coverage is 91.80328% with 5 lines in your changes missing coverage. Please review.

Project coverage is 94.03%. Comparing base (276a50e) to head (c2bc971).

Files	Patch %	Lines
casanovo/casanovo.py	91.80%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##              dev     #349      +/-   ##
==========================================
+ Coverage   89.98%   94.03%   +4.04%     
==========================================
  Files          12       12              
  Lines         979     1022      +43     
==========================================
+ Hits          881      961      +80     
+ Misses         98       61      -37     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[bittremieux]

A few small comments.

Two key things are still missing as well:

1. Documentation of this functionality on readthedocs (maybe on the file formats page).
2. Unit tests for the download behavior.

[bittremieux]

However this implementation does have an issue where if the weight file at a URL is updated, but the CLI has previously downloaded weights from that URL, the new weights won't be downloaded because there will still be a cache hit.

You could also check the "last-modified" information for the URL and verify whether this is more recent than the download time or not.

[Lilferrit]

I adjusted the hashing implementation such that now files downloaded from a user specified URL will now be downloaded to <cache_dir>/<url_hash>/<file_name>.ckpt (where filename is the actual filename in the url). Files downloaded automatically by _get_model_weights are still downloaded directly to the cache directory, the hash sub-dirs are only created when a custom url is specified.

[Lilferrit]

A few small comments.

Two key things are still missing as well:

1. Documentation of this functionality on readthedocs (maybe on the file formats page).
2. Unit tests for the download behavior.

In regards to the readthedocs documentation it looks like weight files aren't mentioned at all in the "Input Files" section on the file formats page. Is this something we want to add?

[bittremieux]

Yes, this new (and also old) behavior of weights file downloading should still be added to the documentation.

[bittremieux]

It's starting to look quite nice for a complex piece of code. A few more comments.

[bittremieux]

todo: I'm reluctant to include a library with rather different functionality for just a utility function (even though it's part of the standard libraries). Do we need extra functionality that strptime doesn't offer?

[Lilferrit]

Good call, I changed it to use strptime instead.

[bittremieux]

todo: I don't think this is necessary (rather the relevant import is urllib.parse a few lines further down).

[bittremieux]

todo: Add a similar logging statement.

[bittremieux]

typo: "Casanovo"

[bittremieux]

Very nice implementation and great work on the unit tests. 👏