Merge pull request #42 from ENSTA-U2IS/dev

Add MIMO, NotMNIST, improve coverage, and Misc

Author: alafage

.coveragerc

@@ -1,7 +1,7 @@
 [run]
 branch = True
 include = */torch-uncertainty/*
-omit = *tests*, */datasets/*, setup.py
+omit = */tests/*, */datasets/*, setup.py
 
 [report]
 exclude_lines =

.github/workflows/run-tests.yml

@@ -5,7 +5,7 @@ on:
     branches:
       - main
       - dev
-  pull_request_target:
+  pull_request:
   schedule:
     - cron: "42 7 * * 0"
   workflow_dispatch:
@@ -76,7 +76,7 @@ jobs:
 
     - name: Upload coverage to Codecov
       uses: codecov/codecov-action@v3
-      if: ${{ github.event_name != 'pull_request_target' }}
+      if: ${{ github.event_name != 'pull_request' }}
       continue-on-error: true
       with:
         token: ${{ secrets.CODECOV_TOKEN }}

CONTRIBUTING.md

@@ -5,7 +5,7 @@ contributors to help us build a comprehensive library for uncertainty
 quantification in PyTorch.
 
 We are particularly open to any comment that you would have on this project.
-In particular, we are open to changing these guidelines as the project evolves.
+Specifically, we are open to changing these guidelines as the project evolves.
 
 ## The scope of TorchUncertainty
 
@@ -21,7 +21,7 @@ Monte Carlo dropout, ensemble methods, etc.
 
 If you are interested in contributing to torch_uncertainty, we first advise you
 to follow the following steps to reproduce a clean development environment
-ensuring continuous integration does not break.
+ensuring that continuous integration does not break.
 
 1. Install poetry on your workstation.
 2. Clone the repository.
@@ -37,21 +37,21 @@ poetry install --with dev
 pre-commit install
 ```
 
-We are using black for code formatting, flake8 for linting, and isort for the
-imports. The pre-commit hooks will ensure that your code is properly formatted
+We are using `black` for code formatting, `flake8` for linting, and `isort` for the
+imports. The `pre-commit` hooks will ensure that your code is properly formatted
 and linted before committing.
 
 Before submitting a final pull request, that we will review, please try your
-best not to reduce the code coverage and do document your code.
+best not to reduce the code coverage and document your code.
 
 If you implement a method, please add a reference to the corresponding paper in the ["References" page](https://torch-uncertainty.github.io/references.html).
 
 ### Post-processing methods
 
 For now, we intend to follow scikit-learn style API for post-processing
-methods (except that we use a validation dataloader for now). You can get
+methods (except that we use a validation dataset for now). You can get
 inspiration from the already implemented
-[temperature-scaling](https://github.com/ENSTA-U2IS/torch-uncertainty/blob/dev/torch_uncertainty/post_processing/temperature_scaler.py).
+[temperature-scaling](https://github.com/ENSTA-U2IS/torch-uncertainty/blob/dev/torch_uncertainty/post_processing/calibration/temperature_scaler.py).
 
 ## License
 

README.md

@@ -9,16 +9,16 @@
 [![Code style: black](https://img.shields.io/badge/code%20style-black-black.svg)](https://github.com/psf/black)
 </div>
 
-_TorchUncertainty_ is a package designed to help you leverage uncertainty quantification techniques and make your neural networks more reliable. It aims at being collaborative and including as many methods as possible, so reach out to add yours!
+_TorchUncertainty_ is a package designed to help you leverage uncertainty quantification techniques and make your deep neural networks more reliable. It aims at being collaborative and including as many methods as possible, so reach out to add yours!
 
-:construction: _TorchUncertainty_ is in early development :construction: - expect massive changes but reach out and contribute if you are interested by the project!
+:construction: _TorchUncertainty_ is in early development :construction: - expect massive changes, but reach out and contribute if you are interested in the project! **Please raise an issue if you have any bugs or difficulties.**
 
 ---
 
 This package provides a multi-level API, including:
 
 - ready-to-train baselines on research datasets, such as ImageNet and CIFAR
-- baselines available for training on your datasets
+- deep learning baselines available for training on your datasets
 - [pretrained weights](https://huggingface.co/torch-uncertainty) for these baselines on ImageNet and CIFAR (work in progress 🚧).
 - layers available for use in your networks
 - scikit-learn style post-processing methods such as Temperature Scaling
@@ -27,12 +27,14 @@ See the [Reference page](https://torch-uncertainty.github.io/references.html) or
 
 ## Installation
 
-Install the desired pytorch version in your environment. Then, the package can be installed from PyPI:
+The package can be installed from PyPI:
 
 ```sh
 pip install torch-uncertainty
 ```
 
+Then, install the desired PyTorch version in your environment.
+
 If you aim to contribute (thank you!), have a look at the [contribution page](https://torch-uncertainty.github.io/contributing.html).
 
 ## Getting Started and Documentation
@@ -45,17 +47,18 @@ A quickstart is available at [torch-uncertainty.github.io/quickstart](https://to
 
 ### Baselines
 
-To date, the following baselines are implemented:
+To date, the following deep learning baselines have been implemented:
 
 - Deep Ensembles
 - BatchEnsemble
 - Masksembles
+- MIMO
 - Packed-Ensembles (see [blog post](https://medium.com/@adrien.lafage/make-your-neural-networks-more-reliable-with-packed-ensembles-7ad0b737a873))
 - Bayesian Neural Networks
 
 ### Post-processing methods
 
-To date, the following post-processing methods are implemented:
+To date, the following post-processing methods have been implemented:
 
 - Temperature, Vector, & Matrix scaling
 

auto_tutorials_source/tutorial_bayesian.py

@@ -1,5 +1,4 @@
 # -*- coding: utf-8 -*-
-# fmt: off
 # flake: noqa
 """
 Train a Bayesian Neural Network in Three Minutes
@@ -41,7 +40,7 @@
 from torch_uncertainty.routines.classification import ClassificationSingle
 
 # %%
-# We will also need to define an optimizer using torch.optim as well as the 
+# We will also need to define an optimizer using torch.optim as well as the
 # neural network utils withing torch.nn, as well as the partial util to provide
 # the modified default arguments for the ELBO loss.
 #
@@ -61,13 +60,15 @@
 # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 # We will use the Adam optimizer with the default learning rate of 0.001.
 
+
 def optim_lenet(model: nn.Module) -> dict:
     optimizer = optim.Adam(
         model.parameters(),
         lr=1e-3,
     )
     return {"optimizer": optimizer}
 
+
 # %%
 # 3. Creating the necessary variables
 # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -76,15 +77,18 @@ def optim_lenet(model: nn.Module) -> dict:
 # logs, and to fake-parse the arguments needed for using the PyTorch Lightning
 # Trainer. We also create the datamodule that handles the MNIST dataset,
 # dataloaders and transforms. Finally, we create the model using the
-# blueprint from torch_uncertainty.models. 
+# blueprint from torch_uncertainty.models.
 
 root = Path(os.path.abspath(""))
 
-with ArgvContext("--max_epochs 1"):
+# We mock the arguments for the trainer
+with ArgvContext(
+    "file.py",
+    "--max_epochs 1",
+    "--enable_progress_bar=False",
+    "--verbose=False",
+):
     args = init_args(datamodule=MNISTDataModule)
-    args.enable_progress_bar = False
-    args.verbose = False
-    args.max_epochs = 1
 
 net_name = "bayesian-lenet-mnist"
 
@@ -156,21 +160,20 @@ def imshow(img):
     plt.imshow(np.transpose(npimg, (1, 2, 0)))
     plt.show()
 
+
 dataiter = iter(dm.val_dataloader())
 images, labels = next(dataiter)
 
 # print images
 imshow(torchvision.utils.make_grid(images[:4, ...]))
-print('Ground truth: ', ' '.join(f'{labels[j]}' for j in range(4)))
+print("Ground truth: ", " ".join(f"{labels[j]}" for j in range(4)))
 
 logits = model(images)
 probs = torch.nn.functional.softmax(logits, dim=-1)
 
 _, predicted = torch.max(probs, 1)
 
-print(
-    'Predicted digits: ', ' '.join(f'{predicted[j]}' for j in range(4))
-)
+print("Predicted digits: ", " ".join(f"{predicted[j]}" for j in range(4)))
 
 # %%
 # References