updating requirements, adding pre-commit, and formatting code with black

Author: rafaelgreca

.pre-commit-config.yaml

@@ -1,20 +1,31 @@
---- 
 repos:
 
--
-  repo: https://github.com/ambv/black
-  rev: 20.8b1
-  hooks: 
-    - 
-      id: black
-      language_version: python3
+  -   repo: https://github.com/pre-commit/pre-commit-hooks
+      rev: v2.3.0
+      hooks:
+      -   id: check-yaml
+      -   id: end-of-file-fixer
+      -   id: trailing-whitespace
+      -   id: check-added-large-files
+      -   id: debug-statements
+          language_version: python3
 
--   repo: local
-    hooks:
-    -   id: python-tests
-        name: pytests
-        entry: pytest src/tests
-        language: python
-        additional_dependencies: [pre-commit, pytest, pandas, sklearn, matplotlib]
-        always_run: true
-        pass_filenames: false
+  -   repo: https://github.com/psf/black
+      rev: 22.10.0
+      hooks:
+      -   id: black
+          args: [--safe]
+
+  -   repo: local
+      hooks:
+      -   id: pylint
+          name: pylint
+          files: .
+          entry: pylint
+          language: system
+          types: [python3]
+          args: [
+            "-rn", # Only display messages
+            "-sn", # Don't display the score
+            "--rcfile=.pylintrc", # Link to your config file
+          ]

data/README.md

@@ -12,4 +12,4 @@ Finally, you can download the dataset using the following command:
 bash download_data.sh
 ```
 
-The dataset will be temporarily saved locally (inside the `data` folder) and transferred to your AWS S3 bucket. After that, the dataset will be deleted. If you choose to not use an AWS S3 Bucket, then the dataset will be stored into the `data` folder.
+The dataset will be temporarily saved locally (inside the `data` folder) and transferred to your AWS S3 bucket. After that, the dataset will be deleted. If you choose to not use an AWS S3 Bucket, then the dataset will be stored into the `data` folder.

data/download_data.sh

@@ -39,4 +39,3 @@ if [[ "$CONFIG_S3" != "YOUR_S3_BUCKET_URL" ]]; then
 
     # deleting the create folder
     rm Original_ObesityDataSet.csv
-    

notebooks/README.md

@@ -4,7 +4,7 @@ Here go the notebooks used for research and development. The main idea is to try
 
 ## Setup Credentials
 
-If you haven't your credentials yet, please check the `docs` folder first before following along. 
+If you haven't your credentials yet, please check the `docs` folder first before following along.
 
 1. Set your `AWS Credentials` and `Kaggle API Credentials` (used to download the dataset) in the `credentials.yaml` file.
 
@@ -44,4 +44,4 @@ sudo docker log <CONTAINER_ID>
 - Run the `EDA` notebook.
 - Run the `Data Processing` notebook.
 - Run the `Experimentations` notebook (will test different Machine Learning models, different hyperparameters for each model, and do some feature engineering and selection).
-- Register the best models to the MLflow model registry using the `Experimentations` notebook (last cell) or the MLflow's user interface.
+- Register the best models to the MLflow model registry using the `Experimentations` notebook (last cell) or the MLflow's user interface.

notebooks/VERSION

@@ -1 +1 @@
-1.1.0
+1.3.0

notebooks/dev_Dockerfile

@@ -17,4 +17,4 @@ WORKDIR /e2e-project
 RUN pip install --no-cache-dir -U pip
 
 # installing requirements
-RUN pip install -r notebooks/requirements_dev.txt
+RUN pip install -r notebooks/requirements_dev.txt

notebooks/docs/SETUP_AWS.md

@@ -196,7 +196,7 @@ aws ec2 authorize-security-group-ingress \
     --group-id "sg-0613261580cd87115" \
     --protocol tcp \
     --port 5000 \
-    --cidr "0.0.0.0/0" 
+    --cidr "0.0.0.0/0"
 ```
 
 The output should look like this:
@@ -224,7 +224,7 @@ aws ec2 authorize-security-group-ingress \
     --group-id "sg-0613261580cd87115" \
     --protocol tcp \
     --port 22 \
-    --cidr "18.206.107.24/29" 
+    --cidr "18.206.107.24/29"
 ```
 
 The output should look like this:
@@ -579,4 +579,4 @@ pipenv install mlflow boto3 psycopg2-binary awscli
 pipenv shell
 
 aws configure
-```
+```

notebooks/docs/SETUP_KAGGLE.md

@@ -1,3 +1,3 @@
 # Setting up Kaggle's Account
 
-To use the Kaggle API, sign up for a Kaggle account at https://www.kaggle.com. Then go to the 'Account' tab of your user profile (https://www.kaggle.com/<username>/account) and select 'Create API Token'. This will trigger the download of kaggle.json, a file containing your API credentials. Set your `Kaggle API Credentials` (used to download the dataset) in the `credentials.yaml` file.
+To use the Kaggle API, sign up for a Kaggle account at https://www.kaggle.com. Then go to the 'Account' tab of your user profile (https://www.kaggle.com/<username>/account) and select 'Create API Token'. This will trigger the download of kaggle.json, a file containing your API credentials. Set your `Kaggle API Credentials` (used to download the dataset) in the `credentials.yaml` file.

notebooks/requirements_dev.txt

@@ -12,4 +12,4 @@ optuna==3.6.1
 pandas==1.5.2
 scikit_learn==1.3.2
 seaborn==0.13.2
-xgboost==2.1.1
+xgboost==2.1.1

requirements.txt

@@ -1,11 +1,12 @@
-scikit-learn>=0.23
-pandas
-seaborn
-matplotlib
-joblib
-numpy
-ibm_watson_machine_learning
-pyyaml
-pytest
-pytest-dependency
-pre-commit
+boto3==1.35.6
+fastapi==0.115.5
+joblib==1.3.2
+loguru==0.7.2
+mlflow==2.17.2
+numpy==2.1.3
+pandas==1.5.2
+pydantic==2.9.2
+pytest==8.3.3
+PyYAML==6.0.2
+scikit_learn==1.3.2
+xgboost==2.1.2